April 19, 2014, 08:22:22 AM

Author Topic: What external 'scratch' disk setup do you use/recommend?  (Read 3266 times)

cayenne

  • 1D X
  • *******
  • Posts: 1199
    • View Profile
What external 'scratch' disk setup do you use/recommend?
« on: April 24, 2013, 02:38:10 PM »
Hi All,

My set up:
macbook pro (late 2011 model, 16GB ram), Dell U2711 monitor.

Now, as you can imagine, shooting all RAW, making multiple versions of some images in Post...and doing lots of video, editing it...versions after color grading, etc.

Well, I'm eating LOTS of disk space...that's not even keeping in mind, the software I've installed on said computer (aperture, fcpx, plugins, VMware, GIMP...etc, etc).

I'm having a terrible time keeping disk space open. As soon as I finish a project, I just move all the files off to an external single drive I plug into a USB connected drive docking station.

I'm thinking that I need some sort of fairly speedy, maybe dual disk NAS type thing....?  Maybe do it RAID 1 (mirrored) there.

I'm trying to fix up a separate computer to build a freeNAS system for large backups, haven't gotten that up and running yet, but I think for right now...I just need a large, fast disk storage system for holding and working on my current few projects.

Does anyone have a suggestion for that?

My macbookpro, has one thunderbolt connector on it...I'd think that would be the way to go, however, I'm using that with a thunderbolt to displayport cable/adapter...to run the Dell U2711 monitor, and I don't think there is a way to chain off that...?

So, likely firewire or sadly, USB2 is about my only option for connecting....

Anyway, does anyone have a suggestion on what I might look towards?

Thanks in advance!!

cayenne

canon rumors FORUM

What external 'scratch' disk setup do you use/recommend?
« on: April 24, 2013, 02:38:10 PM »

CR Backup Admin

  • Administrator
  • 1D Mark IV
  • *****
  • Posts: 750
    • View Profile
Re: What external 'scratch' disk setup do you use/recommend?
« Reply #1 on: April 24, 2013, 02:56:05 PM »
When you say "scratch disk", I think of the setting for scratch disk in photoshop.  It needs to be very fast and if you use a external disk, USB and Firewire are painfully slow.

You might be better off getting a large fast SSD, a 960GB Crucial SSD costs $600.

http://www.crucial.com/store/listmodule/SSD/~983040~~M500~/list.html

cayenne

  • 1D X
  • *******
  • Posts: 1199
    • View Profile
Re: What external 'scratch' disk setup do you use/recommend?
« Reply #2 on: April 24, 2013, 04:42:01 PM »
When you say "scratch disk", I think of the setting for scratch disk in photoshop.  It needs to be very fast and if you use a external disk, USB and Firewire are painfully slow.

You might be better off getting a large fast SSD, a 960GB Crucial SSD costs $600.

http://www.crucial.com/store/listmodule/SSD/~983040~~M500~/list.html


Perhaps I'm using the wrong terminology...

Basically the situation is, the hardisk in my macbookpro (750GB?) is getting full. I only have about 35-60GB free at any given time here lately.

I want to have my RAW photo projects and Video projects I'm working on...on some sort of external drive setup, that would be fast enough to keep up with working on them....like with FCPX editing/rendering...with Davinci Resolve Lite...etc.

But..sounds like you're saying that since I likely only have USB and firewire on my computer, they won't be fast enough?

Hmm...wonder if there is any possible way I could hook something externally on the one thunderbolt connector, and then, chain off that storage unit...to a displayport adapter for my monitor?

I just gotta do something, as that my internal harddrive is just too full and most of it seems to be from my installed applications and iTunes maybe. I think it is frankly mostly installed apps.

I need to set up somehow externally for photo and video work...

C

Mt Spokane Photography

  • 1D X
  • *******
  • Posts: 7711
    • View Profile
Re: What external 'scratch' disk setup do you use/recommend?
« Reply #3 on: April 24, 2013, 06:16:28 PM »
I wonder if something like this would work??  Its a thunderbolt hub with a extra port for daisy chaining and two esata ports for external hard drives.  The question is if it will work with a display.
 
http://www.newegg.com/Product/Product.aspx?Item=N82E16822154646&Tpk=thunderbolt%20hub&IsVirtualParent=1

Grumbaki

  • EOS M2
  • ****
  • Posts: 266
    • View Profile
Re: What external 'scratch' disk setup do you use/recommend?
« Reply #4 on: April 24, 2013, 10:58:49 PM »
If you are really blowing through disk space, I d recommand a full on NAS that can run raid 5 or 10 depending on your budget and your space needs.

I have an older version of that
http://www.synology.com/products/product.php?product_name=DS412%2B&lang=us

Not cheap but excellent investment all around. mine runs 3Tb of useable space in raid 5 (3/4rth of your phisical space if all disks are of equal size), one disk can go to hell you don't lose anything. Ethernet is used so speed isn't an issue. I just see slow down when my router gets prompts from too many devices (3 or 4 out of the 6 we have at home). I'm stuck with a crappy one due to my internet provider (yay china) using weird ass protocols and the absence of competent english speaking techie so right now I'd say router might be the weak link of the set up.

The day it's affordable i get the NAS to run on SSD as the HDD are the next weak point.

(and the day i own my flat and not rent it I get in walls fiber :D )

« Last Edit: April 24, 2013, 11:04:57 PM by Grumbaki »

scottkinfw

  • 6D
  • *****
  • Posts: 623
    • View Profile
    • kasden.smug.com
Re: What external 'scratch' disk setup do you use/recommend?
« Reply #5 on: April 24, 2013, 11:30:46 PM »
I have a great suggestion for you to look into.  I wouldn't suggest RAID 1 either because what you gain in speed, you lose in protection.  Why back up only to lose it all- right?

Look at Drobo.  a tad expensive but fast, protected, hot swappable, gives you notice when a drive will fail, if a drive fails, all data is saved on other disks.  When you fill up the drives, replace with bigger ones.  If you max out Drobo (possible, but with capacious drives, hard to imagine at this point) you can daisy chain them.  There are a good number to choose from. 

I have been using mine a couple of years and love it.  It is easy to set up.  Additionally, you can start with two drives and add more when you get more $$.  It is a lot easier than the setup you describe (I have tried NAS before and they can be a pain to set up and keep up).  I am on Mac's and no problem.

I would finally suggest that you keep your Macbook clean and simple.  Just keep your programs on there and keep you pics etc. on the Drobo.  With fast drives in the Drobo, you shouldn't have a problem doing intensive editing.

Hope this helps.

sek

Hi All,

My set up:
macbook pro (late 2011 model, 16GB ram), Dell U2711 monitor.

Now, as you can imagine, shooting all RAW, making multiple versions of some images in Post...and doing lots of video, editing it...versions after color grading, etc.

Well, I'm eating LOTS of disk space...that's not even keeping in mind, the software I've installed on said computer (aperture, fcpx, plugins, VMware, GIMP...etc, etc).

I'm having a terrible time keeping disk space open. As soon as I finish a project, I just move all the files off to an external single drive I plug into a USB connected drive docking station.

I'm thinking that I need some sort of fairly speedy, maybe dual disk NAS type thing....?  Maybe do it RAID 1 (mirrored) there.

I'm trying to fix up a separate computer to build a freeNAS system for large backups, haven't gotten that up and running yet, but I think for right now...I just need a large, fast disk storage system for holding and working on my current few projects.

Does anyone have a suggestion for that?

My macbookpro, has one thunderbolt connector on it...I'd think that would be the way to go, however, I'm using that with a thunderbolt to displayport cable/adapter...to run the Dell U2711 monitor, and I don't think there is a way to chain off that...?

So, likely firewire or sadly, USB2 is about my only option for connecting....

Anyway, does anyone have a suggestion on what I might look towards?

Thanks in advance!!

cayenne
sek
Cameras: 5D III, 5D II, 50D Lenses:  24-70 208 II, 24-105 f4L, 70-200 f4L IS, 70-200 f2.8L IS II, EF 300 f4L IS, EF 400 5.6L, 600EX-RT, ST-E3-RT

Niterider

  • Rebel SL1
  • ***
  • Posts: 88
    • View Profile
Re: What external 'scratch' disk setup do you use/recommend?
« Reply #6 on: April 24, 2013, 11:48:30 PM »
If you dont use the cd/dvd drive too often, I have heard that you can remove it and put a ssd in its place. Because that will be a sata connection, it will be the fastest.

I have a server with windows server 2012 on it, but I wouldn't be caught dead using one of it's disks as a scratch disk. Way too slow. NAS would be even slower.

canon rumors FORUM

Re: What external 'scratch' disk setup do you use/recommend?
« Reply #6 on: April 24, 2013, 11:48:30 PM »


dirtcastle

  • 7D
  • *****
  • Posts: 391
    • View Profile
    • Eric Nord Flickr Page
Re: What external 'scratch' disk setup do you use/recommend?
« Reply #8 on: April 25, 2013, 01:59:51 AM »
My macbookpro, has one thunderbolt connector on it...I'd think that would be the way to go, however, I'm using that with a thunderbolt to displayport cable/adapter...to run the Dell U2711 monitor, and I don't think there is a way to chain off that...?

New Apple displays have an extra T-bolt on the back. But obviously that doesn't help in your case. :-(

I wonder if you could connect your monitor through the FireWire (?), which would open up the Thunderbolt port.

Or you could get a FireWire 7200rpm drive and settle for that. I've got a 7200rpm G-Drive and it seems reasonably fast with After Effects, although I have no idea how it really stacks up against thunderbolt and/or SSD.

privatebydesign

  • 1D X
  • *******
  • Posts: 1600
  • Ermintrude says "moo"
    • View Profile
Re: What external 'scratch' disk setup do you use/recommend?
« Reply #9 on: April 25, 2013, 08:26:16 AM »
Any decent TB device can be used as a daisychain, just plug a storage device into the computer TB socket then plug your display into the storage devices second TB socket.

I am running an older MBP 17" with an eSATA card. I have a 256GB SSD in the original HDD space, this is partitioned into two, a 200GB operating system and apps drive, 50+ as a scratch disc and temporary storage place. I then have a 1TB HDD in the place of the CD/DVD tray, this works really well, so well in fact that I got a Mac Mini and did the same thing to it as a "home" desktop.

I have six external HDD's attached to the MBP, two via eSATA, the other four via daisychained Firewire 800. I have no RAID setups. I use Carbon Copy Cloner and Time Machine to cover my backup tasks.

Use a dedicated disc for Time Machine, it is a great program that will save your butt. I use CCC for everything else, including fully bootable backups of both internal HDD's and all external drives copied, apart from the disc I use for TM.

The real problem with DROBOS (which has a very mixed review history) and most other one cable RAID solutions, they use proprietary software to write your info to disc, this means if you have a serious gear failure the only way your info can be read is by sending your HDD's off to the manufacturer and paying them to get all that stuff back off your discs that they encoded!

That is why I use CCC to make simple copies, at a block level, of my HDD's as backups. Just plug any copy into your computer and it works, everything is there, including working app's, preferences, everything.
The best time to plant a tree is twenty-five years ago. The second best time is today.

cayenne

  • 1D X
  • *******
  • Posts: 1199
    • View Profile
Re: What external 'scratch' disk setup do you use/recommend?
« Reply #10 on: April 25, 2013, 11:04:40 AM »
Dang, wish I'd seen that Lacie option earlier....

I found a sale and went with this:

http://www.newegg.com/Product/Product.aspx?Item=N82E16822108095

Just to have something to play with and give me room for now.

I'll try it on the network as a NAS, but if that isn't speedy enough hooked to my router, I was thinking of trying to maybe get a crossover cable, and plug it DIRECTLY into my macbook pro's ethernet port. If both are gigabit...I'd think that would be fast enough to work with?

I'd just use wireless on the mac for network connectivity in place of the wired ethernet port....wondering if that might work?

Anyway, this is a stopgap measure, and I'll be able to use this unit repurposed later....the deal ended today with the code, I got it for $184 which was about as cheap as I'd seen around anywhere.

I should get it in on Sat and will play with it. Again, if it doesn't work fast enough, I can use it on the network for other things.

Thanks for the insight so far....I'm still researching a better end solution...

cayenne

cayenne

  • 1D X
  • *******
  • Posts: 1199
    • View Profile
Re: What external 'scratch' disk setup do you use/recommend?
« Reply #11 on: April 25, 2013, 11:06:57 AM »
I have a great suggestion for you to look into.  I wouldn't suggest RAID 1 either because what you gain in speed, you lose in protection.  Why back up only to lose it all- right?



sek



I'm a little confused.....maybe I need to review my knowledge of RAID levels.

I thought RAID 1 was mirroring.....so, if I lose one drive, I still have the mirrored one there to rebuild with?

cayenne

Mt Spokane Photography

  • 1D X
  • *******
  • Posts: 7711
    • View Profile
Re: What external 'scratch' disk setup do you use/recommend?
« Reply #12 on: April 25, 2013, 11:07:08 AM »
I have a 12TB NAS but its not the way to go for editing directly from it.  Even with two 1 GB network cables in parallel, its slow for editing.   I won't even edit still directly from it unless its urgent.
Its great for storing files and backups, which is what I use it for.

canon rumors FORUM

Re: What external 'scratch' disk setup do you use/recommend?
« Reply #12 on: April 25, 2013, 11:07:08 AM »

ahab1372

  • EOS M2
  • ****
  • Posts: 304
    • View Profile
Re: What external 'scratch' disk setup do you use/recommend?
« Reply #13 on: April 25, 2013, 11:11:44 AM »
NAS is great to store files. I can also recommend Synology.

For the you are currently working on (especially video projects), I'd consider an external drive attached with Firewire 800, USB 3 or Thunderbolt (not for the 2011 Macbook though). The network connection of a NAS, even over a Gigabit network, could still be a bottleneck, depending on the networking performance of the NAS and/or your computer.
Use the internal drive as the Scratch disk, and keep the video files on the external drive until your done. Use the NAS for long term storage and backup.

cayenne

  • 1D X
  • *******
  • Posts: 1199
    • View Profile
Re: What external 'scratch' disk setup do you use/recommend?
« Reply #14 on: April 25, 2013, 01:41:49 PM »
I have a 12TB NAS but its not the way to go for editing directly from it.  Even with two 1 GB network cables in parallel, its slow for editing.   I won't even edit still directly from it unless its urgent.
Its great for storing files and backups, which is what I use it for.
What do you recommend for working from them, that is external to the computer (in my case a laptop)?

TIA,

cayenne

canon rumors FORUM

Re: What external 'scratch' disk setup do you use/recommend?
« Reply #14 on: April 25, 2013, 01:41:49 PM »