December 17, 2014, 08:16:49 PM

Author Topic: Which Storage Devices ???  (Read 10247 times)

pdirestajr

  • 1D Mark IV
  • ******
  • Posts: 774
    • View Profile
    • flickr
Re: Which Storage Devices ???
« Reply #15 on: May 18, 2011, 06:20:26 PM »
I recently got so freaked out about how in this digital era there aren't really any tangible records anymore- so I went out and bought a used EOS-3 to shoot some slides! Also a cheap way ($200) to get a full frame camera (I'm shooting on a 7D). I'm also getting more into printing those special images.
7D | 5DII | EOS-3 | Nikon F3 | Mamiya 645 Pro-TL

canon rumors FORUM

Re: Which Storage Devices ???
« Reply #15 on: May 18, 2011, 06:20:26 PM »

Mike Ca

  • Guest
Re: Which Storage Devices ???
« Reply #16 on: May 19, 2011, 04:13:45 AM »

I haven't tried it yet but has anyone considered an online backup service such as Carbonite?

I have considered Carbonite and other online backup services, but when you get into the details, way too expensive.

Carbonite has $59 per year per PC with unlimited data storage, but only for the internal hard drive of your PC or Mac. I have all my photo data on an external (usb) raid 1 unit. So for external drive backup, you have to buy the small business package. For the small bus package, if you store 300-500GB it costs $250 a month!!!

I've found most of the online backup companies don't support external hard drives in their low price packages, and they get petty expensive for the amount of data photographers shooting RAW can generate.

Definitely need to do a better job of deleting my non-keepers.

leGreve

  • EOS M2
  • ****
  • Posts: 217
  • Full time photographer and film maker omnifilm.dk
    • View Profile
    • leGreve Photography
Re: Which Storage Devices ???
« Reply #17 on: May 19, 2011, 04:19:31 AM »
You're going to want to get he Lacie Little Big with Thunderbolt once it arrives in 2 months or so... anything else would be pointless and outdated.

I bought the 15" mbp, removed the stock HD, installed a new one, then I took the stock drive and put it in an IcyBox and wupti a free external drive. It will serve me decently till the Lacie arrives :D
5D III -  Zeiss Otus 55mm 1.4 ~ 24-70 2.8L II ~ 70-200 2.8L II IS USM ~ 100 2.8L IS USM Macro ~ 16-35 2.8L II ~ Canon Extender 2x III

dstppy

  • 1D Mark IV
  • ******
  • Posts: 958
    • View Profile
Re: Which Storage Devices ???
« Reply #18 on: May 19, 2011, 08:19:05 AM »
Comment:

Off-site, web storage is a good idea with a few exceptions; the biggest of which is 'who do you trust?'*

Honestly, following standard sysadmin style backup is the way to go, and every week/big shoot, you drop all of your UNEDITED stuff on an "FTP"** server.  I would say avoid 'solutions' -- as my boss always says, "I don't buy solutions, I buy hardware and software".  Avoid anything 'image' related, because there is a tendency to compress or alter the images to do you a favor; hell, avoid a web interface.

* Who do you trust?  I like my web host, but I certainly don't love them.  They have had their billing information compromised twice in less than as many years, and I'm now shopping around for an alternative.  Things to consider:
1) Do they patch the servers diligently? Do you have the ability to verify server versions?
2) Is billing and hosting separate? (Make sure you use different credentials)
3) What does their EULA say?
3a) Who owns YOUR stuff (most places have a line that says they do)?
3b) Do they guarantee that what you delete gets deleted?
4) Will they protect you?
4a) Will they turn over your account information willingly to 'the authorities' (don't forget, this includes people CLAIMING to be the authorities or you [social engineering])
4b) How do they handle 'takedown' notices -- if you're doing something web based with this.

**Why did I put FTP in quotes? Because I meant it more as a relationship style than actually endorsing the protocol; there may be faster/more secure ways at this point, I haven't investigated recently.

Question:
For those of you using off-site data storage, who and what do you use?
Canon Rumors is presently creating photographer shortages in Middle Earth (all the trolls emigrated here)

motorhead

  • Canon 70D
  • ****
  • Posts: 267
    • View Profile
Re: Which Storage Devices ???
« Reply #19 on: May 19, 2011, 08:30:36 AM »
Dymonds,

Based on my own experience, locating a file on an external hard drive is only a couple of mouse clicks more than finding the file on the PC's own hard drive. Apart from the time taken to make those two clicks, it's no slower at all, so don't worry about it taking longer. One extra heart beat, maybe two and thats it.

scottkinfw

  • 1D Mark IV
  • ******
  • Posts: 780
    • View Profile
    • kasden.smug.com
Re: Which Storage Devices ???
« Reply #20 on: May 22, 2011, 04:30:51 AM »
For the field, I would recommend Hyperdrive UDMA.  I purchased the unit, a separate hdd, and assembled them (no problem at all). 

The unit is self powered, and accepts common memory cards.  You can buy some other accessories to increase functionality if desired.

I recently took it on safari to Tanzania, and every day downloaded cf & sd cards. It worked like a champ.  It has a color screen to view pics and a histogram, etc.

Here are links:  http://www.hypershop.com/HyperDrive-COLORSPACE-UDMA-Casing-Only-p/hdcsu-000.htm (cheapest way to do it, and it is easy to do)
On the go adapter:  http://www.hypershop.com/COLORSPACE-UDMA-OTG-Sync-Adapter-Hard-Carry-Case-p/hdcsu-acs.htm (a bit confusing documentation, but uesful).

I have no financial ties to the company.

sek
sek Cameras: 5D III, 5D II, EOS M  Lenses:  24-70 2.8 II IS, 24-105 f4L, 70-200 f4L IS, 70-200 f2.8L IS II, EF 300 f4L IS, EF 400 5.6L, 300 2.8 IS II, Samyang 14 mm 2.8 Flashes: 580 EX II600EX-RT X 2, ST-E3-RT
Plus lots of stuff that just didn't work for me

autochrome

  • Guest
Re: Which Storage Devices ???
« Reply #21 on: May 30, 2011, 09:20:20 PM »
Whatever you do, just don't be cheap on HDDs, and backup, and backup often, it will save you from a massive disappointment one day, that's guaranteed. If possible use RAID1, or if you can afford hardware RAID controllers, you need a 4 ports controller to have RAID5 mode, in which one of the hdds is a parity disk, so if any single hdd of the remaining fails you can reconstruct the missing image, but you need an 8 ports controller for a RAID6 mode (with 2 parity disks), which brings you an extra level of redundancy. People spend big amounts of money on cameras, L glass, devote their time to create good images, process them and that's fine, but here's nothing more frustrating than watching IO errors on the hdd and realizing a week, a month or more of work are gone, beyond repair. For the most time, you'll be asking yourself why on Earth are you doing weekly backups, spent a considerable amount of money on a RAID6 array, enclosure, enterprise quality hdds, until one day you'll suddenly find out why, and in that moment all that investment in time and money justifies itself.
This of course is somewhat relative, if the information you store is just a hobby or if it's work, these two situations might warrant different levels of investment.

canon rumors FORUM

Re: Which Storage Devices ???
« Reply #21 on: May 30, 2011, 09:20:20 PM »

Chewy734

  • EOS M2
  • ****
  • Posts: 238
    • View Profile
Re: Which Storage Devices ???
« Reply #22 on: June 07, 2011, 06:30:09 PM »
If photography is your business, get a dedicated RAID server for your backups.  It doesn't have to be hardware RAID-5 or 6, as we use software RAID-5 or 6 for a lot of mission critical stuff.  If photography is your hobby, then RAID-1 should be good enough with 2 large HDs.  It all depends on your requirements (speed, capacity, etc).

gmrza

  • Canon 7D MK II
  • *****
  • Posts: 497
    • View Profile
Re: Which Storage Devices ???
« Reply #23 on: June 07, 2011, 06:45:17 PM »
If photography is your business, get a dedicated RAID server for your backups.  It doesn't have to be hardware RAID-5 or 6, as we use software RAID-5 or 6 for a lot of mission critical stuff.  If photography is your hobby, then RAID-1 should be good enough with 2 large HDs.  It all depends on your requirements (speed, capacity, etc).

There are two points here that a few people have discussed:
1) data protection (RAID is the most common technology to address this); and
2) backups.

The two should not be confused.
There is no one-size-fits all storage solution - we all need to analyse our needs, and build a solution that fits.

What is important to remember is that RAID will protect you from a single hard drive failure (or two hard drive failures with RAID6 / RAID-DP - if you are using Netapp storage).
RAID will not protect you from a NAS or controller failure.  To do that you need to look at solutions like replication (e.g. replicate data between two NASes).

RAID also does not protect against human error - i.e. accidental file deletion, or against operating system failure - e.g. your system crashes due to power loss, and the file system is trashed when you try to boot up again.

Try to consider all the failure modes you need to protect against, and then architect a storage solution that fits your needs.

For instance, RAID is not for everyone.  You may find that individual hard drives are large enough, and that replicating between 2 NASes protects you from drive and controller failure, while backing up to the cloud protects you from finger trouble...
Zeiss Ikon Contax II, Sonnar 50mm f/2, Sonnar 135mm f/4

prestonpalmer

  • Guest
Re: Which Storage Devices ???
« Reply #24 on: June 08, 2011, 09:03:49 AM »
Ignore all these guys. If you want something that works, out of the box with extreme reliability and you can use your existing drives in, purchase a DROBO.  There simply is no better product on the market for professional photographers.  I own 4 of the USB3 DROBO S 5 bay models and have been 100% pleased with performance and reliability.  Fantastic product. Checkout their website and watch some videos and check out the dual disk redundancy options.  I have had drives fail and with a drobo it just isn't an issue.  BTW. I have no financial gain from pitching drobo! I just LOVE their products! :)

prestonpalmer

  • Guest
Re: Which Storage Devices ???
« Reply #25 on: June 08, 2011, 06:55:41 PM »
Get a DROBO

morphers82

  • Guest
Re: Which Storage Devices ???
« Reply #26 on: June 15, 2011, 11:23:53 PM »
There are some great options for storage on the go.  There's a breakdown on this site http://bit.ly/lr3lqw of the different options

autochrome

  • Guest
Re: Which Storage Devices ???
« Reply #27 on: June 16, 2011, 07:11:30 PM »
If photography is your business, get a dedicated RAID server for your backups.  It doesn't have to be hardware RAID-5 or 6, as we use software RAID-5 or 6 for a lot of mission critical stuff.  If photography is your hobby, then RAID-1 should be good enough with 2 large HDs.  It all depends on your requirements (speed, capacity, etc).

There are two points here that a few people have discussed:
1) data protection (RAID is the most common technology to address this); and
2) backups.

The two should not be confused.
There is no one-size-fits all storage solution - we all need to analyse our needs, and build a solution that fits.

What is important to remember is that RAID will protect you from a single hard drive failure (or two hard drive failures with RAID6 / RAID-DP - if you are using Netapp storage).
RAID will not protect you from a NAS or controller failure.  To do that you need to look at solutions like replication (e.g. replicate data between two NASes).

RAID also does not protect against human error - i.e. accidental file deletion, or against operating system failure - e.g. your system crashes due to power loss, and the file system is trashed when you try to boot up again.

Try to consider all the failure modes you need to protect against, and then architect a storage solution that fits your needs.

For instance, RAID is not for everyone.  You may find that individual hard drives are large enough, and that replicating between 2 NASes protects you from drive and controller failure, while backing up to the cloud protects you from finger trouble...

Overall that's the difference between a hobby and work. Clients don't want excuses, so you shell out for the battery backup modules hw RAID card, and you have a good UPS.
A journalling filesystem helps, and specially SGI XFS has a consistently high performance when dealing with series of large size files - it was developed by SGI for the visual effects workstations - all the compositing and post-production work was done on scanned 35mm film, which meant you ended up with imagine (the following is nothing abnormal), frame sequences of 5000 or 10000 16bpc TIFF files, or Kodak Cineon DPX files. Now mostly OpenEXR half-float, but still a very large number of frames of relatively the same size. Then you had to "record" the frame sequences into film again, but the all-digital workflow of today simplified that a lot. You still end up with vast sequences of large files for post-production and visual effects work, since that's the highend applications expect (mostly developed in-house, such as Nuke by the software department of Digital Domain, then marketed and sold, others not marketed at all, such as the compositor used by ILM, but they share an SGI IRIX ancestry, and porting them to Linux was relatively easy (and now some, such as Nuke, exist on OSX and Windows as well).
This just to point out that the choice of the file system is also a factor in the problem at hands. Some other filesystems, Reiser3 are great for very large series of small files, others are generalists, with average performance, such as Ext3 (i won't mention Ext2 since it doesn't has journalling), and there are other filesystems which record "snapshots" of the filesystem state at user-defined periods, i'm thinking mostly about Sun ZFS.
If the battery backup module of the hw RAID card, and UPS fail at the same time (unlikely), you still have the filesystem journal/snapshot, and for the most time you'll be able to savage the data without inconvenience. In some of these filesystems you might even place the filesystem journal/metadata in other hdds, mirrored, extra redundancy.

The backup plan, that's trickier and more expensive. Safe backups imply copies in different physical locations, tapes or hdds in fire proof vaults, and so on. There are companies that provide this type of service for critical data. It just depends what your definition of critical data is and how much you think it is worth. Granted this is overkill for the majority of people, but it's worth mentioning nevertheless.

I agree with your views, just wanted to add those details regarding filesystems.

canon rumors FORUM

Re: Which Storage Devices ???
« Reply #27 on: June 16, 2011, 07:11:30 PM »