October 25, 2014, 11:12:14 PM

Author Topic: Best Methods For Long Term File Storage ??  (Read 17496 times)

1255

  • PowerShot G1 X II
  • ***
  • Posts: 54
    • View Profile
Best Methods For Long Term File Storage ??
« on: November 10, 2012, 06:25:16 PM »
Hi all.

Would like to ask for your thoughts on best methods for long term file storage, meaning, external hard drives or DVD/Blue Ray? Or something else?

I had always thought that external hard drives were best, at the very least because who knows when we'd need to convert those DVD or Blue Rays to a new technology.

Then I just read in another post that external hard drives corrupt files over time. Is this true? I had always thought that it was the process of dragging files and/or copy and pasting them over and over that impacted data bits and degraded the files.

What do you think is the best method for long term file storage?

As always, thanks to all of you for your input.

Much appreciated.

canon rumors FORUM

Best Methods For Long Term File Storage ??
« on: November 10, 2012, 06:25:16 PM »

Drizzt321

  • 1D X
  • *******
  • Posts: 1674
    • View Profile
    • Aaron Baff Photography
Re: Best Methods For Long Term File Storage ??
« Reply #1 on: November 10, 2012, 08:00:02 PM »
Hard drives suffer from bit-rot (as do some other forms of storage). It can occur in a couple of cases, such as improper storage, flooding, etc, or cosmic rays, or other similar high energy events.

There are a couple of ways to mitigate this. First, keep a copy on several different storage devices, preferably of different types. This helps mitigate one of them going bad. If it's all hard drives, get ones with serial numbers which aren't right near each other, which can help mitigate if a particular lot of hard drives has issues (it happens, manufacturing isn't perfect).

Further, you can use a filesystem or storage mechanisms which provide a high level of error correction and detection. You can use something like Par2/Parchive which provides parity and checksuming which can detect, and often correct for bit-rot errors.

A good archiving solution will including multiple storage media, and proper storage of that media. In general, a high quality tape backup with proper storage tends to be the best, very long term archival mechanism and is still very popular and widely used in various enterprise backup/archive solutions.
5D mark 2, 5D mark 3, EF 17-40mm f/4L,  EF 24-105mm f/4L IS USM, EF 135mm f/2L, EF 85mm f/1.8
Film Cameras: Mamiya RB67, RB-50, RB-180-C, RB-90-C, RB-50, Perkeo I folder, Mamiya Six Folder (Pre-WWII model)
http://www.aaronbaff.com

1255

  • PowerShot G1 X II
  • ***
  • Posts: 54
    • View Profile
Re: Best Methods For Long Term File Storage ??
« Reply #2 on: November 13, 2012, 09:46:21 AM »
thanks very much

Standard

  • Guest
Re: Best Methods For Long Term File Storage ??
« Reply #3 on: November 13, 2012, 10:20:09 AM »
I simply archive all my files onto DVDs. If there are important files that I worry about, I make sure to archive them onto several copies for redundancy (I do this over a period of time) so as to avoid possible disc errors. All discs are then catalogued (with a cataloguing app for quick search and retrieval at a later time), then stored onto large 200+ disc binders and placed in metal file drawers. Simple as that. No issues thus far. And yes. I archive an extra copy of select important files onto Lacie 1+ TB external hard drives.

Iahcon

  • SX60 HS
  • **
  • Posts: 7
    • View Profile
Re: Best Methods For Long Term File Storage ??
« Reply #4 on: November 13, 2012, 10:22:34 AM »
I keep two copies on separate external hard drives.
5D II | 1D II | 17-40L | 24-70L | 70-200 2.8 IS | 100-400L | 85 1.8 | nifty fifty | 580 EX II | 580 EX | PW TT5 x 2 | PW TT1

neuroanatomist

  • CR GEEK
  • ********
  • Posts: 14758
    • View Profile
Re: Best Methods For Long Term File Storage ??
« Reply #5 on: November 13, 2012, 10:28:52 AM »
I keep two copies on separate external hard drives.

As do I, and the two HDDs are stored in different physical locations (home and work).  Those copies are made as soon as the images are processed (and until then, they're still on the CF cards - since getting the 1D X, I always have at least 2 copies of every image).  Quarterly, I make a DVD backup that goes in a safe deposit box in a third location (different town than home or work).
EOS 1D X, EOS M, and lots of lenses
______________________________
Flickr | TDP Profile/Gear List

florianbieler.de

  • EOS M2
  • ****
  • Posts: 192
    • View Profile
    • florianbieler.de photography
Re: Best Methods For Long Term File Storage ??
« Reply #6 on: November 13, 2012, 10:57:20 AM »
Got mine synced to a 2,5" external hdd which i use not that often.
EOS 5D Mark III · EF 16-35 4.0L IS · Σ 35 1.4 · Σ 85 1.4 · EF 70-200 4.0L IS · T 150-600 VC
EOS M · EF-M 22 2.0 · florianbieler.de

canon rumors FORUM

Re: Best Methods For Long Term File Storage ??
« Reply #6 on: November 13, 2012, 10:57:20 AM »

Crapking

  • Canon 6D
  • *****
  • Posts: 350
  • "Whatever you are....be a good one." AL
    • View Profile
    • Crapking Photos
Re: Best Methods For Long Term File Storage ??
« Reply #7 on: November 13, 2012, 11:39:42 AM »
What about on-line "cloud" storage sites which allow access without physically requiring the disk drive? 
Of course this is in addition to the physical backups, but I use Phanfare.com {with a reasonable annual fee}, which allows web-based access, including RAW storage (for an extra fee) for the really important files.
1Dx, 1DIV, 5D3, 7D, (Sigma 15 FE)
16-35/2.8; 24-70/2.8 II; 70-200/2.8 II, 100-400L
35/1.4, 40/2.8; 50/1.2, 85/1.2, 135/2; 200/2

myocyte

  • Power Shot G7X
  • **
  • Posts: 20
    • View Profile
Re: Best Methods For Long Term File Storage ??
« Reply #8 on: November 13, 2012, 12:29:31 PM »
I've been meaning to sign up for this, but Amazon provides a great, affordable solution for cloud storage. If anyone has tried it, I'd be interested to hear about their experience.

http://aws.amazon.com/glacier/
5DIII | 24-70 f/2.8L II | 50 f/1.2L | 85 f/1.2L | 70-200 f/2.8L II

sb

  • Guest
Re: Best Methods For Long Term File Storage ??
« Reply #9 on: November 13, 2012, 01:08:03 PM »
I keep two copies on separate external hard drives.

As do I, and the two HDDs are stored in different physical locations (home and work).  Those copies are made as soon as the images are processed...

Same here with a small difference - I make a copy immediately after dumping the files on the primary computer. Processing is all in LR metadata anyway, and LR catalogues are also included in my backup script.

My backup script is a hand made batch file using the "Robocopy" command line utility.

My back up drives reside on a second computer which is identical with all the same applications installed as the primary. That way if my primary fails, I can continue working uninterrupted.

I don't use any optical media (DVD/Bluray) because they become unreadable after a few years. I also don't use external drives because for the price of a "network storage" I can build myself a fully functional second computer, so what's the point?

neuroanatomist

  • CR GEEK
  • ********
  • Posts: 14758
    • View Profile
Re: Best Methods For Long Term File Storage ??
« Reply #10 on: November 13, 2012, 01:19:46 PM »
I don't use any optical media (DVD/Bluray) because they become unreadable after a few years.

How many is 'a few'?  I recently listened to The Cars Greatest Hits, a CD that I bought in 1985.   :o
EOS 1D X, EOS M, and lots of lenses
______________________________
Flickr | TDP Profile/Gear List

Drizzt321

  • 1D X
  • *******
  • Posts: 1674
    • View Profile
    • Aaron Baff Photography
Re: Best Methods For Long Term File Storage ??
« Reply #11 on: November 13, 2012, 02:13:38 PM »
I don't use any optical media (DVD/Bluray) because they become unreadable after a few years.

How many is 'a few'?  I recently listened to The Cars Greatest Hits, a CD that I bought in 1985.   :o

That's also a pressed CD, vs a burned CD/DVD. The material in a burnable CD/DVD is, generally, at least partially organic and over time degrades. Some of them are 'archival' quality, but I believe that's generally only guaranteed for only 40-50 years. Not sure, I'd have to look that number up, but I know it's not forever.
5D mark 2, 5D mark 3, EF 17-40mm f/4L,  EF 24-105mm f/4L IS USM, EF 135mm f/2L, EF 85mm f/1.8
Film Cameras: Mamiya RB67, RB-50, RB-180-C, RB-90-C, RB-50, Perkeo I folder, Mamiya Six Folder (Pre-WWII model)
http://www.aaronbaff.com

Emeyerphoto

  • Power Shot G7X
  • **
  • Posts: 26
    • View Profile
    • Erik Meyer Photography
Re: Best Methods For Long Term File Storage ??
« Reply #12 on: November 13, 2012, 02:29:50 PM »
1. Dump on to external as soon as get home.
2. Modified images get saved on secondary external.
3. Backblaze backs them up onto the cloud, can recover them anywhere, and if necessary I can receive an external hard drive with my files on them for a fee.
4. Relax, since they are on the cloud and do not need to mess with safety deposit boxes, nuclear proof storage, or 10 external hard drives of the same data.
5d Mk2, Canon 40d (backup), 24-70 f/2.8L, 70-200 f/4L

canon rumors FORUM

Re: Best Methods For Long Term File Storage ??
« Reply #12 on: November 13, 2012, 02:29:50 PM »

cayenne

  • 1D X
  • *******
  • Posts: 1213
    • View Profile
Re: Best Methods For Long Term File Storage ??
« Reply #13 on: November 13, 2012, 02:39:54 PM »
Lots of good posts here.
ONE thing that should stand out...redundancy, and multiple backup places and methods.

CDs/DVDs/BRs...all can degrade, and depend also on the quality of your starting materials...cheap will usually rot quickly.

I'm looking to build a fairly large, redundant NAS (Network Accessible Storage) system. I'm looking to use the freeNAS system:

http://www.freenas.org/features/feature-overview/category/features

Wikipedia:

http://en.wikipedia.org/wiki/FreeNAS

What makes this nice...is using the ZFS filesystem, which allows you to add different sized / speed drives over time...and can be hot swappable....and can lose 2 (or more depending on set up I believe) drives at the same time in the system, and still not lose data.

I just bought a special at newegg the other day, a barebones computer I'm about to assemble for about $168...8GB ram (which I'll likely upgrade to 16 for about $50 or so)...

Anyway, set that up with a 5 disk enclosure I bought awhile back to start with, and build it from there.  In the future to have the comfort of redundancy offiste , in case of fire, flood, hurricane (I live in NOLA after all), I'm going to set up a similar system at my mom's house out of state...and that way, I can justify paying for her internet connectivity since it will be a 'business expense'.

Anyway, I'm getting off topic...but if you don't mind a little DIY spirit....take a look at the links and systems I've listed above. It is very high quality stuff, and can be turned into a highly reliable, flexible system that most any computer you own can connect to and backup, whether it be a Windows PC, or Linux box...or even OSX on a mac with time machine, which can often be finicky at times on its own right.

HTH,

cayenne

cliffwang

  • Canon 7D MK II
  • *****
  • Posts: 491
    • View Profile
Re: Best Methods For Long Term File Storage ??
« Reply #14 on: November 13, 2012, 02:46:20 PM »
Always keep your files duplicated.  Any kind of media is fine.  However, you have to ensure the files you backed up is always good.  DVD is not my suggest.  Nowadays external drive is very cheap, so you can get 3TB or bigger external drive to backup your files.  Do not waste your money to buy too big external drive because hard drive price drops very fast.
Here is my setting.
* 6 2TB hard drives @ RAID 5 for master drive
* 1 3TB external drive for backing up important files
* Using MS SynToy to keep my files always updated.
Canon 5D3 | Samyang 14mm F/2.8 | Sigma 50mm F/1.4 | Tamron 24-70mm F/2.8 VC | Canon 70-200mm F/2.8 IS MK2 | Canon 100mm f/2.8 Macro L | Canon Closed-up 500D | 430EX | Kenko 2x Teleplus Pro 300 | Manfrotto Tripod

canon rumors FORUM

Re: Best Methods For Long Term File Storage ??
« Reply #14 on: November 13, 2012, 02:46:20 PM »