Best Methods For Long Term File Storage ??

[1255]

Hi all.

Would like to ask for your thoughts on best methods for long term file storage, meaning, external hard drives or DVD/Blue Ray? Or something else?

I had always thought that external hard drives were best, at the very least because who knows when we'd need to convert those DVD or Blue Rays to a new technology.

Then I just read in another post that external hard drives corrupt files over time. Is this true? I had always thought that it was the process of dragging files and/or copy and pasting them over and over that impacted data bits and degraded the files.

What do you think is the best method for long term file storage?

As always, thanks to all of you for your input.

Much appreciated.

[Drizzt321]

Hard drives suffer from bit-rot (as do some other forms of storage). It can occur in a couple of cases, such as improper storage, flooding, etc, or cosmic rays, or other similar high energy events.

There are a couple of ways to mitigate this. First, keep a copy on several different storage devices, preferably of different types. This helps mitigate one of them going bad. If it's all hard drives, get ones with serial numbers which aren't right near each other, which can help mitigate if a particular lot of hard drives has issues (it happens, manufacturing isn't perfect).

Further, you can use a filesystem or storage mechanisms which provide a high level of error correction and detection. You can use something like Par2/Parchive which provides parity and checksuming which can detect, and often correct for bit-rot errors.

A good archiving solution will including multiple storage media, and proper storage of that media. In general, a high quality tape backup with proper storage tends to be the best, very long term archival mechanism and is still very popular and widely used in various enterprise backup/archive solutions.

[1255]

thanks very much

[Standard]

I simply archive all my files onto DVDs. If there are important files that I worry about, I make sure to archive them onto several copies for redundancy (I do this over a period of time) so as to avoid possible disc errors. All discs are then catalogued (with a cataloguing app for quick search and retrieval at a later time), then stored onto large 200+ disc binders and placed in metal file drawers. Simple as that. No issues thus far. And yes. I archive an extra copy of select important files onto Lacie 1+ TB external hard drives.

[Iahcon]

I keep two copies on separate external hard drives.

[neuroanatomist]

I keep two copies on separate external hard drives.

As do I, and the two HDDs are stored in different physical locations (home and work).  Those copies are made as soon as the images are processed (and until then, they're still on the CF cards - since getting the 1D X, I always have at least 2 copies of every image).  Quarterly, I make a DVD backup that goes in a safe deposit box in a third location (different town than home or work).

[florianbieler.de]

Got mine synced to a 2,5" external hdd which i use not that often.

[Crapking]

What about on-line "cloud" storage sites which allow access without physically requiring the disk drive? 
Of course this is in addition to the physical backups, but I use Phanfare.com {with a reasonable annual fee}, which allows web-based access, including RAW storage (for an extra fee) for the really important files.

[myocyte]

I've been meaning to sign up for this, but Amazon provides a great, affordable solution for cloud storage. If anyone has tried it, I'd be interested to hear about their experience.

http://aws.amazon.com/glacier/

[sb]

I keep two copies on separate external hard drives.

As do I, and the two HDDs are stored in different physical locations (home and work).  Those copies are made as soon as the images are processed...

Same here with a small difference - I make a copy immediately after dumping the files on the primary computer. Processing is all in LR metadata anyway, and LR catalogues are also included in my backup script.

My backup script is a hand made batch file using the "Robocopy" command line utility.

My back up drives reside on a second computer which is identical with all the same applications installed as the primary. That way if my primary fails, I can continue working uninterrupted.

I don't use any optical media (DVD/Bluray) because they become unreadable after a few years. I also don't use external drives because for the price of a "network storage" I can build myself a fully functional second computer, so what's the point?

[neuroanatomist]

I don't use any optical media (DVD/Bluray) because they become unreadable after a few years.

How many is 'a few'?  I recently listened to The Cars Greatest Hits, a CD that I bought in 1985.   

[Drizzt321]

I don't use any optical media (DVD/Bluray) because they become unreadable after a few years.

How many is 'a few'?  I recently listened to The Cars Greatest Hits, a CD that I bought in 1985.   

That's also a pressed CD, vs a burned CD/DVD. The material in a burnable CD/DVD is, generally, at least partially organic and over time degrades. Some of them are 'archival' quality, but I believe that's generally only guaranteed for only 40-50 years. Not sure, I'd have to look that number up, but I know it's not forever.

[Emeyerphoto]

1. Dump on to external as soon as get home.
2. Modified images get saved on secondary external.
3. Backblaze backs them up onto the cloud, can recover them anywhere, and if necessary I can receive an external hard drive with my files on them for a fee.
4. Relax, since they are on the cloud and do not need to mess with safety deposit boxes, nuclear proof storage, or 10 external hard drives of the same data.

[cayenne]

Lots of good posts here.
ONE thing that should stand out...redundancy, and multiple backup places and methods.

CDs/DVDs/BRs...all can degrade, and depend also on the quality of your starting materials...cheap will usually rot quickly.

I'm looking to build a fairly large, redundant NAS (Network Accessible Storage) system. I'm looking to use the freeNAS system:

http://www.freenas.org/features/feature-overview/category/features

Wikipedia:

http://en.wikipedia.org/wiki/FreeNAS

What makes this nice...is using the ZFS filesystem, which allows you to add different sized / speed drives over time...and can be hot swappable....and can lose 2 (or more depending on set up I believe) drives at the same time in the system, and still not lose data.

I just bought a special at newegg the other day, a barebones computer I'm about to assemble for about $168...8GB ram (which I'll likely upgrade to 16 for about $50 or so)...

Anyway, set that up with a 5 disk enclosure I bought awhile back to start with, and build it from there.  In the future to have the comfort of redundancy offiste , in case of fire, flood, hurricane (I live in NOLA after all), I'm going to set up a similar system at my mom's house out of state...and that way, I can justify paying for her internet connectivity since it will be a 'business expense'.

Anyway, I'm getting off topic...but if you don't mind a little DIY spirit....take a look at the links and systems I've listed above. It is very high quality stuff, and can be turned into a highly reliable, flexible system that most any computer you own can connect to and backup, whether it be a Windows PC, or Linux box...or even OSX on a mac with time machine, which can often be finicky at times on its own right.

HTH,

cayenne

[cliffwang]

Always keep your files duplicated.  Any kind of media is fine.  However, you have to ensure the files you backed up is always good.  DVD is not my suggest.  Nowadays external drive is very cheap, so you can get 3TB or bigger external drive to backup your files.  Do not waste your money to buy too big external drive because hard drive price drops very fast.
Here is my setting.
* 6 2TB hard drives @ RAID 5 for master drive
* 1 3TB external drive for backing up important files
* Using MS SynToy to keep my files always updated.