Backup to Blu-ray

EchoLocation said:
I'm a big music fan and archived a bunch of concerts(maybe 500cds) on the highest quality cd's I could buy(at a reasonable price, maybe $30 for a 50pack) about ten years ago...sony
I made double copies and kept the second copies in jewel cases in a box in a cool, dry closet. I never played them, or touched them.
ten years later, almost half of them have lost their sony paint on the top side and are now basically clear, and totally unplayable. I'm super pissed.
don't believe that cd's, dvd's or blu ray will last 100 years. that is 100% BS.
I now have triple backups of everything on different external hard drives in different locations(one at home, one at my mom's house, etc)

When I ran my business I would do backups onto rewritable CDs weekly. Two different manufactuers disks copy of the backup on each type each week, swapping disks every week so I had 4 disks going with this weeks and last weeks backups. I wound up the business some years ago, and found the disks when unpacking when we moved into our new house a couple of years ago, about 7 years after writing the data.

All data was unrecoverable.

CD-ROMs might have been better, but I'm not sure they'd be good for a decade.

IMHO there are three ways.

1. Keep data in a current live format.. i.e. in immediately accessible backup drives, ensure your data is stored with one set in a fire safe or in an alternative building (what I'm setting up)

2. Print REALLY valuable data using best inks/dyes onto paper in 2D barcode style. last time I looked it was about 1Mbyte per A4 page, you should easily get 20 years from top of the line printing, probably far more with no digital degradation.

3. Print the images you really want to keep using top end inks/dyes and store carefully, as we always did with those couple of pictures of Great Great Grandpa... and those lasted >100 years.. this is what I'm starting to do for those family pictures we have so we don't suffer digital amnesia.
 
Upvote 0
rfdesigner said:
3. Print the images you really want to keep using top end inks/dyes and store carefully, as we always did with those couple of pictures of Great Great Grandpa... and those lasted >100 years.. this is what I'm starting to do for those family pictures we have so we don't suffer digital amnesia.
My backup procedure is to take my pictures of Great Great Grandpa and using layers in photoshop I paste in next to him a copy of a naked celebrity and post it on the internet. They tell me it lasts forever.
 
Upvote 0

Marsu42

Canon Pride.
Feb 7, 2012
6,310
0
Berlin
der-tierfotograf.de
PropeNonComposMentis said:
Even just a few bits is bad enough.

Pure hashes are just there to *detect* changes, but there are recovery record schemes for data reconstruction when some parts are lost like par2 and rar. I used the latter for burning data to cdr, it served me well when some damage occurred.

PropeNonComposMentis said:
Analogue Storage is the only way.

+1 - I'm always thinking of those poor kids being burdened with terabyte-data of their childhood videoed and shot with high-mp cameras (containing nice bokeh, of course). Much better to print 'em or do vhs copies, at least they fade away over time and it gets less embarrassing :)

LookingThroughMyLens81 said:
MDisc is the only way to go. For those of you complaining about it being 25GB, the next update to the MDisc line which comes this year will be BD-XL discs and they pack 100GB on single-sided discs.

Somehow, simple /me doesn't buy the "one millenium" life span - that's the time of the collapse of the Maya civilization and when the Vikings first settled on American soil.
 
Upvote 0
I agree with Analog as being superior. Nothing beats a few photo albums in the cabinet that are easy to pick up, look through and enjoy. Anything more becomes cumbersome at best and impossible at worst.

But our world is now digital. Most seldom print anything. So it all must be preserved somehow. Print a few, great! But wouldn't it be great if we still had the negatives to all those ancient photos from 100 years ago? Now we do as long as they aren't deleted they can live forever in a digital format as long as they are cared for and maintained.

Unfortunately the true reality is that there is now so much overwhelming data that it is a real possibility that most of what we have now will never be seen after the initial creation and will likely be deleted/discarded at some point if not by us then by someone else in the future. Sort of like a lot of the studio music tapes from the 50's, 60's and 70's of famous music artists. Tape was expensive and if the music wasn't making money or it wasn't a "keeper", studios would record over the earlier sessions because they had no desire or need to keep hundreds or thousands of tapes 'just because'.
 
Upvote 0
We have to face a hard fact:

The files that we store today in hard disc, or any other media that allows you to delete will be deleted someday.

Imagine that after his death, his children will seek the childhood photos of a million RAW files ... What are the chances of it find the file he seeks, and can view it and process it before printing?

Does anyone have the illusion that other people have the same care you have with your files?
 
Upvote 0
ajfotofilmagem said:
We have to face a hard fact:

The files that we store today in hard disc, or any other media that allows you to delete will be deleted someday.

Imagine that after his death, his children will seek the childhood photos of a million RAW files ... What are the chances of it find the file he seeks, and can view it and process it before printing?

Does anyone have the illusion that other people have the same care you have with your files?

Yep. Pretty much. It's up to the 'person in charge'. At this time, that is likely you. Later, it will be someone else. If I am any indication, based on my personal OCD habits and 'save stuff' mentality, we're all doomed because as much as I hate to delete or toss things, I still don't have much (or care much) with regard to my parents' stuff. A few prized items that mean something to me but otherwise, nada.

So, I'm trying to force myself to realize deep down that 99.9% of my valuable stuff that I have worked hard to buy/acquire/care for/store/obsess over/worship/whatever will be gone in the dump or sold off in probably 20+ years. Little by little that is helping me "detach" from it since it's essentially gone at some point anyway whether I am here and in an institution or dead and buried it will all still be out of my hands and concern. :-\

That's pretty much it. Back in my teens, 20's and 30's I was going to live forever and I wanted to acquire stuff. Now that I'm in my 40's, my older relatives are dying off and I'm faced with dealing with their stuff, I'm realizing that it's all just crap that gets old and will end up in a garage sale, trash can or on eBay. :-\

Ashes to ashes and all that...

The nice thing about photography these days is that a few great photos have the potential to live on for someone who cherishes them. So in doing something that you love, you will brighten the life of someone else in the process. There aren't too many hobbies that work in that way. That's why I have so many images uploaded to zenfolio for everyone else in the scout troop, church and school to access. If those images mean something to others enough for them to download and share them, they might live on down the line for others that cherish them and all my efforts will not be in vain... :D
 
Upvote 0

LDS

Sep 14, 2012
1,771
300
PropeNonComposMentis said:
2.5" Ext Drives.

These are usually cheap drives designed for laptops, and not designed to last long (you can see it from their actual prices). There are drives designed for long term storage, but it's not those. They may last longer if you connect them just for backups, and then store them properly, but don't expect much. They are cheap, though. Anyway, even magnetic archiving will decay with time (flash memories included).

The quest for the definitive backup media continues.... :)
 
Upvote 0
There appears to be some confusion in this thread. Since long-term archiving of digital data is essential for me (and no doubt for many of you), I have investigated this in some detail:

Hard drives. Great for short-term storage, backups, but not for long-term archives. Biggest problem is not de-magnetisation of the disks (half-life is typically ~70 yr) but aging of lubricants etc, resulting in mechanical failure after 3-5 yr. The problem is actually exacerbated by storing the drives powered off, off-line. Best strategy is to store multiple copies and re-write the data every 2 yr or so. On-line discs are ok for backup, but too volatile for archive.

SSD/flash drives lose their charge over time and typically last 5-10 yr unless re-written.

Cloud storage requires high bandwidth connection, useful for smaller amounts of shorter-term backup, but not for archives. Do you trust the cloud storage provider to still keep your data safe 20 yr from now? Will the company still be around?

Optical media - considered to be the ultimate digital archival solution, adopted by libraries etc. Note that there is a huge difference between pressed media and write-able discs. Write-able CDs and DVDs tend to use organic dye that deteriorate in time - could be as bad as in a few years only. There are some exceptions, m-disc being a notable one which uses inorganic dye (in the form of "rock powder"!), but requires specially designed writers with extra powerful lasers. Regular blue ray discs also use inorganic dyes, so should be fine. There is also an m-disc BD version, but it is not clear if there is any longevity advantage yet (although they are marketed as 1000-yr solutions). Stay clear from the BD LTH discs, however, which are developed to use the same manufacturing processes as CDs/DVDs with organic dye, to significantly reduce the manufacturing costs. They have similar lifetimes as the regular write-able CDs/DVDs. Note that pressed media are different and should have lifetimes on the order of 100's of years. They are typically not useful for archival purposes, though, since you need a very big series before it becomes economically feasible to press discs.

Magnetic tape - the most economical archival solution for huge data sets is to use magnetic tapes, such as LTO. There are archival WORM tapes ("write once, read many") that are certified for 40 yr storage, and you can get a 2.5 TB tape for less than $60. These tapes are typically used by huge data centers. A problem is of course that the tape drives are typically > $1500. Perhaps they are rentable.

A good practice to protect from bit errors is to save redundancy information together with the data, in the form of error correcting codes (ECC). Popular software to generate these are ICE ECC, MultiPar, and dvdisaster (the last optimised for ECC on optical media).

Some general, disorganised rants about archiving that I nevertheless found useful.

In summary, hard drives are good for short term backups, (non-LTH) BDs for long-term storage of limited data generation rates (less than ~ TB/yr), magnetic tapes for large data rates (several TB/yr).
 
Upvote 0
FTb-n said:
Anyone writing images to Blu-ray discs as part of you backup strategy? If so, which size disc?

I currently backup to multiple external hard drives. Previously, before 25+MB RAW images, I also burned images to DVD. A couple years ago I started burning to 25 GB Blu-ray, but I've fallen behind. I've also had write issues with some discs failing. Grouping images to fit 25 GB discs can also be a pain. It's now cheaper and easier to get another external drive, like the WD My Passport to use as a backup. But, will a hard drive only strategy be reliable in the long run?

There is something about write-once media without moving parts (like Blu-ray discs) that intuitively seems to be more reliable for long-term storage than moving-platter-based media. But, I've had issues with burning discs on one drive that can't be read on another (even after finalizing them). This leaves me with the fear that I could burn a bunch of discs on a drive that might drift out alignment only to find out years later that I can't read them.

I would like to incorporate the cloud, but last year (2014), I accumulated over 33,000 RAW images that consume roughly 950 GB. A TB/year can of data on the cloud can get expensive quick.

Backblaze is $50/year if you pay for the full year at once. Unlimited backup of any drive except NAS. Does not require Java which is a great thing for computer security.
 
Upvote 0

Old Sarge

CR Pro
Nov 6, 2012
247
16
Joe M said:
If you want to get paranoid, make sure you have a few, store some in other locations if you can or buy fireproof/waterproof ones.
You mentioned the word paranoid so I thought I would tell a story on my son. He has always been great at backing up his system(s). It is done automatically. I believe he has a RAID set up and then a separate back-up system in another room of his house. A few years back he placed a computer on top of the gun safe at my house and at 4 a.m. it does an incremental back-up of his system (1.7 miles away in the same city) over the web. As he installed the system he asked if I thought another system at his mother's house (roughly seventy miles away) might be a good idea in case both of our houses burned at the same time. That is paranoid.

I like the dock that tolusina posted. I have an older external Thermaltake dock for hds and use that for casual personal backups on various drives too. I think my next tower build will include that dock. With the price of SSDs dropping too, that will likely become the practical backup solution to simply pop in one of those and backup a ton of stuff in seconds.
My son, the paranoid geek, spent some of his time over the holidays to get two Synology DiskStation DS1515+ 5-Bay NAS Servers (bought from B&H) up and running. Eventually one will be at his house and one at mine, replacing the computer we are now using (an old case with minimal equipment). Like I say, he is serious about backing up data.
 
Upvote 0
Mar 25, 2011
16,847
1,835
PropeNonComposMentis said:
ajfotofilmagem said:
I used the DVD-RAM media (rewritable professional standard) for more than 10 years, writing and rewriting again and again, and I know it really last many decades. Unfortunately, the high price made it uncompetitive, and today the market is most new readers incompatible with this type of disc.
....
DVD-RAM was Mil-Spec, but the Military never used it. Thats the whole story.
Re-Writable media will always out-last Write-Once. It comes down to construction method and materials used.

But you guys are still not getting it !!
Here is a hint:- I have Video Tape - User Recordable, from the very beginnings of the 80's. Picture is still perfect, but the Audio is not so good.

The down side of using NAS+RAID, beyond the bleeding obvious. The drives are manufactured within seconds of each other, genetic clones - effectively.
They spend there life powering-up together, spinning-down together, power-spikes together, brown-outs together, replicating errors together. This is Logic, is it not !

No,


I have never bought drives with consecutive serial numbers, I do check to see that they are from different batches, they often have different firmware versions as well, but I do generally buy them from the same manufacturer. My drives are powered up 24/7, they are on a UPS and protected from spikes and brownouts. The raid 5 arrays can withstand 2 drives failing and keep on going. My current set of six 2TB drives are reaching 5 years old, with no failures. I'll likely be replacing them with either 6TB Drives, or SSD's. The NAS before this one lived 5 years then the power supply died, but the disks were not affected. Even before that, I had a 4 drive Raid Disk Array of 150TB Drives, no failures, Before that, 4 - 72 TB Drives, Before that, 4 36 TB and before that a 4 Drive Raid Array of 13GB Disks (1990's) with no failures. All of my CD's made during the late 80's to late 90's rotted away, but I still have that old data on my hard drives.

Where you get into trouble with hard drives is shutting them down and letting them sit for years.
The bearings freeze up, so the drive won't start spinning.
 
Upvote 0
LDS said:
PropeNonComposMentis said:
2.5" Ext Drives.

These are usually cheap drives designed for laptops, and not designed to last long (you can see it from their actual prices). There are drives designed for long term storage, but it's not those. They may last longer if you connect them just for backups, and then store them properly, but don't expect much. They are cheap, though. Anyway, even magnetic archiving will decay with time (flash memories included).

The quest for the definitive backup media continues.... :)

Umm... you're off a bit on the 2.5" drives. The 2.5" external drives are simply SATA laptop hard drives. Nothing more, nothing less. Yes, the drives are usually less expensive because they are typically not the highest performance, being 5400 RPM which is fine for their intended use. Their price is usually market driven. But they function just like any other laptop hard drive for better or worse.

I'm not sure what you mean about 'There are drives designed for long term storage'. Hard drive technology is pretty universal when it comes to the platters and their magnetic retention. There are drives designed for enterprise level use and durability but in this case we aren't talking about MTBF, we are simply talking about magnetic retention. Hard drives don't typically fail based on any logic. They just fail whenever they feel like it. They are mechanical devices. Use them less, they will usually last longer. I spent time for years with a friend that owned a data recovery company and I know a thing or two about hard drives. And all the companies make lemon drives, sometimes on purpose (profit) and sometimes by accident (bad parts lot, etc). Regardless, hard drives are pretty reliable on the whole. Sort of like airplane travel.
 
Upvote 0

Marsu42

Canon Pride.
Feb 7, 2012
6,310
0
Berlin
der-tierfotograf.de
PropeNonComposMentis said:
I would recommend against using drives with more than One TeraByte per Platter. The Areal-Density of such drives is simply pushing the laws of physics.

That's what people said long before, but the industry came up with gmr, and it's now at 1,3TB/platter - who knows what's next in the pipeline: http://rml527.blogspot.de/2010/10/hdd-platter-database-seagate-35.html

Imho the problem is about *new* tech as new developments reach the market quickly given the competition, and that's even after only very few hdd manufacturers are left standing. And it's about the *amount* of platters inside a hd - more platters = more problems.

Problem is: Both issues are connected, because less density = more platters. When in doubt, personally I'd definitely go for a less platters, lower spindle speed and higher density - as long as it isn't the very newest platter tech.
 
Upvote 0

LDS

Sep 14, 2012
1,771
300
RustyTheGeek said:
Umm... you're off a bit on the 2.5" drives. The 2.5" external drives are simply SATA laptop hard drives.
Nothing more, nothing less.

Exactly. Not designed to last long. Most external drives are built to be cheap. Not to be long term storage devices. Good to shuffle files around, not for long term backups. There are external adapters that let you plug in the disks of your choice. IMHO far better choice than pre-assembled ones where you often don't know what disk is inside, for backups.

RustyTheGeek said:
Hard drive technology is pretty universal when it comes to the platters and their magnetic retention.

But construction is not. There are different quality levels. The choice of materials and parts, production line specs and quality controls, etc. etc. After all, are the low end Canon DSLRs and lenses built with the same materials and specs of the high end ones? After all the technology is pretty universal when it comes to CMOS sensors, shutters and lenses... isn't it?

RustyTheGeek said:
There are drives designed for enterprise level use and durability but in this case we aren't talking about MTBF, we are simply talking about magnetic retention.

But MTBF matters too. Sure, you can use multiple copies to reduce the risk - that's why I always alternate new sets to ensure they don't come near the MTBF together - but disks are more complex devices and thereby with more points of failure.

The quality of the controller and drive electronics/firmware has impact on the "correctness" of data written and its lifespan, as well the quality of the magnetic layer used for recording. Why high-end disks usually come in lower capacities (and longer warranties) than the lower-end ones? Because until they can warrant the same quality, they don't risk to sell you a disk that may fail too soon because the technology is not mature enough.

RustyTheGeek said:
I spent time for years with a friend that owned a data recovery company

I spend part of my working time with Very Large Database applications running on SANs (Storage Area Networks) made of hundreds of disks (and lately, a lot of flash ones as well). Also the company I work for produces also military spec hardware, which needs to work in far worse conditions. Guess I learnt a couple of things too :)

Anyway, companies like Facebook, Amazon, Google and the like are now faced with the need to store huge archives for decades (of course, unless they go bankrupt earlier) - because users in they early 20s or 30s expect them to store their "memories" for that long - and are experimenting with different technologies to keep data available at least near-line (as users are less likely to access often older contents) cheaply. And Facebook is now experimenting with blue-ray 100GB disks. If you store just the platters, and not whole disks, it's far cheaper.

Some companies (Sony and Panasonic) are working on "optical disks cartridges" able to store 1-1.5 TB each with RAID capabilities Currently Panasonic already offers the ADA cartridges, but they are only 50GB in size, and are somewhat a proprietary format. With the proper recording layer (one which won't decay if a few years, or decades) optical cartridges could become am interesting long-term storage for static data.
 
Upvote 0

Valvebounce

CR Pro
Apr 3, 2013
4,549
448
57
Isle of Wight
Hi Folks.
I do progressive improvements, just upgraded a pair of mirrored 500GB discs to 1TB, the old drives will be moved to my off site (is it backup or archive) storage.
A lot of mine is JPEG only, but where I have RAW's and Dop's I will leave the processed JPEGs off to save space, yes I know the bit about not deleting stuff to make space but with the RAW's and Dop's', I figure I'm only really deleting processor time.
By doing it this way the information gets re written regularly.
It works for me, I hope it is sufficient to have the main data, mirrored, the backup at home mirrored and a single off site copy with the pair of drives rotated off site to home, intending to do this weekly but it may drop to monthly with the exception being after a special photo is taken.

Cheers, Graham.
 
Upvote 0
Mar 25, 2011
16,847
1,835
PropeNonComposMentis said:
Mt Spokane Photography said:
I have never bought drives with consecutive serial numbers, I do check to see that they are from different batches, they often have different firmware versions as well, but I do generally buy them from the same manufacturer. My drives are powered up 24/7, they are on a UPS and protected from spikes and brownouts. The raid 5 arrays can withstand 2 drives failing and keep on going. My current set of six 2TB drives are reaching 5 years old, with no failures. I'll likely be replacing them with either 6TB Drives, or SSD's. The NAS before this one lived 5 years then the power supply died, but the disks were not affected. Even before that, I had a 4 drive Raid Disk Array of 150TB Drives, no failures, Before that, 4 - 72 TB Drives, Before that, 4 36 TB and before that a 4 Drive Raid Array of 13GB Disks (1990's) with no failures. All of my CD's made during the late 80's to late 90's rotted away, but I still have that old data on my hard drives.
Maybe you meant GigaByte.

@> Moderator ! Can we get this post from Mt Spokane Photography Pinned or made into a sticky. This is important first-hand information, and about as succinct as we will get.
Can we drop the SSD's. The Tech' is not mature enough for archive.

Yes, once I started on TB, I forgot to shift to GB.

There is lots of good information posted here, the fact is that no one really knows how to store digital information for long periods. You can store it, but it may not be usable.

When my Father passed away, I collected hundreds of family photos and negatives, some going way back were tin types. I scanned them all, organized them by family, and sent out about 20 or more DVD's to all my cousins or others who wanted them in jpeg format.

Some will keep them, some may lose them, but it does increase the chances that they will survive a fire or loss by a few individuals.

I also posted many of the photos on Ancestry.com, where they were and still are being incorporated into family trees of more distant relations. I have also found photos there of my ancestors, so that is another way to preserve them.

Most of the photos before 1950 are black and white, older color photos and negatives have pretty much faded away. Only chrome based color slides seem to survive, and there were none of them in my family photos.

Of course, there are photos that have been over painted with oil paints that have survived. I even have a scan of a old tintype that has a bit of color showing where it was touched up with oils.

Most of the color polaroid photos I have have faded away since the 1960's and 1970's. The B&W ones are suffering as well.
 
Upvote 0