March 05, 2015, 02:18:31 PM

Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Messages - thedge

Pages: [1]
Technical Support / Re: How do you store and archive your images?
« on: February 17, 2013, 04:04:21 PM »
First, thanks for the feedback.  It's a tremendous help.

For those who burn BluRay, what brand media do you like?  I'm using an ASUS external drive and recently went through a 25pk of Verbatim BD-R 25GB 6x discs.  Of the 25, 8 were unusable, the drive simply rejected them.

as i wrote, as soon as the millenniata BLU RAY´s are on the market i will buy them.

for the time being i use the normal verbatim (not LTH) .. never had a problem with them.
make sure you did not buy LTH media.. or make sure your ASUS drive supports them.

U.S.-based Millenniata ( today announced it will offer Blu-ray M-DISCs in the second quarter of 2013, increasing both the storage capacity and the accessibility of the M-DISC.
The announcement, made at the Storage Visions Conference in Las Vegas, marks a major step forward in permanent data storage solutions for businesses and consumers by making the M-DISC available in all the standard optical disc formats.
The other major advance in the growing world-wide acceptance of the M-DISC DVD and the forthcoming Blu-ray M-DISC is the marketing and distribution partnership with Imation Corp. announced at the Storage Visions Conference by Millenniata. Millenniata and Imation, the leading world-wide distributor of data storage products, have agreed to co-brand and distribute both the M-DISC and Blu-ray M-DISC under Imation's TDK, Memorex and Imation brands.
RITEK Corporation, the leading manufacturer of optical storage media in the world, will produce the new Blu-ray M-DISC as part of its manufacturing agreement with Millenniata. In addition, RITEK has signed a license agreement with Millenniata to distribute and co-brand both the DVD and Blu-ray M-DISCs through its established distribution and reseller channels as a secondary distribution channel for the M-DISC products behind Imation.
The new Blu-ray M-DISCs will be writable and readable on any Blu-ray combo drive - an enormous step for Millenniata and the convenience of this permanent storage technology. The Blu-ray M-DISCs will also offer at least five times the amount of storage as the standard 4.7GB M-DISC.

Whether you’re a scrapbooker, a business owner, a genealogist, a photographer, or all of the above, you can’t afford to lose your priceless memories or data. M-DISC™ is designed to last for up to 1,000 years, which means once written, your documents, data, medical records, photos, and more can be read forever.

Unlike computer hard-drives and optical discs (CDs and DVDs), M-DISC™ preserves and protects your files by engraving your information into a patented rock-like layer, resistant to light, temperature, humidity, and more. In fact, The U.S. Department of Defense put this to the test, and M-DISC™ was the only solution that passed. The M-DISC™ cannot be overwritten, erased, or corrupted by natural processes. Best of all, it’s compatible with any DVD player, which means you can access your data anywhere and anytime.

Those have a pretty terrible $ to GB ratio.... $3 for only 4.7GB.....

Technical Support / Re: How do you store and archive your images?
« on: February 17, 2013, 03:48:51 PM »
I use a NAS running a form of Solaris, giving me the ZFS file system. My catalog, previews and all photos are stored on a RAID6 of 8 hard drives (RAID5 is dangerous with large drives, RAID10 isnt really redundant enough). ZFS is discussed in another thread, but the rundown is that it has built in checksumming (prevents file bit rot) and built in filesystem snapshots. Mine snapshots the photography array every 15 minutes and keeps hourly, daily, weekly, monthly and yearly snapshots of the file system state. I can go back one year (server is about a year old, eventually ill be able to go back further) and retrieve pictures deleted at that time.

That is replicated to external eSATA drives that are rotated off site. The snapshots are replicated as well.

I don't bother with online backup. I may at some point export all my photos as jpegs and back those up online but until then I don't see it as a viable enough backup strategy given my upload speed and size of my data (both photos and other important data).

I wont ever not use RAID for important data, far too much of a risk to the data. Its a vital part of data security, just like off site backups. It is IMHO completely foolish to not use both together.

Technical Support / Re: Best Methods For Long Term File Storage ??
« on: November 22, 2012, 12:12:55 PM »
And to answer the OP's question...

To me as an IT guy with data backup OCD the current best solution for longer term file storage is ZFS. Its the only solution that is reliable and has integrated file checksumming to check for bit rot, corruption, memory errors, etc. Yes you can run third party programs to do checksumming on any file system, but ZFS does it on file access in addition to a full scan whenever you want.

Significant Benefits to ZFS:
-File checksumming to check for corruption
-Scrubbing your drives, which is a complete read of every single file to check it against its checksum for corruption
-Hardware agnostic and "port agnostic", meaning a pool of ZFS drives can be moved to any other computer that can read ZFS, plugged in and read, without worrying about correct order of the drives in SATA ports (as with most hardware RAID solutions) or cross vendor compatibility (aka, an Areca RAID array cant be read on an LSI card), with ZFS you only worry about the pool software version which is easy to work with (backwards compatible and pool version can be upgraded etc)
-Immediate detection of failing hard drives, cables, interfaces, etc, far earlier than most solutions so far in my experiences with failing hard drives
-Resilvering a pool (aka, rebuilding an array) is based on data on the array, not the entire thing, so a ZFS pool that has 20TB capacity but only 2TB of data only has to rebuild the 2TB of data when a drive fails vs hardware RAID which rebuilds the 20TB including 18TB of nothing
-Copy on write, which leads to (among many other awesome things) snapshots of the data, which is fantastic for back ups (oops, deleted that file, but can be copied out of the snapshot that was automatically taken 15 minutes ago, or the one from an hour ago, two hours ago, or a day ago, or a week ago, or a month ago, or a year ago)
-Filesystem is always consistent, there is no file system corruption if the power cord is yanked out or some other sudden failure
-ARC and L2ARC (Adaptive Read Cache and Level Two Adaptive Read Cache) caching, which means that a file that is read from ZFS will get stored in the servers RAM (ARC) or on a designated SSD for faster access in the future

I could go on but those are the most of the juiciest ones.

I have all my images and my Lightroom catalog stored on my NAS which is running OpenSolaris and ZFS. Yes there is a performance hit for having the catalog over the network, but with ZFS's ARC and L2ARC it is quite mitigated. If I power the server off (which clears out the ARC) and power it up, then open my catalog there is a noticeable slowdown as it is read from the disks. Then after that it is faster as the parts of the catalog that are being read regularly are sitting in the RAM on the server. Same with previews, its slow the first time a folders previews are loaded then fast after that, even after Lightroom is closed and reopened since they are still in ARC.

ZFS is not perfect, it does need some computer knowledge to make use of it. Its other downside is it is not well suited to running on crap old hardware. It needs a 64 bit CPU and 4GB RAM to start. More RAM is better. It wont work well on that old Celeron in the basement for example. But barring that, its pretty fantastic.

For the curious people, the specs on my ZFS NAS are:
Intel Core i3-2100 3.1Ghz
Supermicro X9SCL motherboard
Three M1015 cards, each with 8 SATA ports (yes, 24 ports total)
2x 16GB SSDs as mirrored boot drives (aka, RAID 1)
4x 300GB Raptor 10,000RPM drives in striped mirror (aka, RAID 10)
2x 50GB SSD as write cache drives
1x 120GB SSD as L2ARC read cache
8x 500GB Seagates in RAIDZ2 (aka, RAID 6)
8x 1TB WD Scorpios in RAIDZ2 (aka RAID 6)

Yes, a lot of disks. The four 300GB Raptors and two 50GB SSDs are storage for my VMWare server. The eight 500GB drives store my photos, Lightroom catalog, previews, documents, email, backups, etc. The eight 1TB Scorpios I bought as cheap refurbs to store my many TB of downloaded TV shows and such until drive prices come down and I can upgrade with 8-10 3TB drives.

Technical Support / Re: Best Methods For Long Term File Storage ??
« on: November 15, 2012, 04:58:36 PM »
Amazon glacier starts to get expensive as you add more data.  500GB of storage seems to work out to $5 USD a month.  I use as they hook me up with webspace, email addresses, and UNLIMITED webspace for just $7/month.  Just ftp all my pictures up there.  The only limitation is you can't have more than 250,000 files.

FYI, if you read the terms of service for 1&1 (and most or all webhosts), using the FTP space as a backup is not allowed. Them finding out is different, but it is against the ToS.

Canon General / Re: Suggestions for website hosting?
« on: November 11, 2012, 08:21:03 PM »
I have been using for years now. No downtime that ive ever noticed, reasonable price, great service.

EOS Bodies - For Stills / Re: Firmware update for 7D
« on: August 06, 2012, 09:47:36 PM »
I downloaded it at over 1MB/sec. Only took a few seconds to download for me...

Auto ISO is a bit of a letdown, can only choose full stop ISOs. Not in between :(

Pages: [1]