Personal cloud vs raid array?

[Ozarker]

Well, it is time for me to take a look at mass storage to protect my media files (photos) and my wife's (video of the grandson). What I would like to do is have an external setup. I do have two old external drives (Different brands. 1tb (8 yrs old, and 2tb 5 years old) and they are working fine, however, I would rather have a four disk enclosed raid or cloud setup.

My assumption is that a personal cloud unit is only storage and not Raid capable. Raid, instead, would add redundancy and give protection that a cloud doesn't offer. Am I correct?

I have extra bays in my PC case, but would still rather have something portable to swap between machines. The disks wouldn't have to be massive (4TB each in 4 bays? Raid 1?) Yes, I know those are big drives, but there are far bigger.

So, what would y'all recommend? There are many different features on some of these pre-built devices. What should I look for? If my computer only supports DDR3, should I still consider DDR4? USB, Firewire, or SATA?

Do you also backup to DVD disk?

Thanks!

 

[LDS]

Cloud storage usually has some form of redundancy to avoid to lose your data (it may not be something like RAID, it doesn't scale that large). Of course, not all "clouds" are equal. Depending on what you buy (or get for free), there are different "service level agreements" that tells how much safe your data could be.

Some cloud storage are accessed like disks, others may need specific applications to store and access data.

Advantage of cloud services is they are off-site and good ones are replicated across different sites in different areas of the world, and managed by professionals who know what to do. Disadvantages is the bandwidth required to store and access data, and that they will delete your data if you stop paying (also there could be security issues, your data being on someone else's system).

DDR is a specification for memory (RAM), not for disks. Firewire is now fully obsolete, and replaced by Thunderbolt. There could be advantages in using disc-specific protocols like SATA when writing to disks instead of plain USB because they are more reliable for mass-storage devices. That said, USB 3.0 implements UAS (USB Attached SCSI) that should be faster and more reliable. Be aware UAS needs support both from the OS and the disk.

Disks are also moving to NVMe protocol which can take full advantage of SSD speeds. For spinning disks, SATA 3.0 is adequate.

With 14TB disks around, 4TB are in the "small" range, nowadays but of course you need to buy the storage space you need. You can read a lot about the different RAID level around, and what are each one strengths and weaknesses. There are also newer file systems that try to overcome the shortcomings of pure hardware RAID (i.e. ZFS, ReFS, Btrfs)

Take also into account that there disks built for performance, and others for reliability.

With "local" system you have a choice of DAS (Direct Attached Storage) and NAS (Network Attached Storage), The first is attached to a single PC, usually using USB or Thunderbolt (eSATA is now getting obsolete), while a NAS is attached to a network (cable or WiFi), and can be shared across many devices easily.

The advantage of local storage is higher access speeds, and everything fully under you control, and you don't have to keep on paying to avoid deletions. The disadvantages it's on-site, and maintenance is full up to you.

To backup to DVD you'd need probably the larger Blu-ray ones (from 25GB to 100GB), and it becomes soon less practical to burn, store and catalog them, and they reliability is sometimes not that great. I'm experimenting with M-DISC (they require a compatible writer) that should be able to last far more than plain DVDs, but only for a subset of my images, as an archiving system for long storage.

You should start to assess what technology suits best your needs.

 

[Ozarker]

Cloud storage usually has some form of redundancy to avoid to lose your data (it may not be something like RAID, it doesn't scale that large). Of course, not all "clouds" are equal. Depending on what you buy (or get for free), there are different "service level agreements" that tells how much safe your data could be.

Some cloud storage are accessed like disks, others may need specific applications to store and access data.

Advantage of cloud services is they are off-site and good ones are replicated across different sites in different areas of the world, and managed by professionals who know what to do. Disadvantages is the bandwidth required to store and access data, and that they will delete your data if you stop paying (also there could be security issues, your data being on someone else's system).

DDR is a specification for memory (RAM), not for disks. Firewire is now fully obsolete, and replaced by Thunderbolt. There could be advantages in using disc-specific protocols like SATA when writing to disks instead of plain USB because they are more reliable for mass-storage devices. That said, USB 3.0 implements UAS (USB Attached SCSI) that should be faster and more reliable. Be aware UAS needs support both from the OS and the disk.

Disks are also moving to NVMe protocol which can take full advantage of SSD speeds. For spinning disks, SATA 3.0 is adequate.

With 14TB disks around, 4TB are in the "small" range, nowadays but of course you need to buy the storage space you need. You can read a lot about the different RAID level around, and what are each one strengths and weaknesses. There are also newer file systems that try to overcome the shortcomings of pure hardware RAID (i.e. ZFS, ReFS, Btrfs)

Take also into account that there disks built for performance, and others for reliability.

With "local" system you have a choice of DAS (Direct Attached Storage) and NAS (Network Attached Storage), The first is attached to a single PC, usually using USB or Thunderbolt (eSATA is now getting obsolete), while a NAS is attached to a network (cable or WiFi), and can be shared across many devices easily.

The advantage of local storage is higher access speeds, and everything fully under you control, and you don't have to keep on paying to avoid deletions. The disadvantages it's on-site, and maintenance is full up to you.

To backup to DVD you'd need probably the larger Blu-ray ones (from 25GB to 100GB), and it becomes soon less practical to burn, store and catalog them, and they reliability is sometimes not that great. I'm experimenting with M-DISC (they require a compatible writer) that should be able to last far more than plain DVDs, but only for a subset of my images, as an archiving system for long storage.

You should start to assess what technology suits best your needs.

Thank you for the reply. RE Cloud: Yes, I was thinking of a pre-built case with the hard drives already in place and not the off site pay sites. And yes, 4TB are on the small size these days. However, I store relatively few of the photos I take. I have a friend that stores every single photo he takes. I like to keep only the best.

Both solutions seem to be about the same $, and I am leaning towards RAID. But you are right. I need to read more about what cloud offers. Thanks again!

 

[Mt Spokane Photography]

If you have Amazon Prime, they let you upload your photos to their cloud servers free, but you need pretty fast internet to upload a massive set of data, you can't beat the price if you already have prime.

You can keep a backup of your photos on a removable hard drive, that will provide some redundancy in case of a computer drive crash, but a house fire, for example might take everything. You can keep a copy off site, but keeping it up to date might get neglected.

I have two Raid NAS units that I keep copies on(one backs up the other), a NAS is susceptible to the same house fire that a single drive is, so a off site backup is still useful. You can also I also have a remote pc that I use as a backup and keep copies on.

Some brands of NAS units (I have Qnap) allow a snapshot to be taken which is a backup that is stored in the NAS in such a way that it is not susceptible to ransomware, ransomware will infect your pc, your NAS, and any backups attached to your pc.

You need to think it out pretty carefully and you still might lose it all. If you can, upload photos to a safe place like Amazon cloud, and to a removable drive. Disconnect the drive after backing the images up.

 

[Ozarker]

If you have Amazon Prime, they let you upload your photos to their cloud servers free, but you need pretty fast internet to upload a massive set of data, you can't beat the price if you already have prime.

You can keep a backup of your photos on a removable hard drive, that will provide some redundancy in case of a computer drive crash, but a house fire, for example might take everything. You can keep a copy off site, but keeping it up to date might get neglected.

I have two Raid NAS units that I keep copies on(one backs up the other), a NAS is susceptible to the same house fire that a single drive is, so a off site backup is still useful. You can also I also have a remote pc that I use as a backup and keep copies on.

Some brands of NAS units (I have Qnap) allow a snapshot to be taken which is a backup that is stored in the NAS in such a way that it is not susceptible to ransomware, ransomware will infect your pc, your NAS, and any backups attached to your pc.

You need to think it out pretty carefully and you still might lose it all. If you can, upload photos to a safe place like Amazon cloud, and to a removable drive. Disconnect the drive after backing the images up.

Great idea! I will check that out as I am a Prime member. Thanks!

 

[stevelee]

I periodically make a CD of my financial data, and every blue moon or so I'll make a DVD or Blu-Ray of stuff that I would hate to lose in a house fire or to theft. The discs go to my safe deposit box. The financial data are cumulative, so all of my tax forms for years will appear on each CD, and Quicken data includes transactions back to 1993, and so they all appear on each version of a disc, with current data up to that disc's date.

For local backups, I just have Time Machine send to an external drive. Stuff I want to archive rather than trash, but don't really feel need for backup, I just move to a 6TB drive (not the Time Machine disk). That would include most of my RAW files. The PSD files of my picks of the litter will be backed up at the least, and the best with make it on disc at the bank.

The internal drive is a 1TB SSD, so I try to keep it not much more than half full. Having a lot of speedy space for cache files and such makes Photoshop, FCP X, etc., fly. Even a fairly short video editing session will eat up most of the free space on the SSD, so I tell FCP X to delete its work files when I'm ready to close it. I realize there are alternative ways to set that up, but the course of least resistance seems like it would give snappiest performance.

About the only thing I keep in the cloud are manuals for my cameras and things that might come in handy to read off my devices when away.

There are lots of good arguments against my process, but it is really convenient, cheap, and safe enough for my purposes. I don't do weddings, so I won't have crying brides wanting old photos that I have lost or deleted. Dunno if any of that helps anyone, especially since your circumstances may be very different from mine.

 

[LDS]

Some brands of NAS units (I have Qnap) allow a snapshot to be taken which is a backup that is stored in the NAS in such a way that it is not susceptible to ransomware, ransomware will infect your pc, your NAS, and any backups attached to your pc.

Snapshots are slightly different from backups, because for unmodified files only one copy exists, so, if it is damaged (i.e. disk bad sector), it could be still lost (unless, of course, RAID or other redundancy features can recreate the data).

Snapshots work creating a new file copy for any file modified after the snapshot was taken (CoW, Copy-on-Write). The original, unmodified file still exists, but can't be accessed as a normal file, which makes it resilient to most ransomware attacks.

Of course they are also useful for protection against accidental changes and deletions, they take less space than full backups, and can be taken more frequently. Backup files too can be protected using snapshots. Multiple snapshot can be usually taken, and deleted to recover the used space.

Some cloud services may offer snapshot-like features - i.e. DropBox and OneDrive (the latter only with a subscription), cloud services which are accessed as disks can be victims of ransomware too.

 

[Kit.]

1. Amazon Prime Photos are not a replacement for your personal backup copies. Ideally, you would want to have both. While Amazon cloud is much less likely to lose data by itself than your USB backup disk, ransomware effects and accidental deletes can still replicate to the cloud.

2. Don't configure two-way synchronization with the online archive. The local side should be configured as the read-only master source. That would minimize the chances that glitches in your synchronization client would delete local files.

 

[Zeidora]

I keep RAID 1/10 arrays. Important to whatever you choose is locality redundancy. If you store on 5 drives all at home, home burns down, all five drives are toast. I keep one set of RAIDs at home, one set at work. It is much less likely that both burn down at the same time. Re cloud/ISP storage, sometimes you can also physically mail a company some hard drives. I'd consider cloud storage only as a fail-safe measure (= second location) but would not count on accessing it on a regular basis. Uploads as you go may work if you have decent internet speed. At my old house, it was pathetic, now it is very good, but still not quite T1 speed. For really large uploads (GB) I still use the work network.
I had NAS before, but got burnt with NAS controller going south, so now just use RAIDs. I can transfer files form different computers onto the external drives by doing local 1 GB network (LAN) at home. Looked at 10 GB ethernet, but that is still prohibitively expensive, and you would have to be VERY careful with hardware selection. One weak link and it is all for nothing.

 

[privatebydesign]

I don't know how appropriate it is for any other user but here is my storage history.

Computer only - upgrade computer HDD - put more HDD's in computer - external HDD's, started with one - ended up with 6 daisy chained FW800/esata Lacie drives - change out drives in Lacie cases - same again - same again - QNAP 453A NAS 4 x 10TB (realized 1GbE NAS really wasn't up to editing video or even big PS files without slowdowns) - added two drive striped enclosure attached to computer and backed up to NAS.

This gives me the performance and backup that I want and feel comfortable with, my attached storage is fast enough to do the work I want and the NAS gives me the space to backup everything and have it connected to the internet so I can access a copy of it anywhere.

Now some will argue that my backup is weak because it is in the same location, my answer to their scenario of burning down houses is this, if my house burns down to the extent that all my pictures are lost, first, I have much bigger things to worry about than some pictures, two, I am not self absorbed enough to imagine my images are any kind of loss to anybody but myself so go back to first point.

As an addendum, there are a lot of interesting things happening with NAS's at the moment with 10GbE being fed over Thunderbolt3 as direct attached (though it behaves as network attached), if I was starting from new and had a USB-C/Thunderbolt3 computer I'd get a QNAP TS-453BT3, this seems to offer the speed needed for locally attached storage and offer NAS.

 

[Mt Spokane Photography]

Snapshots are slightly different from backups, because for unmodified files only one copy exists, so, if it is damaged (i.e. disk bad sector), it could be still lost (unless, of course, RAID or other redundancy features can recreate the data).

Snapshots work creating a new file copy for any file modified after the snapshot was taken (CoW, Copy-on-Write). The original, unmodified file still exists, but can't be accessed as a normal file, which makes it resilient to most ransomware attacks.

Of course they are also useful for protection against accidental changes and deletions, they take less space than full backups, and can be taken more frequently. Backup files too can be protected using snapshots. Multiple snapshot can be usually taken, and deleted to recover the used space.

Some cloud services may offer snapshot-like features - i.e. DropBox and OneDrive (the latter only with a subscription), cloud services which are accessed as disks can be victims of ransomware too.

Of course, I operate my two NAS units in Raid 5 and 6 mode, I can't imagine not doing that. But, yes, each type of backup is subject to lots of potential issues.
Using snapshot removes some of the risk, but not all. I also use removable drives for offline storage, but those get updated less frequently. I do not use cloud storage because my internet speed going up is far too slow.

Thats why its good to use multiple independent means of backing up data. I also have some "M" disks, but doing all my raw files is out of the question, so I only save family photos to them.

 

[LDS]

Of course, I operate my two NAS units in Raid 5 and 6 mode, I can't imagine not doing that. But, yes, each type of backup is subject to lots of potential issues.

Correct. I was just pointing out differences so people still learning could build the best strategy around their needs. Snapshots are usually a file system feature, and can work even on a single disk (or group of disks without redundancy, like JBOD or RAID 0), even Windows (NTFS) supports a form of snapshots, restore points are based on this technology and can be enabled so "previous versions" of files are available.

For photos, I usually take a specific snapshot after an import so the next one doesn't risk to overwrite something (and, of course, against malicious activities). Using mostly a non-destructive editor like Lightroom there's less need to protect against accidental changes, anyway because snapshots are "lightweight", taking them every n hours or as needed helps to increase data availability. They could still help to minimize the loss if a LR catalog gets corrupted, for example. If I were using Photoshop, I believe I would use a somewhat different approach.

Then of course there's the need to take care of data using multiple disks with redundant data, backups and multiple copies, and moving some copies off-site.

That also make local and remote (cloud) storage more complementary than alternative - albeit that increase costs.

 

[kaihp]

Well, it is time for me to take a look at mass storage to protect my media files (photos) and my wife's (video of the grandson).

@LDS did a very good write-down of your options, so I'll just add a few bits.

You appear not to have much data, when comparing to modern disk sizes (you might be surprised by the GBs chewed up by video of the grandson over time though), so storing all the data on a single (backup) disk is feasible.

Backup and RAID serve two orthogonal purposes: backup protects against willfull or accidential deletion, and RAID protects against catastrophic disk errors.

For backup considerations, a good rule of thumb is the 3-2-1 rule: 3 copies, 2 different media, 1 offsite. 2 different media protects you against drive disasters. The offsite copy protects you against theft, fire and the like.

Depending on how much you value your data, a less elaborate backup scheme can be applied, say a local NAS box with 2 x 4TB drives + an online backup.

I have gone for the 3-2-1 scheme, with my NAS being the primary copy (it has snapshots so simple errors can be fixed easily), a single 6TB disk being the secondary onsite backup (gets connected and synchronized once a week), and an online backup (Crashplan Small Business).

While this can seem to be a bit OCD (and yes, it probably is), consider the following scenarios:

• I've lost an entire disk worth of data by dropping it out of a sleeve 3' down on a wooden floor (a 2.5" WD Elements portable drive; it was unpowered at the time. Probably the R/W head was damaged).
• Twice I've had to RMA drives due to media errors creeping up (saved by SMART). Both times no data were lost, because data was covered by the redundancy built into the systems.
• A friend of mine had a break-in into their home recently. The thieves took his 4 external drives with the last 14 years of photos and videos. He was left with the 1280x1024px versions in their electronic picture frame.

A QNAP or Synology 2- or 4-drive box sounds like it will fit your bill well for the immediate redundancy.

My assumption is that a personal cloud unit is only storage and not Raid capable.

Your assumption is incorrect as "personal cloud" and RAID are really orthogonal issues. Case in point: WD makes the My Cloud Mirror "personal cloud" product (link) with 2 drives in a RAID1/mirror configuration.

It seems that vendors have started calling a NAS device a "personal cloud" device, since "cloud" is sassy and shows you're on the beat with the transition to move away from local, dedicated systems to renting somebody else's systems.

 

[cayenne]

For home storage and BU off my working drives, I have some NAS units. I like the Synology brand, my 4 bay one is a bit older but works great, I have the Diskstation DS413j

That is a good one to start with....just get the newer equivalent.

You plug it into your network, and it is accessible by any computer you grant access to it on your network. You "can" even open it up to the greater internet for you to access when out and about, but I don't like that kind of security exposure.

But buy one of those, slap your 4 x 4TB drives in it, follow the instructions for whatever RAID level you want and it will format them and you'll be in business.

I've got that locally.

Im about to set up a business internet acct. at my Mom's house out of state...and set up a couple of small servers there and set up NAS stations and use that for my offsite backups.

It isn't rocket surgery....and prices are a bit more reasonable these days.

Just make sure to get some good drives, not the cheapest you can buy. But ones that are set up for backup and NAS stations.

I forgot and will have to look it up, but there was one famous commercial backup site out there, that published their drives THEY used and the longevity and performance of them...I went with their recommendations and I think at the time, the Toshiba drives were some of the better ones to get and so far, mine have all lasted years...I think I've had ONE go out this whole time and had to swap it out and let it automatically rebuild the RAID.....

HTH,

cayenne

 

[Don Haines]

As has been said above, off site backup is essential. With everything in one room, fire or theft will clean you out.

Talk to a friend or two who share the same problem.... I keep a backup at a friends house and have her backup at mine, and use a portable hard drive to move things over. Another advantage of an off line backup is protection from ransomware.....

 

[Halfrack]

Let's slice this into a few different components of discussion. As it looks you've got a 3tb of disk space between two old drives, how much of that is backup of one drive onto the other? How big are your 'photos' directories?

I'm going to start by saying a NAS is going to be a bad deal for you, and given the size of data we're working with it would be a huge overhead in costs and management. When you're at 6-10tb of data, then we can start talking about a NAS - keep in mind this storage density keeps growing, simple is better.

Reasons to get a NAS compared to local (DAS) storage:

• Mac and PC both able to access the same files
• Multiple machines access the data at the same time
• Primary machine being a laptop
• Fun remote access features (vpn home to get a file on the road)
• Huge data sets (15tb and larger)
• Higher performance for storage (having multiple drives ready/write together)

Reasons to get a basic USB3 disk

• Simple management
• Simple upgrades
• Backup works just like any other drive
• Faster than a NAS (until you get into 10gbps setups)

I wouldn't recommend a cloud as your primary storage, but would heavily recommend a cloud backup provider. Part of why I recommend single or JBOD storage for most folks is that things like BackBlaze will backup the data on external drives. NAS systems have different backup methods and they're all more expensive than $5/month for unlimited storage.

For under 4tb of current data, I'd get 2-3 disks in the 4-6tb sizes. The laptop drives are smaller, but slower performing. Take a 4tb desktop external USB3 drive and plug it directly into the computer. Copy the data off your 2 existing drives onto that single drive. Next, take those 1&2tb drives to a friends place and leave them there. Setup online backup and let things trickle up to the cloud. Next I'd add another external USB3 disk and have my computer backup the internal disks, plus that initial 4tb drive to the second disk (usually a larger 5-6tb drive). Unplug it once the backup is done and set a reminder to plug it back in on a set schedule.

So you end up with:
- USB3 primary photos drive
- USB3 backup drive
- Offsite of original data drives (1&2tb)
- Offsite backup to cloud

The final would be to sway the 'backup drive' with a 3rd external drive rather than disconnecting and reconnecting the 2nd drive.

 

[Halfrack]

Costco normally has 5-6tb disks for $90-$120

Here's a way to think about it - let's say you went with the plan and got 1 4tb drive for 'primary photos' and 2 6tb drives for 'backups'. To 'grow' your primary photo space, do a backup to one of the 6tb drives. Next format the other 6tb backup drive, and copy everything from the 4tb 'primary' drive to the empty 6tb drive. That 4tb drive is now a point in time archive of all your photos - find a safe place for it and label it well. The 6tb drive that was a backup, now becomes your new, larger 'primary' photos drive. Purchase a new 8tb USB3 drive and use that as the second backup drive.

The goal of this is to keep the number of disks to keep tack of to a minimum, and the costs as simple as possible. If you wanted to clean up the cables a bit, getting a 2 bay JBOD enclosure works well, just don't set any 'RAID' on a dial or dip switches - leave it on JBOD so you see the 2 separate disks.

 

[Halfrack]

Here's the article that cayenne is referencing about disks - https://www.backblaze.com/blog/hard-drive-stats-for-q2-2018/

Backblaze has great information as to hard drive reliability, and shows that no matter how much you spend on a drive, you're only buying more performance or a longer replacement warranty. Personally, I'd rather have 2-3 copies of data and smash a dead hard drive than try and get a warranty replacement.

 

[cayenne]

Here's the article that cayenne is referencing about disks - https://www.backblaze.com/blog/hard-drive-stats-for-q2-2018/

Backblaze has great information as to hard drive reliability, and shows that no matter how much you spend on a drive, you're only buying more performance or a longer replacement warranty. Personally, I'd rather have 2-3 copies of data and smash a dead hard drive than try and get a warranty replacement.

Thank you!!
That indeed was the article/place I was thinking of!!

In my setup, I have an older MBP late 2011...I upgraded it with 16GB ram and replaced the local HD with 1 TB SSD.

I keep my work on a thunderbolt unit that holds a SATA drive (spinner) which is about 3TB, that is my main work drive, I have another thunderbolt unit daisy chained off that to a SSD SATA drive..that is what I used for cache on my apps.

I have 2 NAS units. The first one, is a 2 drive one, mirrored.....that one is 3TB and I use that to backup my working 3TB drive I mentioned just before.

I have a 4 bay NAS (both are synology)...I have it loaded with 4TB drives. ON that I backup all my images when I unload my CF/SD cards from my camera...one copy goes to work drive other copy goes to that larger NAS.

I also use that larger NAS to house and store my install binaries for all my apps (CS6 suite, On1, Davinci....anything I've bought or installed).

I like the NAS...as they are set up with basically RAID 5....it is somewhat a proprietary version from Synology that helps manage if you happen to have different size disks in the NAS. This is quite handy when 'upgrading' to larger disks. I used to have 3TB drives in the 4 bay one....I bought all new 4TB ones and added them one at a time replacing the 3TB ones...and let it rebuild till I had all 4TB drives in there.

I like having NAS over just single backup drives...as that if the single drive goes tits up, you have a problem, but with NAS....one drive goes out...you can swap it out and rebuild the NAS from the other drives.

I plan to set up, as I mentioned in other post, to an offsite place I have control of, out of state...and set up NAS units there and do a little rsync job maybe as a cron job and keep things mirrored between my local NAS and the remote NAS units.

At that point, I feel a bit better about having my file safe.

Thing is, I didn't built all of this at once. Like many, I started with NO backup.
I then bough dual bay NAS as first try.....then 4 bay one...and likely at some point a larger NAS with more redundancy....look for sales and SAVE and buy as you can.

Too many folks think they have to spend $$$$$ all at once, and get a big complex system....and sometimes just give up and don't do anything.

Remember NONE is bad, a simple solution is better than NONE...and as you can afford....more is better than ONE.


Hope that helps!!

C