July 20, 2018, 01:13:31 AM

Author Topic: NAS storage advice  (Read 2344 times)

ethanz

  • EOS 5D Mark IV
  • ******
  • Posts: 620
  • 1DX II
    • my website
Re: NAS storage advice
« Reply #15 on: July 03, 2018, 10:29:52 AM »

Also, will a USB-3 hub work? I am a bit of a techno-numpty but I understand that these allow 2 devices to communicate directly so if you have USB 3 card reader and USB3 drive, they talk while shortcutting some of the limitations in the computer (but I may be wrong)

USB 3 (as with all connections) has a finite bandwidth. So even with a USB 3 hub, you are still limited to fitting all that through one USB 3 connection. With just a card reader and external drive you will still get great speeds with USB 3.
1DX II, 16-35L f/4 IS, 24-70L f/2.8 II, 70-200L f/2.8 IS II, 200-400L f/4 IS w/1.4 EXT
http://ethanzentz.com/

canon rumors FORUM

Re: NAS storage advice
« Reply #15 on: July 03, 2018, 10:29:52 AM »

Mikehit

  • Canon EF 300mm f/2.8L IS II
  • ********
  • Posts: 2628
Re: NAS storage advice
« Reply #16 on: July 03, 2018, 10:32:39 AM »

Also, will a USB-3 hub work? I am a bit of a techno-numpty but I understand that these allow 2 devices to communicate directly so if you have USB 3 card reader and USB3 drive, they talk while shortcutting some of the limitations in the computer (but I may be wrong)

USB 3 (as with all connections) has a finite bandwidth. So even with a USB 3 hub, you are still limited to fitting all that through one USB 3 connection. With just a card reader and external drive you will still get great speeds with USB 3.

I know. But USB3 is still quicker than USB2 which was my point. And much quicker if my experience with the PCIe card is anything to go by.

jolyonralph

  • EOS 5D Mark IV
  • ******
  • Posts: 741
    • Every Other Shot
Re: NAS storage advice
« Reply #17 on: July 03, 2018, 11:12:26 AM »
I know many people have had bad experiences with them, but I still like the Drobo NAS systems. In fact, I currently have 5 of them (2 Drobo FS, a Drobo 5N, a 5D and a brand new 5D3)

In several years of use I have never had any failure that has lost data.   I had one device entirely stop working, and I was able to move the disk pack to another Drobo Unit and that worked perfectly first time.

I've had issues with corruption when power is lost, but this has always been recoverable with tools.

I did have to replace a dead fan in one of the older Drobos after it died, but it uses standard cheap PC case fans so that was easy.

I've had to talk to Drobo a few times, but I've always had things resolved.

The thing that's good about Drobo is that you don't have to mix and match the same types of drives. You throw in any* old SATA drive and you can mix them all together.   You can replace lower capacity drives with higher capacity drives, one at a time, to increase the capacity of your RAID dynamically without even needing to stop using the device while it rebuilds.


* Certain 8TB+ drives use a system called "shingled" recording which isn't compatible with Drobo (and probably not reliably with other NAS systems.  These drives die in a matter of days in a Drobo. So avoid any drives listed as SMR or Shingled.  Also avoid any Seagate drive 8TB or higher except if it specifically says it is a PMR device (Barracuda *PRO* only I believe) because those are SMR without disclosing it, as I found out after killing two of them.


« Last Edit: July 03, 2018, 11:14:58 AM by jolyonralph »
Jolyon Ralph

Cameras: 5DSR, A7RII, 5D III, EOS M6/M5/M3, Mavic Pro, DXO One.  Oh, and more lenses than I care to count.

Mt Spokane Photography

  • CR GEEK
  • ***************
  • Posts: 14321
Re: NAS storage advice
« Reply #18 on: July 04, 2018, 12:01:06 AM »
I use a windows pc, and am always fighting Microsoft upgrades which alternately loose the ability to see a NAS or other pc's using Windows explorer, so I have to specifically assign a drive letter to the folder on the NAS that I want to use.
I've given up fooling with it, I get it working and 3 months later, they break it again.

Mt Spokane Photography

  • CR GEEK
  • ***************
  • Posts: 14321
Re: NAS storage advice
« Reply #19 on: July 04, 2018, 12:03:07 AM »
There is no particular reason that I'm aware of to want a managed switch.  I have a HP 24 port managed switch and a Cisco 24 port Managed switch as a backup, but I leave them at factory defaults, it just happens that they had other features I wanted.

pwp

  • Canon EF 300mm f/2.8L IS II
  • ********
  • Posts: 2491
Re: NAS storage advice
« Reply #20 on: July 04, 2018, 01:48:36 AM »
I use a windows pc, and am always fighting Microsoft upgrades which alternately loose the ability to see a NAS or other pc's using Windows explorer, so I have to specifically assign a drive letter to the folder on the NAS that I want to use.
I've given up fooling with it, I get it working and 3 months later, they break it again.
Interesting...I'm also on Win10 PC and keep up with all those annoying MS updates. Also have assigned a drive letter to both the NAS and a Current Projects folder inside the NAS. It's a Synology 5 Bay and has never got lost with an MS update. Go figure.  ???  I wonder what we're doing differently?

-pw

Valvebounce

  • Canon EF 400mm f/2.8L IS II
  • *********
  • Posts: 3790
  • Doing my best to get all of this to work together.
Re: NAS storage advice
« Reply #21 on: July 04, 2018, 03:59:44 AM »
Hi Folks.
I also use W10, a Buffalo NAS box, never lost after ms updates.
What we do get often but not related to updates as far as I can tell, is machines that hibernate and then don’t connect to the wifi properly when woken back up, they connect to the signal but have no access to the network, we have to drop the wifi and connect again then it works perfectly. A first world problem as it takes all of 5 seconds out of my busy life ::) once I have realised I don’t have access which usually takes the old brain longer! ;D
Oh by the way they are MS Surface Pro 4, Linksys router.

Cheers, Graham.

I use a windows pc, and am always fighting Microsoft upgrades which alternately loose the ability to see a NAS or other pc's using Windows explorer, so I have to specifically assign a drive letter to the folder on the NAS that I want to use.
I've given up fooling with it, I get it working and 3 months later, they break it again.
Interesting...I'm also on Win10 PC and keep up with all those annoying MS updates. Also have assigned a drive letter to both the NAS and a Current Projects folder inside the NAS. It's a Synology 5 Bay and has never got lost with an MS update. Go figure.  ???  I wonder what we're doing differently?

-pw
7DII+Grip, 1DsIII, 7D+Grip, 40D+Grip, EF 24-105 f4L EF-S 17-85, EF-S 10-22, EF 70-200 f2.8 L IS II, EF 1.4xIII, 2xIII, EF 100-400 f/4.5-5.6l IS II, Σ17-70 f2.8-4 C, EF 50mm f1.8, YN600EX-RT, YN-E3-RT, Filters, Remotes, Macro tubes, Tripods, heads etc!

1DsIII, 20D, 24-105, 17-85, Nifty 50 pre owned

canon rumors FORUM

Re: NAS storage advice
« Reply #21 on: July 04, 2018, 03:59:44 AM »

LDS

  • EOS 5DS R
  • ******
  • Posts: 1291
Re: NAS storage advice
« Reply #22 on: July 04, 2018, 04:02:52 AM »
I may also need a managed switch - why, what configuration will I need to do on the switch?

A managed switch may offer advanced features like link aggregation (aka bonding aka teaming, etc.), VLANs, QoS (Quality of Service, you can give priority to some traffic), 802.1X port authentication, layer 3 (TCP/IP) functionalities (like routing, multicast) etc.

Some features improve speed and/or availability, other security. This usually comes at a price, because this switches have processor and software running to offer those features, software that needs also to be kept updated.

Some switches are also branded as "smart switch" or "web managed", they usually are a little less expensive but offer only a limited subset of a true managed switch features, often through a simpler web interface.

Just, setup requires some knowledge of networking and its jargon. Assess your needs, you may spend a lot for features you'll never need, otherwise.

If it's just for speed, i.e Link Aggregation, take into account that:

1) You need n ports for each device using link aggregation (i.e. 2 ports for the NAS and 2 ports for the PC), it may increase the overall ports needed, and thereby switch cost.
2) It's more cables to run around
3) You'll need the corresponding network cards on each device (supporting aggregation) - which usually means laptops are out
4) The NAS disks system has to be able to exploit the available bandwidth. Depending on the disks type and speed, controller, cache, etc. they may not. Disks designed for performance may be less reliable for long term storage like backups or archiving.

Mt Spokane Photography

  • CR GEEK
  • ***************
  • Posts: 14321
Re: NAS storage advice
« Reply #23 on: July 04, 2018, 10:29:28 AM »
I use a windows pc, and am always fighting Microsoft upgrades which alternately loose the ability to see a NAS or other pc's using Windows explorer, so I have to specifically assign a drive letter to the folder on the NAS that I want to use.
I've given up fooling with it, I get it working and 3 months later, they break it again.
Interesting...I'm also on Win10 PC and keep up with all those annoying MS updates. Also have assigned a drive letter to both the NAS and a Current Projects folder inside the NAS. It's a Synology 5 Bay and has never got lost with an MS update. Go figure.  ???  I wonder what we're doing differently?

-pw
If you assign a drive letter, then you are ok, but the NAS folders do not appear under Network as they used to, except when you have a folder mapped.  Even so, currently one of my NAS units shows up under network, but the other does not. 

 Microsoft has said the changes are for security.  So assigning drive letters is a workaround.

snoke

  • EOS M5
  • ****
  • Posts: 225
Re: NAS storage advice
« Reply #24 on: July 04, 2018, 10:39:44 AM »
Drobo + USB 3.

LDS

  • EOS 5DS R
  • ******
  • Posts: 1291
Re: NAS storage advice
« Reply #25 on: July 04, 2018, 11:36:29 AM »
NAS folders do not appear under Network as they used to, except when you have a folder mapped.  Even so, currently one of my NAS units shows up under network, but the other does not. 
Microsoft has said the changes are for security.  So assigning drive letters is a workaround.

Creating the list user by "Network" requires a quite "chatty" protocol where system "announce" themselves and that became a security issue outside very trusted networks  It also depends on a machine on the network acting as the "master browser" - and if this machine is turned off, it may take some time before a new machine is elected and the list rebuilt.

Here it's explained how it works: https://technet.microsoft.com/pt-pt/library/cc737661(v=ws.10).aspx.

Thereby Microsoft decided to disable it by default (and also tried to replace it with other solutions like HomeGroup, but it didn't last). It can be re-enabled, but the NAS also needs to be configured correctly to appear in the list - usually it's the Samba service configuration.

The issue with driver letters is that path may become dependent on their assignment, and may not work from other machines.

Ladislav

  • EOS 80D
  • ****
  • Posts: 268
Re: NAS storage advice
« Reply #26 on: July 04, 2018, 12:23:33 PM »
Going through all which was written here. I actually gave up on spending £700-£1000 on NAS now when I will have to make investment to a new computer soon anyway and that £1000 will be better used there. I'm still tempted with NAS solution but it will have to wait.

So for now, I will probably just buy a new WD Black 6 TB drive which I will later move to the new computer.

I already made some configuration of what could my new computer look like:
  • AMD Ryzen 2700x
  • AMD x470 Motherboard
  • 32GB (4x8GB) RAM @ 3200MHz CL16
  • AMD RX580 GPU 8GDDR5
  • Samsung 970 Evo 1TB SSD
  • 2x WD Black 6TB - either in RAID1 or Windows mirroring
  • 600W PSU - at least 80 Plus Gold
  • Windows 10 Pro


That should work for me for 5+ years. That setup costs about £2000. I could probably buy it right now but since I didn't plan it so soon it would stretch my budget too much. I will need to wait a bit.
5D4 | 6D | 16-35 | 24-70 | 70-200 | 70-300 | 150-600 | 17 TS | 24 TS | 50 | 100 Macro | 430 EX II

ethanz

  • EOS 5D Mark IV
  • ******
  • Posts: 620
  • 1DX II
    • my website
Re: NAS storage advice
« Reply #27 on: July 04, 2018, 01:28:48 PM »
Good idea
1DX II, 16-35L f/4 IS, 24-70L f/2.8 II, 70-200L f/2.8 IS II, 200-400L f/4 IS w/1.4 EXT
http://ethanzentz.com/

canon rumors FORUM

Re: NAS storage advice
« Reply #27 on: July 04, 2018, 01:28:48 PM »

awair

  • EOS Rebel SL2
  • ***
  • Posts: 80
    • AWAIR photography
Re: NAS storage advice
« Reply #28 on: July 12, 2018, 11:25:31 PM »
I'm not a very regular contributor here, but have been messing with most of these issues for over 15 years, with NAS, bandwidth, backup and access issues.

  • There's always a risk of hardware failure
  • The network is never fast enough
  • You will always run out of space/speed on your primary computer

My recommendations echo many of the other comments made previously.

Use DAS as opposed to NAS
Use RAID (1 for simplicity, 5 for more storage capacity)
Use multiple discs
Use multiple backups
Use SSD
Use Thunderbolt or external drives with multiple interface options.

I use an SSD equipped computer, with images backed up to:
A portable SSD (while culling/editing)
A portable HDD
A non-portable HDD (RAID-1)
A RAID-1 array in a 6-bay NAS (the logic being, I can pull one drive and keep my data with ability to install in a replacement same-brand NAS)
TimeMachine (automatic) backup on multiple discs.

If you don't use multiple machines, there is no need for NAS (however, consider 'multiple' as including computer, iPad, phone & remote access via your own private cloud...)

QNAP make a thunderbolt combined DAS/NAS, which could offer the best of both.

My best advice would be to buy bare hard-drives (warranty is typically longer) and buy a reputable brand case with the appropriate connectors (Thunderbolt, USB 3.1, 3.0 etc). Make sure that you always have at least one 'run away' disk!

Good luck.

Canonet>FX3>AE1>T90>S10>20D>7D>7D2>6D>1D X & 100D
135/2L, 300/4L, 70-200/2.8L

scottkinfw

  • EOS 5DS R
  • ******
  • Posts: 1461
  • Wildlife photography is my passion
    • www.kasden.smug.com
Re: NAS storage advice
« Reply #29 on: July 13, 2018, 12:09:49 AM »
Yesterday I returned from Tankfest with about 80GB of pictures and when trying to import them to Lightroom I realized that I don't have enough space on my hard drives to copy them to my computer. I have been thinking about buying a proper storage for quite some time but money went always to something more interesting (understand some new lens).

My current storage approach is really terrible but it worked for me so I used it for 5+ years. I'm using two 2TB drives (they may be about 6-7 years old now). One is master, second is backup. They are not in RAID. My computer is 10 years old and I didn't use its SW RAID provided by motherboard utilities. I'm doing backups manually since I'm also applying some logic in my backups (processed folders go to different place than those not processed yet). Those drives are mostly used for photo storage. Most of photos stored there are garbage which are waiting to be processed but that is another story - right now they have to be stored.

Since my computer is 10 years old (Windows desktop), it has other limitations - no USB 3.0, no SATA 3, etc. I could just put a new disk to my computer and continue with the way how I was doing things for past few years but I'm also considering replacing the computer this or next year and having photos stored on external device would simplify migration later. It would also open possibility to go for laptop instead of desktop.

I expect NAS storage to solve my problem with space and provide me very reliable automatic backup mechanism. I would also expect it to be significantly faster than just single disk but also significantly more expensive. I don't really care much about all the video transcoding and server features modern NAS storages provide. I think my requirements are:
  • 4 bay
  • 1Gb Ethernet (1 have 1Gb network at home) - 2 ports are better
  • Support for M.2 SSD module - I will add it in the future as cache
  • Not sure how performant it should be but most of devices in this category are 4 core and can have up to 8GB RAM
  • It should have USB 3.0
  • It should work with both Windows and Mac

It looks like best match are Synology DS918+ and QNAP TS-423Be although they seem to be bit overkill. They support many more features I don't need.

I would configure it with 4x4TB or 4x6TB WD Red or Seagate IronWolf drives. Even 16TB (about 8TB real storage) should cover me for years.

It is also good to point out, that this is not my business. I'm just enthusiasts and this is my hobby. I never made a penny by photography yet.

What are your experiences with NAS storage and accessing RAW photos from Lightroom over network?

Before someone asks why just not to use use cloud storage?
  • I'm using cloud storage as backup for processed photos which are usually about 10-20% of what I shot.
  • I would still want local copy.
  • I have asymmetric Internet connection - fast downloads, slow uploads. Trying to upload 80GB would take ages and would be best way to end up with fair use policy clause in the contract.

Look into Drobo.  I have 4 (home and office) and love them.

They are reasonably priced, easy to set up, reliable, and scalable.

sek
Cameras: 1DXII,5D III, 5D II.  Lenses    24-70 2.8L II IS, 70-200 f4L IS, 70-200 f2.8L IS II, EF 400 5.6L, 300 2.8 IS II, Samyang 14 mm 2.8.   Flashes: 600EX-RT X 2, ST-E3-RT, 580 EX II.
Plus lots of stuff that just didn't work for me

canon rumors FORUM

Re: NAS storage advice
« Reply #29 on: July 13, 2018, 12:09:49 AM »