October 21, 2014, 01:11:16 PM

Author Topic: Bottleneck when opening images of CF card?  (Read 2677 times)

tolusina

  • Canon 6D
  • *****
  • Posts: 392
    • View Profile
Re: Bottleneck when opening images of CF card?
« Reply #15 on: November 05, 2013, 11:58:51 AM »
I simply cannot imagine why you would want to open a large collection of file directly from the memory card, maybe open a very few by file name as you've noted on your camera's info screen.
For a large quantity of files, copy them all to the PC first.
---
I'm also getting ready to build a new PC.
I'm figuring on two 256GB internal SSDs, multi layer BluRay discs and 1-2 TB hard drives for back up.
Backup hard drive(s), only one to start with, will slot into one of these.......
http://www.newegg.com/Product/Product.aspx?Item=N82E16817998185
 
I was planning on building in a card reader, didn't care for the number of negative reviews of the ones I found at NewEgg.
Then I found this set up which is Jeff Cable approved......
http://www.bhphotovideo.com/c/search?Ntt=Lexar+HR1+Hub&N=0&InitialSearch=yes&sts=ma&Top+Nav-Search=
Readers can be used individually or up to four can be mixed and/or matched into the hub, contents of up to four cards can be downloaded to your PC simultaneously. Even four of the fastest cards should not be able to saturate the USB3 bus.
---
Back to PC drive configuration.
I briefly considered some RAID possibilities, researched a bit and decided not.
Here's my plan...
Install one SSD, install Window 7, maybe partition the drive into halves for dual booting, 128GB should be more than plenty for my non-gaming OS and programs. My current XP with about 70 programs lives on a 50GB partition that still has 1/3 free space.
Windows up and running, second half of the primary drive left un-formatted, Shut down, connect second drive, reboot.
Use Windows Disk Management to format (and partition if desired) the second physical drive, create a single folder in the root of that fresh E(?) drive, call that folder something clever like eMyDocuments.

In XP, I'm hoping Win 7 is similar, right click on the "My Documents" folder on the desktop, select >Properties.
In the Properties tab is a window where you can specify "My Documents" target location, type e:\eMyDocuments or simple select the browse tab. Windows should ask if you want to move/copy/migrate/whatever the contents of the current My Documents, choose YES!! My Photos, My Music and all those will move right over along with each folder's special properties.

Windows has now "symbolically linked" or "mapped" your "My Documents" to your second SSD, since the default save location for most programs is My Documents, those programs will now automatically save to you second drive.
 
The idea here is for the OS and programs to be on one physical drive, data on another, each can be backed up separately.
 
At this point, I sure don't know exactly how DPP and LightRoom will want to play with this set up, when offered options, I'll aim ALL data files to the second SSD.
 
The external drive dock will be for hot swapping in bare HDDs or SSDs as I choose for back ups.
 
 

 
.
40 on 6

canon rumors FORUM

Re: Bottleneck when opening images of CF card?
« Reply #15 on: November 05, 2013, 11:58:51 AM »

vlad

  • PowerShot G1 X II
  • ***
  • Posts: 48
    • View Profile
Re: Bottleneck when opening images of CF card?
« Reply #16 on: November 05, 2013, 02:59:46 PM »
Here's my plan...
Install one SSD, install Window 7, maybe partition the drive into halves for dual booting, 128GB should be more than plenty for my non-gaming OS and programs. My current XP with about 70 programs lives on a 50GB partition that still has 1/3 free space.
Windows up and running, second half of the primary drive left un-formatted, Shut down, connect second drive, reboot.
Use Windows Disk Management to format (and partition if desired) the second physical drive, create a single folder in the root of that fresh E(?) drive, call that folder something clever like eMyDocuments.

In XP, I'm hoping Win 7 is similar, right click on the "My Documents" folder on the desktop, select >Properties.
In the Properties tab is a window where you can specify "My Documents" target location, type e:\eMyDocuments or simple select the browse tab. Windows should ask if you want to move/copy/migrate/whatever the contents of the current My Documents, choose YES!! My Photos, My Music and all those will move right over along with each folder's special properties.

Windows has now "symbolically linked" or "mapped" your "My Documents" to your second SSD, since the default save location for most programs is My Documents, those programs will now automatically save to you second drive.
 
The idea here is for the OS and programs to be on one physical drive, data on another, each can be backed up separately.

A word of warning.  I built my current Win7 system such that the OS is on an SSD and the Users folder is on a 3TB Raid1 HDD.  I used audit mode to set it up (google audit mode user profile for more info).  The system works fine, but it won't allow an upgrade to Win8.  It's just not a supported configuration.  Which is a bummer, because I actually quite like Win8, and would love to upgrade this machine instead of having to reinstall.

If you are building a new Win7 or Win8 system and want good separation of your OS and data, just use the Libraries feature.  You can create your documents folder on your external HD, or Raid1, or whatever, right click it, Include in Library, and you're set.

JPAZ

  • Canon 7D MK II
  • *****
  • Posts: 566
  • If only I knew what I was doing.....
    • View Profile
Re: Bottleneck when opening images of CF card?
« Reply #17 on: November 05, 2013, 04:00:36 PM »
Interesting.  I, too, am working on a new machine.  My plan is to have an SSD for the OS, Software and Lightroom Catalog as well as for the PS "scratch" drive (what we used to call virtual RAM).  I presently have 2 1TB drives in a RAID 1 (mirror) configuration but I am planning on two 3 TB drives in the new machine.   Over the years, the mirror RAID has saved my hide more than once, when a single HDD failed.  All I needed to do was replace the faulty drive with a new one and the mirror rebuilt itself. 

But, the RAID setup does not replace a backup.  If both drives fail or if the computer / motherboard fails or if the house burns down (you get the idea) you can lose everything even though the drive is self-duplicated.  Being a big believer in the belt and suspenders approach, I also have a continuous backup of my photos to the "cloud" (I use Crashplan) and do periodic backups to an external HDD, as well.  The initial synch of my very large collection of phot files to the cloud took a while, but now it runs continuously in the background,  I'll continue this plan with my new PC.

I use Win 7 Pro 64 bit and will continue to do this.  Be sure to get as much RAM as you can (at least 16 GB).  Another thought.....don't skimp on the cooling.  Heat is one of the biggest dangers to the internals of a computer.  Fans are cheap and most cases give lots of options for cooling fans.
5d Mkiii; Eos-M; too many lenses; 430 EXii and a whole lot of stuff

Drizzt321

  • 1D X
  • *******
  • Posts: 1674
    • View Profile
    • Aaron Baff Photography
Re: Bottleneck when opening images of CF card?
« Reply #18 on: November 05, 2013, 04:33:39 PM »
Interesting.  I, too, am working on a new machine.  My plan is to have an SSD for the OS, Software and Lightroom Catalog as well as for the PS "scratch" drive (what we used to call virtual RAM).

Are you sure you don't mean Virtual Memory? Where the OS automatically overflows to disk? Or are you referring to something specific in Photoshop where it'd use a specific drive for data?
5D mark 2, 5D mark 3, EF 17-40mm f/4L,  EF 24-105mm f/4L IS USM, EF 135mm f/2L, EF 85mm f/1.8
Film Cameras: Mamiya RB67, RB-50, RB-180-C, RB-90-C, RB-50, Perkeo I folder, Mamiya Six Folder (Pre-WWII model)
http://www.aaronbaff.com

JPAZ

  • Canon 7D MK II
  • *****
  • Posts: 566
  • If only I knew what I was doing.....
    • View Profile
Re: Bottleneck when opening images of CF card?
« Reply #19 on: November 05, 2013, 05:48:35 PM »
"Virtual RAM" and Virtual Memory are the same thing, I think.  That is when the PC uses the hard drive as RAM because the physical RAM is otherwise occupied.
5d Mkiii; Eos-M; too many lenses; 430 EXii and a whole lot of stuff

pwp

  • 1D X
  • *******
  • Posts: 1605
    • View Profile
Re: Bottleneck when opening images of CF card?
« Reply #20 on: November 05, 2013, 05:56:07 PM »
OK, IT guy here.  Sorry for the long post but I hope this advice helps.

(trimmed)

Finally, don't use this nice fast photo pc to do everything.  If you install a bunch of stuff like QuickBooks,  Office, HP Printer software, huge antivirus security packages, etc, you will kill the performance.  Just use it for photography.  If you don't use it to surf porn, play online games or do your online banking, don't install antivirus or just install the perfectly fine Microsoft Antivirus.  Seriously, you'll thank me later.

Thanks RustyTheGeek, that's a very fine, generous post. Home PC builders should bookmark this one.

Your final comment is an area that is too often overlooked. My photo PC's do just that...all that's installed are the required Adobe CC components, PhotoMechanic, Firefox, Microsoft Security Essentials and a handful of required utilities.

All the Office, MYOB, iTunes and basically unnecessary entertaining stuff is on a separate modestly powered machine, in this case an entry-level Mac Mini running OSX Mavericks and Windows 7 installed via VM Ware.

-pw


Drizzt321

  • 1D X
  • *******
  • Posts: 1674
    • View Profile
    • Aaron Baff Photography
Re: Bottleneck when opening images of CF card?
« Reply #21 on: November 05, 2013, 06:25:17 PM »
"Virtual RAM" and Virtual Memory are the same thing, I think.  That is when the PC uses the hard drive as RAM because the physical RAM is otherwise occupied.

Ah, you were using the terms interchangeably. I've never really heard the term Virtual RAM associated with it since it's not the appropriate industry/technical term. At least not that I'm familiar with.
5D mark 2, 5D mark 3, EF 17-40mm f/4L,  EF 24-105mm f/4L IS USM, EF 135mm f/2L, EF 85mm f/1.8
Film Cameras: Mamiya RB67, RB-50, RB-180-C, RB-90-C, RB-50, Perkeo I folder, Mamiya Six Folder (Pre-WWII model)
http://www.aaronbaff.com

canon rumors FORUM

Re: Bottleneck when opening images of CF card?
« Reply #21 on: November 05, 2013, 06:25:17 PM »

RustyTheGeek

  • Buy and Sell
  • 1D Mark IV
  • ********
  • Posts: 917
    • View Profile
    • Images I've Shot...
Re: Bottleneck when opening images of CF card?
« Reply #22 on: November 05, 2013, 07:04:16 PM »
OK, IT guy here.  Sorry for the long post but I hope this advice helps.

(trimmed)

Finally, don't use this nice fast photo pc to do everything.  If you install a bunch of stuff like QuickBooks,  Office, HP Printer software, huge antivirus security packages, etc, you will kill the performance.  Just use it for photography.  If you don't use it to surf porn, play online games or do your online banking, don't install antivirus or just install the perfectly fine Microsoft Antivirus.  Seriously, you'll thank me later.

Thanks RustyTheGeek, that's a very fine, generous post. Home PC builders should bookmark this one.

Your final comment is an area that is too often overlooked. My photo PC's do just that...all that's installed are the required Adobe CC components, PhotoMechanic, Firefox, Microsoft Security Essentials and a handful of required utilities.

All the Office, MYOB, iTunes and basically unnecessary entertaining stuff is on a separate modestly powered machine, in this case an entry-level Mac Mini running OSX Mavericks and Windows 7 installed via VM Ware.

-pw

Thanks for the kind words pwp.  Yes, these days hardware is cheap.  People sometimes don't realize the benefits of having more than one physical system.  Esp if they have kids!!  Another option is running a separate VM system.  I've been doing this for over 20 years and it still amazes me how poorly written most software is.  It's actually much worse than it was even 10 years ago.  Lazy or overworked programmers and unrealistic deadlines and/or not much talent.  It's sad but this is just the poor state of affairs most software companies are in these days.  Look at how long it  has taken to get a version of Windows out in the past 10 years and then look what we ended up with!  And Adobe regularly releases buggy code.

Most of the time it pays to keep things as simple as possible.
Yes, but what would  surapon  say ??  :D

canon rumors FORUM

Re: Bottleneck when opening images of CF card?
« Reply #22 on: November 05, 2013, 07:04:16 PM »