September 02, 2014, 07:35:34 AM

Author Topic: LR5 + Hyper Threading Slows Down Exporting  (Read 2245 times)

celltech

  • PowerShot G1 X II
  • ***
  • Posts: 46
    • View Profile
LR5 + Hyper Threading Slows Down Exporting
« on: June 15, 2013, 04:15:09 PM »
Just upgraded to LR5 and have been reading abut the supposed speed increases and what not.  It got me thinking about my own machines performance and how to make it faster.  Back when I had LR3 and a Core 2 Q6600 processor (no HT) I remember an export action would make the CPU jump between 50-100%.  I then upgraded to a Sandy Bridge 2500K (no HT either) and the same CPU usage was observed, although I did time the difference and the 2500K was *much* faster.
 
I have since moved to an Ivy Bridge 3770K (with HT) and noticed the CPU would run at 40-70% and didn't think much of it.  I never did a comparison between it and the 2500K.  But I was just playing with what I have and notice that with HT off an export can be much faster.
 
My test involved exporting 205 DNG files created from a 5D3 CR2 import.  I tried doing a single export of all 205 at once and splitting the batch into 2 export actions run together.  I used ThrottleStop to monitor the CPU wattage and core usage.  My results:
 
No HT:
 
All 205 at once:          7:34     50-100% Usage     40-55 Watts
Split into 2 exports:     6:45     90-100% Usage     46-57 Watts
 
HT Enabled:
 
All 205 at once:          9:00     40-70% Usage     40-55 Watts
Split into 2 exports:     6:40     75-100% Usage     56-61 Watts
 
 
Anyone else noticed this type of behavior?   It's easy enough to split a large export...but still, It should not be that much different.  My system specs:
 
Win 8 Pro x64
i7 3770K Overclocked to 4.5Ghz
16Gb Ram
Samsung 830 256Gb SSD
Bodies - 5D3, T2i  Primes - 135 f/2 L, 85 f/1.8, 50 f/1.4, 40 f/2.8, 35 f/2  Zooms - 70-200 f/2.8 L, 70-200 f/4 L, 24-104 f/4 L, 17-55 f/2.8, 17-40 f/4 L

canon rumors FORUM

LR5 + Hyper Threading Slows Down Exporting
« on: June 15, 2013, 04:15:09 PM »

Mt Spokane Photography

  • Canon EF 600mm f/4L IS II
  • ********
  • Posts: 8481
    • View Profile
Re: LR5 + Hyper Threading Slows Down Exporting
« Reply #1 on: June 15, 2013, 04:21:31 PM »
I haven't noticed it being faster or slower, but your measurement is certainly going to be more scientific than my perception.
 
Exporting to jpeg's has always been slow, particularly for several hundred images.  I don't export to DNG.
 
Thanks for the info, post it in the Adobe forums where someone at Adobe might have some insite.

celltech

  • PowerShot G1 X II
  • ***
  • Posts: 46
    • View Profile
Re: LR5 + Hyper Threading Slows Down Exporting
« Reply #2 on: June 15, 2013, 04:39:19 PM »
I had LR import my CR2 files and convert them to DNG before I do my edits.  All my exports are into JPG.

I guess my point for this forum is do you have a CPU that has Hyper Threading, and if you do, does an export result in all CPU resources being utilized?  You can watch the taskmgr or whatever Mac calls their version and see what is going on.

For you pros time is money and it could be useful to take a look...
Bodies - 5D3, T2i  Primes - 135 f/2 L, 85 f/1.8, 50 f/1.4, 40 f/2.8, 35 f/2  Zooms - 70-200 f/2.8 L, 70-200 f/4 L, 24-104 f/4 L, 17-55 f/2.8, 17-40 f/4 L

Mt Spokane Photography

  • Canon EF 600mm f/4L IS II
  • ********
  • Posts: 8481
    • View Profile
Re: LR5 + Hyper Threading Slows Down Exporting
« Reply #3 on: June 15, 2013, 05:45:26 PM »
I tried it with LR 4 and LR 5 while exporting to my Samsung 840 Pro 512 GB Drive, and LR 4 was definitely working the processor harder, I did not time it, but it seemed to go faster.
 
I'd say that I can confirm your observation, LR5 is slower to export (from CR2 to jpegs in my case).

expatinasia

  • 1D Mark IV
  • ******
  • Posts: 901
    • View Profile
Re: LR5 + Hyper Threading Slows Down Exporting
« Reply #4 on: June 15, 2013, 09:47:50 PM »
I may be wrong but your CPU will only play a certain part in the speed. Your graphics card also plays a very important role, and from memory LR does not make use of CUDA, unlike something like Vegas Pro which does, and that cuts rendering times down a lot.
1D X + backup + different L lenses etc.

Mt Spokane Photography

  • Canon EF 600mm f/4L IS II
  • ********
  • Posts: 8481
    • View Profile
Re: LR5 + Hyper Threading Slows Down Exporting
« Reply #5 on: June 15, 2013, 10:02:47 PM »
I may be wrong but your CPU will only play a certain part in the speed. Your graphics card also plays a very important role, and from memory LR does not make use of CUDA, unlike something like Vegas Pro which does, and that cuts rendering times down a lot.

Graphics cards for stills play a minor part.  In any event, I ran them on the exact same hardware, so in that regard, it ran faster with Lightroom 4.  I happen to currently have both installed, and my LR 4 catalog was copied and upgraded for LR5, so its a reasonably good comparison.

celltech

  • PowerShot G1 X II
  • ***
  • Posts: 46
    • View Profile
Re: LR5 + Hyper Threading Slows Down Exporting
« Reply #6 on: June 15, 2013, 10:37:39 PM »
It's my understanding that only Photoshop can us a video card for speeding up certain functions.  I have seen numerous Adobe forum posts that say they never figured out a way to put the LR rendering pipeline into CUDA.

I wish I had compared LR4 to LR5.  I almost want to put LR4 back on my machine just to compare speed.  All I can say is that with both releases, and a HT enabled CPU, exporting a single batch never pegged the cores to anything approaching 100%.
« Last Edit: June 16, 2013, 07:50:14 AM by celltech »
Bodies - 5D3, T2i  Primes - 135 f/2 L, 85 f/1.8, 50 f/1.4, 40 f/2.8, 35 f/2  Zooms - 70-200 f/2.8 L, 70-200 f/4 L, 24-104 f/4 L, 17-55 f/2.8, 17-40 f/4 L

canon rumors FORUM

Re: LR5 + Hyper Threading Slows Down Exporting
« Reply #6 on: June 15, 2013, 10:37:39 PM »

Otara

  • PowerShot G1 X II
  • ***
  • Posts: 37
    • View Profile
Re: LR5 + Hyper Threading Slows Down Exporting
« Reply #7 on: June 15, 2013, 11:34:55 PM »
Have you tried running it without your CPU overclocked?

Its a somewhat puzzling result, although not too meaningful unless doing massive numbers.

Otara

Chewngum

  • PowerShot G1 X II
  • ***
  • Posts: 61
  • F1.6 is the best.
    • View Profile
    • Peter Kingsley Photography
Re: LR5 + Hyper Threading Slows Down Exporting
« Reply #8 on: June 16, 2013, 01:01:26 AM »
In LR4 I get 100% CPU utilisation while an image is being processed with sharp troughs when a new image starts being processed. It has anvaerage of 70% utililisation over a long time period and has 49 threads. This was a single batch export, I did not notice any change by doing mutliple exports though. This was done on a 6-core AMD 1090t without RAM or HDD bottlenecks and going from .cr2(5d3) to JPEG MAX.
Cameras: 5d2, 5d3
Lenses: 17-40L 24-105L 50L 100L s35A s85 70-200L IS II + lots of flashes. About 100 previous bodies/lenses

Chewngum

  • PowerShot G1 X II
  • ***
  • Posts: 61
  • F1.6 is the best.
    • View Profile
    • Peter Kingsley Photography
Re: LR5 + Hyper Threading Slows Down Exporting
« Reply #9 on: June 16, 2013, 01:07:21 AM »
Actually just did it again and noticed a jump from 70% to 93% utilisation but only fro 49 threads to 55 threads which i assume is 1 more image(thread) per core since I have six cores.
Cameras: 5d2, 5d3
Lenses: 17-40L 24-105L 50L 100L s35A s85 70-200L IS II + lots of flashes. About 100 previous bodies/lenses

celltech

  • PowerShot G1 X II
  • ***
  • Posts: 46
    • View Profile
Re: LR5 + Hyper Threading Slows Down Exporting
« Reply #10 on: June 16, 2013, 06:58:15 PM »
Now with fun pictures to show what I see.  I ran 2 tests, both with and without HT.  I started with 1 export and worked my way up to 4 running at the same time.  The red lines are roughly when I started the 2nd, 3rd and 4th export.  After the 2nd one about all that changes is the memory usage.  I did post up on the Adobe forums but nobody seems interested to answer back.

I think that I was most interested in how with HT the overall CPU usage by LR is still lower than without it....even with multiple exports running.  The pictures were generated with Process Explorer x64 and represent just the LR CPU usage.
« Last Edit: June 16, 2013, 07:00:49 PM by celltech »
Bodies - 5D3, T2i  Primes - 135 f/2 L, 85 f/1.8, 50 f/1.4, 40 f/2.8, 35 f/2  Zooms - 70-200 f/2.8 L, 70-200 f/4 L, 24-104 f/4 L, 17-55 f/2.8, 17-40 f/4 L

quartzie

  • Power Shot G16
  • **
  • Posts: 24
    • View Profile
Re: LR5 + Hyper Threading Slows Down Exporting
« Reply #11 on: June 16, 2013, 09:45:23 PM »

I have since moved to an Ivy Bridge 3770K (with HT) and noticed the CPU would run at 40-70% and didn't think much of it.  I never did a comparison between it and the 2500K.  But I was just playing with what I have and notice that with HT off an export can be much faster.
 

You need to understand that even with Hyperthreading the CPU only has the same number of full processing cores as without it. Hyperthreading gives you the ability to re-use some parts of each core which are not being utilized by the thread running on that core - the CPU virtually splits each core into 2, but one of them cannot perform as fast.

This may help when you have a very high number of active threads, but when your actual threads do not exceed the number of HW CPU cores, Hyperthreading is NOT going to help.

What Adobe might need to do is some optimization to run primarily on the "real" cores and jump to HT only when you run a large number of concurrent processes.

That said, since you already have a quad-core, HT wasn't going to help much as export threads in LR remain mostly linear. You may see some improvement if you run, e.g. 4-8 exports at once, but that's probably going to tax other parts as well.

docholliday

  • EOS M2
  • ****
  • Posts: 186
    • View Profile
Re: LR5 + Hyper Threading Slows Down Exporting
« Reply #12 on: June 16, 2013, 11:15:46 PM »
Cache-thrashing is the cause and usually in combination with over scheduling the processor resources (too many software threads for available hardware threads, flooding the processor cache). If the data being loaded on one thread fills a processor cache, and the other thread requests a line read, then the cache gets dumped, loaded for the second one. Then, when the first core requests a line, the cache gets dumped again and the first thread's data gets reloaded from RAM (much slower than from L2/L3 cache). The excessive bus traffic and core context switching further affects the slowdowns. LR has been guilty of that many times, I've had exports that just dragged on when I had other apps working and fighting for proc resources.

I wrote an app that has it's own image processing and experienced quite a few thrashes in multi threaded operation using the thread pool until I split the parallel operations up in a method that takes into advantage not flooding the cache when one thread loads data. It sped up dramatically.

It's monitored using the performance monitor via a set of the 'cache' counters and some of the 'processor' counters.

canon rumors FORUM

Re: LR5 + Hyper Threading Slows Down Exporting
« Reply #12 on: June 16, 2013, 11:15:46 PM »