November 24, 2017, 07:29:24 PM

Author Topic: Do You Wish Lightroom Was Quicker? Adobe Does Too  (Read 24315 times)

Diko

  • EOS 6D Mark II
  • *****
  • Posts: 395
  • 7 fps...
Re: Do You Wish Lightroom Was Quicker? Adobe Does Too
« Reply #75 on: July 14, 2017, 04:26:43 PM »
I talked with some pros who use Lightroom all the time and asked them about the advantages over Photoshop and when and why I might want to use it. The answers they gave were largely in the "time is money" category, and I could see why they used it, but didn't get any reason why I might. I don't often have that many pictures shot under the same lighting conditions, so my rare batches are just a handful of pictures, easily done in Filmstrip mode of Bridge. And I don't feel the need for the database. Just having pictures made on the same day put into the same folder on the computer is organization enough for me. Then if there are multiple days in one project or trip, I'll put those folders into an enclosing folder topically named.

Scott Kelby flat-out tells people to use Lightroom instead of Bridge and Photoshop, but doesn't much elaborate.

Mostly it is about wokrflow. I don't understand why in "Adobe asks for LR performance feedback" topic actually people began discussing LRvsPS+Bridbge - bad AvTvM.  ;-)


Actually Lr has always been meant to be used as:

1/ Culluing processor (most colleagues who cull on pc instead of doing mostly on the camera, however these days avoid doing it in LR, since it's way too slow for this kind of job).
2/ Catalogue
3/ Developing environment
4/ Publishing tool.

Now having in mind that Camera RAW was invented just for 3/ doesn't make sense. When you add bridge which is amazing if you collaborate with people (don't remember: does it have a server part for LAN workflow?) or within few programs like InDesign or AI it is the right tool for the job.

Now if you shoot event (photos shot are usually in the hundreds), or you prefer to tether-shoot, or just seeing the results prior to last culling with applied corrections on a big screen - LR was supposed to be the tool ;-)

So the whole discussion LR vs PS+Bridbge is stupid. Each of us has his/her own workflow - ergo different tools for his/her job.

« Last Edit: July 14, 2017, 04:29:09 PM by Diko »
Luck is what happens when preparation meets opportunity.

canon rumors FORUM

Re: Do You Wish Lightroom Was Quicker? Adobe Does Too
« Reply #75 on: July 14, 2017, 04:26:43 PM »

tpatana

  • EOS 5DS R
  • ******
  • Posts: 1175
Re: Do You Wish Lightroom Was Quicker? Adobe Does Too
« Reply #76 on: July 14, 2017, 04:38:57 PM »
For me culling on LR is fast enough, but the pain is when I go through edits one by one. Especially the crop/rotate tool takes ages to activate. If I restart LR the first 10-20 are fine, but then it slows down again. That's my #1 fix-wish for the next LR version.

ethanz

  • EOS 80D
  • ****
  • Posts: 267
  • 1DX II
    • my website
Re: Do You Wish Lightroom Was Quicker? Adobe Does Too
« Reply #77 on: July 15, 2017, 02:35:15 AM »
That, and half my shoots I'm dealing with 2k+ images and edit 100+ of them. LR saves lives.

Give 'Photo Mechanic' a try. Blazingly fast on the 'cull' and you can then open them directly into Lightroom, Capture One, PS, and keywording is also excellent.
I personally use it to cull from a folder on my hard drive, allocating 1 star for those to be deleted which I do through Photo Mechanic in one batch at the end of my analysis, this leaves me with the photos I wish to keep.
[/quote]

That's how I 'cull' through my images, make them one star in bridge. Then edit all one stars in ps. Not sure how that is any different from lr or how lr saves time. I also have 2-3000 images to go through. I've been using ps for over ten years so I'm used to the interface. I wouldn't call it atrocious. I guess I just don't know what else is out there.
1DX II, 16-35L f/4 IS, 24-70L f/2.8 II, 70-200L f/2.8 IS II, 200-400L f/4 IS w/1.4 EXT
http://ethanzentz.com/

Diko

  • EOS 6D Mark II
  • *****
  • Posts: 395
  • 7 fps...
Re: Do You Wish Lightroom Was Quicker? Adobe Does Too
« Reply #78 on: July 15, 2017, 07:59:30 AM »
...That, and half my shoots I'm dealing with 2k+ images and edit 100+ of them. LR saves lives...
...Give 'Photo Mechanic' a try. That's how I 'cull' through my images, make them one star in bridge...

For culling I use FastRaw viewer. It really counts ;-) Because as they say: "To Choose Correctly You Need to See a Correct Preview".
« Last Edit: July 15, 2017, 08:01:48 AM by Diko »
Luck is what happens when preparation meets opportunity.

EdelweissPirate

  • PowerShot SX60 HS
  • **
  • Posts: 5
Re: Do You Wish Lightroom Was Quicker? Adobe Does Too
« Reply #79 on: July 15, 2017, 01:20:39 PM »
for light editing work, lr is a really good tool, but the database and associated performance issues are a problem.

Others have attempted to convey this to you without success, so maybe I'm tilting at windmills here:

The association between Lightroom's database and its performance issues exists only in your imagination. It's a mistake to claim that SQLite is an a priori cause of Lightroom's performance issues.

All of the fixes you suggest, especially storing all the metadata in the image headers, would be both slower, uglier and more failure-prone than Lightroom's SQLite approach. I'm not trying to insult you here, but your understanding of programming/computer science is not quite what you seem to think it is.

There's no shame in having an incomplete understanding of these subjects, but you might at least consider that people who do this for a living (including other posters on this forum and the Lightroom developers) have thought about these things and rejected them for good reason.

SQLite (or something very much like it) is the right tool for this particular job, though I'd have preferred a true multi-user database to allow catalogs to be stored on networked drives. Others have already explained why OSes are not databases and why Spotlight and the Windows search/indexing function would be worse than Adobe's use of SQLite. One thing I didn't see mentioned is that if you rely on the OS to build its own database for your photos, switching between Windows and OSX (either way) becomes enormously problematic.

Why would you want to lock yourself into a particular OS simply because your photo editing app relies on that OS's database? Additionally, because your "ideal" OS-database would be entirely different for Mac vs. Windows, Adobe would have to spend time and money developing hooks for two entirely independent database structures. Oh, and if those OS-based databases don't support exactly the same metadata, then your feature set diverges on the two platforms. Speaking generally about DAM software, you really want a database, and you really want it to be completely independent of the OS.

AvTvM, you don't seem to reject the idea of Lightroom using a database, but rather you seem to think that Adobe has chosen the wrong one. I'm going to go out on a limb here and speculate that you think that somehow SQLite is a "big," ponderous database, so you pin all of Lightroom's performance issues on SQLite. SQLite isn't a slow database. It's efficient and fast for small data sets, and no matter how big you think your catalog is, it's not a big data set in the grand scheme of things.

If you don't think that SQLite is a big, slow database, then what's your objection to it?

That's not to say that Lightroom doesn't have issues. It does, but I don't pretend to know the root cause of those problems. For example, as far as I can tell, exporting JPEGs and building 1:1 previews (which are basically the same thing) should be embarrassingly parallel or nearly so. Yet, as the plot Diko posted shows, Lightroom doesn't efficiently use more than four CPUs for these tasks. In other words, building 1:1 previews should take half as much time with eight physical cores as it does with four, but instead it takes nearly the same amount of time.

There's a reason for that, and I don't know what it is. But I doubt that it's because all of the Lightroom developers know less about parallel computing than I do. (I don't know very much).

I'd love to see better performance from Lightroom. I agree that Adobe hasn't taken users' performance complaints seriously, and I agree that there are other workflows that have advantages over Lightroom. But pinning all of your complaints on SQLite only highlights your incomplete understanding of the issues at hand.

Khalai

  • EOS 5D Mark IV
  • ******
  • Posts: 708
  • In the absence of light, darknoise prevails...
Re: Do You Wish Lightroom Was Quicker? Adobe Does Too
« Reply #80 on: July 15, 2017, 01:31:15 PM »
That's not to say that Lightroom doesn't have issues. It does, but I don't pretend to know the root cause of those problems. For example, as far as I can tell, exporting JPEGs and building 1:1 previews (which are basically the same thing) should be embarrassingly parallel or nearly so. Yet, as the plot Diko posted shows, Lightroom doesn't efficiently use more than four CPUs for these tasks. In other words, building 1:1 previews should take half as much time with eight physical cores as it does with four, but instead it takes nearly the same amount of time.

According to Puget Systems, exporting images is about the only thing which goes as much parallel as it can, yielding .97 efficiency, while other tasks are much more problematic.

ActionParallel Efficiency (1 is perfect)
Importing images from USB.00
Exporting images to disk.97
Convert from RAW to DNG.69
Generate 1:1 Previews.77
Generate Smart Previews.51
Create HDR image.60
Create Panorama image.44
Facial Recognition.20
6D | Zeiss 21/2.8 ZE | 24-70/2.8L II | 35/1.4L | 50/1.2L | Zeiss 50/2 ZE Makro | Zeiss 85/1.4 ZE | 70-200/2.8L II

EdelweissPirate

  • PowerShot SX60 HS
  • **
  • Posts: 5
Re: Do You Wish Lightroom Was Quicker? Adobe Does Too
« Reply #81 on: July 15, 2017, 03:09:55 PM »
Yeah, not so much. Puget Systems is calculating not the theoretical parallel efficiency of each task but rather the efficiency of Lightroom as you add processors. Some tasks are inherently parallel while others are inherently serial.

Moreover, Puget Systems is calculating the efficiency only at the first few processors. None of those curves shows Puget Systems' calculated efficiency for long for long. When I say that exporting images and generating previews should be embarrassingly parallel, I mean that the time for the task should fall continuously with the number of processors.

All of the Puget Systems curves flatten out after somewhere between 2-8 cores, depending on the task. I'm suggesting that for generating 1:1 previews and exporting images, the line should have (theoretically) a quadratically decreasing slope.

Exporting images doesn't "go as much parallel as it can," in your words. The speedup is quasi-linear but then drops off dramatically. Their task takes 200 seconds with three cores and about 100 seconds with six cores. So far, so good.  But then twelve cores should take 50 seconds, and their plot makes it look more like 90 seconds. So the parallel efficiently may be 0.97 for the first few processors, but it drops off to almost zero after that.

Realistically, the speedup in Puget Systems' export test may be limited by disk or memory bandwidth, which is understandable. But generating 1:1 previews is (insofar as I understand it) should be at least as parallelizable (so to speak), and yet Lightroom doesn't reflect that. This might be an area where speeding up Lightroom is straightforward, but I don't think we know enough about the problem to say for sure.

canon rumors FORUM

Re: Do You Wish Lightroom Was Quicker? Adobe Does Too
« Reply #81 on: July 15, 2017, 03:09:55 PM »

Khalai

  • EOS 5D Mark IV
  • ******
  • Posts: 708
  • In the absence of light, darknoise prevails...
Re: Do You Wish Lightroom Was Quicker? Adobe Does Too
« Reply #82 on: July 15, 2017, 04:50:37 PM »
Yeah, not so much. Puget Systems is calculating not the theoretical parallel efficiency of each task but rather the efficiency of Lightroom as you add processors. Some tasks are inherently parallel while others are inherently serial.

Moreover, Puget Systems is calculating the efficiency only at the first few processors. None of those curves shows Puget Systems' calculated efficiency for long for long. When I say that exporting images and generating previews should be embarrassingly parallel, I mean that the time for the task should fall continuously with the number of processors.

All of the Puget Systems curves flatten out after somewhere between 2-8 cores, depending on the task. I'm suggesting that for generating 1:1 previews and exporting images, the line should have (theoretically) a quadratically decreasing slope.

Exporting images doesn't "go as much parallel as it can," in your words. The speedup is quasi-linear but then drops off dramatically. Their task takes 200 seconds with three cores and about 100 seconds with six cores. So far, so good.  But then twelve cores should take 50 seconds, and their plot makes it look more like 90 seconds. So the parallel efficiently may be 0.97 for the first few processors, but it drops off to almost zero after that.

Realistically, the speedup in Puget Systems' export test may be limited by disk or memory bandwidth, which is understandable. But generating 1:1 previews is (insofar as I understand it) should be at least as parallelizable (so to speak), and yet Lightroom doesn't reflect that. This might be an area where speeding up Lightroom is straightforward, but I don't think we know enough about the problem to say for sure.

Still useful comparision to a degree. Not many people have more than eight core CPUs back home. I don't think that E5-2699v4/v5 is that prolific in home PCs :)
6D | Zeiss 21/2.8 ZE | 24-70/2.8L II | 35/1.4L | 50/1.2L | Zeiss 50/2 ZE Makro | Zeiss 85/1.4 ZE | 70-200/2.8L II

AvTvM

  • Canon EF 300mm f/2.8L IS II
  • ********
  • Posts: 2940
Re: Do You Wish Lightroom Was Quicker? Adobe Does Too
« Reply #83 on: July 15, 2017, 06:42:46 PM »
@edelweiss: i thought i had made it clear that i dont like both: 1. lr using a database at all and 2. using a grossöy underperforming implementation of a database. i think it is proven beyond reasonable doubt, that LR performance problems - even on powerful hardware - are rooted in its database, since other raw converters/image editors - anything from canon DPP to Capture 1 pro to Adobe Bridge - does NOT have those performance problems.

i am fine with LR's (ACR) raw conversion (even though Canon DPP does an even better job on canon raws), i do like LR user interface and the editing functionality. While limited it is all i need. Only thing i don't like and don't need is the database. so for me the preferred solution would be a well performing "LR lite" with identical raw converter and image editor functionality but without database. i have no problem to find image files in windows thanks to my well thought out and disciplined file and folder naming scheme and structure. firthermore there are apps available to conduct AI-based content specific search for images. no manual image tagging needed. no lightroom database needed. fast, efficient, automatic.
« Last Edit: July 17, 2017, 10:45:53 AM by AvTvM »

LDS

  • EOS 5DS R
  • ******
  • Posts: 1051
Re: Do You Wish Lightroom Was Quicker? Adobe Does Too
« Reply #84 on: July 17, 2017, 05:35:07 AM »
i think it is proven beyond reasonable doubt, that LR performance problems - even on powerful hardware - are rooted in its database, since other raw converters/image editors - anything from canon dpp to capture 1 pro to adobe bridge - does NOT have those performance problems.

Here you can find a description of the LR architecture: http://www.troygaul.com/LrExposedC4.html. it's not up to date, but AFAIK it didn't change much since then.

It's not proven that SQLite is the issue. Your empirical observation is based on the fact that LR employs a database, the others don't, so the culprit must be the database. Unluckily, there are many other not so obvious details to take into consideration - i.e the use of Lua by LR.

SQLite is used in many products (see here https://sqlite.org/famous.html) - if it had so many performance issues, it would have been already replaced by something else.

While I would like LR could also support a stand-alone database (so it could be run on a different server, and several LR clients could connect and work on the same catalog...), it would be much heavier and difficult to install for single use than the one actually used by LR.

Anyway, what is important is to exactly pinpoint where performance issues arise. There are tools to measure quantitatively software performance and identify precisely where bottlenecks are. This is the only way to ensure the proper piece of software is optimized - but it's something only Adobe can do with the proper level of detail.

The fact they're asking user what they believe is a priority will tell them where to start to look at.

Mikehit

  • Canon EF 300mm f/2.8L IS II
  • ********
  • Posts: 2162
Re: Do You Wish Lightroom Was Quicker? Adobe Does Too
« Reply #85 on: July 17, 2017, 05:44:05 AM »
I think there are two things driving this - firstly functionality was given a priority over speed and LR has now reached a stage of rapidly diminishing returns on the functionality. More functions mean more data in xmp files mean more data to handle.
The rise of programs like Photomechanic and Breezebowser mean professionals are using those systems to cull photos but also as digital management because they are so much quicker for their purpose and that is Adobe's next step.

Which actually begs the question of where people want the speed. If they want the speed of review and culling, then there is Photomechanic approach where you could go into a review module where you review the embedded raws for sharpness/composition and the only data you apply are 'delete' flag (and maybe a 'rating') and when you have been through your complete ingest of 5,000 photos you press a button to update the library. Then it goes as it is now.

Some people complain about the time it takes to apply edits and that would be completely different set of issues - but I have occasions where LR is quick and others where it is very slow and I wonder if that is due to other things the computer is doing rather than LR itself.

AvTvM

  • Canon EF 300mm f/2.8L IS II
  • ********
  • Posts: 2940
Re: Do You Wish Lightroom Was Quicker? Adobe Does Too
« Reply #86 on: July 17, 2017, 10:50:49 AM »
I think there are two things ...
... speed of review and culling ...
... time it takes to apply edits ...

I agree. Where I don't agree is, why has MIGHTY Adobe up to today NOT been able (or willing) to provide
1. an IMPORT Module in LR, that is every bit as simple , fast and painless as PhotoMechanic?
2. An EDIT module that performs fast by fully utilizing today's hardware (multi-core/Threaded CPUs, lotsa RAM, blazingly fast NVME SSDs etc.)

My explanation: Adobe has been resting on their laurels or rather on their big fat corporate ass and rather than on product development they are primarily focussed on coercing as many users as possible into their rental subscription scheme and raking in the cash. 

Mikehit

  • Canon EF 300mm f/2.8L IS II
  • ********
  • Posts: 2162
Re: Do You Wish Lightroom Was Quicker? Adobe Does Too
« Reply #87 on: July 17, 2017, 10:58:00 AM »
I think there are two things ...
... speed of review and culling ...
... time it takes to apply edits ...

I agree. Where I don't agree is, why has MIGHTY Adobe up to today NOT been able (or willing) to provide
1. an IMPORT Module in LR, that is every bit as simple , fast and painless as PhotoMechanic?
2. An EDIT module that performs fast by fully utilizing today's hardware (multi-core/Threaded CPUs, lotsa RAM, blazingly fast NVME SSDs etc.)

My explanation: Adobe has been resting on their laurels or rather on their big fat corporate ass and rather than on product development they are primarily focussed on coercing as many users as possible into their rental subscription scheme and raking in the cash.

Alternative reading:
You don't change architecture and add functionality at the same time. They saw functionality as the more important to keep ahead of the competition. Now that is in place the architecture is coming under the spotlight more than it was before.
A company like Adobe does not get to be where it is (and maintain that position) by incompetence or luck - and just because they don't agree with you. or you don't understand their decisions, doesn't mean they don't know what they are doing.

canon rumors FORUM

Re: Do You Wish Lightroom Was Quicker? Adobe Does Too
« Reply #87 on: July 17, 2017, 10:58:00 AM »

LDS

  • EOS 5DS R
  • ******
  • Posts: 1051
Re: Do You Wish Lightroom Was Quicker? Adobe Does Too
« Reply #88 on: July 17, 2017, 11:30:45 AM »
1. an IMPORT Module in LR, that is every bit as simple , fast and painless as PhotoMechanic?

The import module of LR does more - it can generate different types or previews, convert to DNG, apply presets, etc. Photomechanics does less, does it better, and cost as much as LR. It's a much more focused applications - which justify its prices only if you need that focus. Probably when USB 2.0 and spinning disk were the norm, the LR code looked adequate - now with USB 3.0/Thunderbolt and SSD disks, it may be not.

2. An EDIT module that performs fast by fully utilizing today's hardware (multi-core/Threaded CPUs, lotsa RAM, blazingly fast NVME SSDs etc.)

Note than not everybody yet has twelve core, 128G of RAM and NVMe disks :) LR is still a $150 application, often sold in bundle with something else, and could end to be used on far less powerful machines. You may need to find a balance between the system requirements, and performance. If it was aimed only at 7D/5D/1D users (or equivalent cameras from other brands) maybe they could think they have matching PCs - but there's a lot of less expensive cameras users, with less powerful PCs, among the LR users as well.

My explanation: Adobe has been resting on their laurels

It's highly probable - as long as a product sell well enough there are little commercial reasons to invest a lot in modifying it deeply, because there are risks there will be more new bugs to chase and fix, and stability issues (just look at GPU support...). Look at how Canon itself is often conservative in new models - it's the same approach.

When competitors get close, or surpass you, if you're not stupid you understand the time has come for the required changes, and risks are balanced by the risk of losing market share.

AvTvM

  • Canon EF 300mm f/2.8L IS II
  • ********
  • Posts: 2940
Re: Do You Wish Lightroom Was Quicker? Adobe Does Too
« Reply #89 on: July 17, 2017, 11:45:41 AM »
2. An EDIT module that performs fast by fully utilizing today's hardware (multi-core/Threaded CPUs, lotsa RAM, blazingly fast NVME SSDs etc.)
Note than not everybody yet has twelve core, 128G of RAM and NVMe disks :) LR is still a $150 application, often sold in bundle with something else, and could end to be used on far less powerful machines. You may need to find a balance between the system requirements, and performance. If it was aimed only at 7D/5D/1D users (or equivalent cameras from other brands) maybe they could think they have matching PCs - but there's a lot of less expensive cameras users, with less powerful PCs, among the LR users as well.

No, not everyone has a fast PC. But it won't matter with LR anyways. because LR does NOT take any advantage of powerful hardware, when it is there. LR lets it sit and idle ... that's the second aspect of the LR performance problem we are discussing here. The other aspect is the poorly performing, unnecessary database (both concept and implementation). 

There is simply no excuse. Adobe said, with CC cloud stuff and monthly payments will come regular ongoing improvements. Not true. Clear lie.

canon rumors FORUM

Re: Do You Wish Lightroom Was Quicker? Adobe Does Too
« Reply #89 on: July 17, 2017, 11:45:41 AM »