Lightroom slows down with big catalog?

[tpatana]

I'm wondering, does lightroom slow down when the catalog file gets big? Should I create second one?

Just checked, the catalog.lrcat file is 5.41GB, and the catalog preview.lrdata folder is 13.4GB.

Especially when working with folders with lot of pics (>3k), it seems LR is slower than it should be on this PC.

 

[privatebydesign]

Setting LR up can be tricky and it took me several configurations before I was happy, I am running a very modest Mac Mini i5 with two internal drives, I have my actual image files on a third, external, drive. My boot volume is a 100GB SSD with OS and all applications on it, it is kept clean and has 70GB free space, 16GB of memory.

My LR cache is set to 40GB on the second internal HDD (a 1TB 7200 2.5" HDD) and it is using 19 of them, my Catalog is 378GB and is also on the second internal and includes Smart Previews which are set to Discard:Never (setting up Smart Previews needs lots of space and several days!). My actual image files on the external are 1.4TB and it is connected via FW800, it contains in excess of 70,000 current images.

On this modest setup LR runs very nicely with no lag between panels, folders, collections or single images, even when set to display all 70,000+ images. I actually got the Mini as a stopgap when I sent a desktop in for repairs, it turned out to be a brilliant stand alone photography computer and has never made it to the intended long term use of the home media server.

I clone everything and have auto backups etc disabled.

Edit: I don't make Smart Previews that was an error, just regular ones, I found Smart Previews slowed everything down too much.

 

[tpatana]

Setting LR up can be tricky and it took me several configurations before I was happy, I am running a very modest Mac Mini i5 with two internal drives, I have my actual image files on a third, external, drive. My boot volume is a 100GB SSD with OS and all applications on it, it is kept clean and has 70GB free space, 16GB of memory.

My LR cache is set to 40GB on the second internal HDD (a 1TB 7200 2.5" HDD) and it is using 19 of them, my Catalog is 378GB and is also on the second internal and includes Smart Previews which are set to Discard:Never (setting up Smart Previews needs lots of space and several days!). My actual image files on the external are 1.4TB and it is connected via FW800, it contains in excess of 70,000 current images.

On this modest setup LR runs very nicely with no lag between panels, folders, collections or single images, even when set to display all 70,000+ images. I actually got the Mini as a stopgap when I sent a desktop in for repairs, it turned out to be a brilliant stand alone photography computer and has never made it to the intended long term use of the home media server.

I clone everything and have auto backups etc disabled.

Hmm, interesting.

Your catalog-file is 378GB? Is there some setting that makes it bigger and thus faster? Is it that smart preview?

Ah, I found the setting for preview.

I have:

size: 1440 pix
quality: medium
discard: after 30 days

Smart previews: total size 0 bytes (so I guess it's off then)

If I change the size/quality, would that improve the speed? It's especially slow if the folder has 3k+ pics, and I jump between pics. Often takes several seconds to do anything. And my edit drive is really fast.

After editing I copy to external drive, and then clone to another external.

 

[tpatana]

Sounds like this setting might be the one I'm looking for:

1:1: These previews are a 100% view of actual pixels and, like Standard previews, the Camera Raw engine processes them. When Lightroom generates 1:1 previews, it also generates minimal and standard previews, so all three are available to the program as needed. Because so much data is being processed, 1:1 previews can take a significant amount of time to create. Any time you zoom to 1:1 or higher in the Library module, Lightroom uses 1:1 previews.

To display and work with photos, Lightroom requires a standard or 1:1 preview, depending on the task. If, upon import, you only tell Lightroom to generate Minimal or Embedded previews, Lightroom creates Standard and 1:1 previews automatically as you’re working in the application. This process hinders performance. To increase your productivity and reduce this disruption, manage when and how you render your 1:1 previews. Render them on import, or set aside time to render them manually.

To render 1:1 previews on import, use the File Handling panel of the import window. Choose Render Previews > 1:1. Although generating high-quality, 1:1 previews on import slows the import process, it makes Lightroom more responsive when you start to work in the Library module.

An alternative, if you want a speedier import process, is to render minimal or standard previews on import. Then, at any time, select multiple photos in the Grid view of the Library module and choose Library > Previews > Render 1:1 Previews. Let Lightroom process the images before you start to work on them.

 

[privatebydesign]

There are several things to do.

First, make sure your cache is not on your boot volume, it is under Preferences:File Handling, make it a good size too (mine is 40GB).

Under Catalog Settings:File Handling I have set 2048 pixels/High/Never, but your Preview folder will get very big (as I said mine is over 378GB).

LR wastes a lot of time and energy making previews, if you set t discard then it has to make them again on the fly.

I made a mistake on my first answer, I do not make Smart Previews, just regular ones, the key is they are never thrown away so they are always available to see.

If you do a big import and it slows down just pause the Building Previews icon, when you are ready to let it rip just go Libraryreviews:Build Standard-Sized Previews. I normally import and go make a coffee and come on CR for a play and let the program do its thing, then get back to work, this works for me either first thing in the morning if I got in late or last thing at night so I can leave the computer on to build everything after a big shoot and it is ready first thing.

 

[ExodistPhotography]

My LR cache is set to 40GB on the second internal HDD (a 1TB 7200 2.5" HDD) and it is using 19 of them, my Catalog is 378GB and is also on the second internal and includes Smart Previews which are set to Discard:Never (setting up Smart Previews needs lots of space and several days!). My actual image files on the external are 1.4TB and it is connected via FW800, it contains in excess of 70,000 current images.

On this modest setup LR runs very nicely with no lag between panels, folders, collections or single images, even when set to display all 70,000+ images. I actually got the Mini as a stopgap when I sent a desktop in for repairs, it turned out to be a brilliant stand alone photography computer and has never made it to the intended long term use of the home media server.
......

378 "GB"? <-- Wondering about the GB part.. LOL
My cat is just 879 "MB" with currently 15,000+ images. 4.49GB in preview data..

BTW, also running a Mac Mini. L2012 i7 2.6Ghz Quad, 8GB DRAM, 256GB Samsung 830series SSD + 1TB Factory HDD (In a 1.25TB fusion array). If ONLY I had a better iGPU then this HD4000 mess......
I have a 2TB WD MyBook Mac for my main photos and youtube files, then a 3TB WD MyBook Mac as my Time Machine backup for it all.

 

[tpatana]

There are several things to do.

First, make sure your cache is not on your boot volume, it is under Preferences:File Handling, make it a good size too (mine is 40GB).

Under Catalog Settings:File Handling I have set 2048 pixels/High/Never, but your Preview folder will get very big (as I said mine is over 378GB).

LR wastes a lot of time and energy making previews, if you set t discard then it has to make them again on the fly.

I made a mistake on my first answer, I do not make Smart Previews, just regular ones, the key is they are never thrown away so they are always available to see.

If you do a big import and it slows down just pause the Building Previews icon, when you are ready to let it rip just go Libraryreviews:Build Standard-Sized Previews. I normally import and go make a coffee and come on CR for a play and let the program do its thing, then get back to work, this works for me either first thing in the morning if I got in late or last thing at night so I can leave the computer on to build everything after a big shoot and it is ready first thing.

Is that the "camera raw cache setting"? It's currently set at 1GB

But my boot drive is the fastest drive by far (Intel 750 series), I think read/write speeds are ~5-6x compared to my "normal" SSD which also is fairly good. So why I shouldn't have it on the boot drive?

The folder I was just editing, I've had it open for days now so shouldn't have to wait longer than that? But all pics in the folder are total 123GB, so maybe it's choking on that? I can try increasing that cache, assuming that's the correct one.

 

[privatebydesign]

Yep, the 'camera raw cache setting' is the right one. I set mine to 40GB after looking at what it wanted over time.

I don't know why it shouldn't be on the boot/application drive, I was just told it shouldn't be, I think it was to do with LR's limited ability to multitask and the need to constantly read/write different actions to the same disc. My boot disc is my fastest too, but I still gained a lot of performance when I moved the cache to a different drive and made it a decent size.

There is no reason for it to choke on a 123GB folder unless it is trying to rewrite previews on the fly, this will kill performance.

Here are some screenshots of my setup.

First is my entire catalog folder.
Second is my lrcat at just under 1GB.
Third is lrdata (previews) at 377GB.
Fourth is the 'camera raw cache setting' options.
Fifth is the Catalog Settings:File Handling options.

 

[Mt Spokane Photography]

I was comparing settings. I have no issues with the cache at 10 GB on my SSD boot drive. Its set to 10GB, which is What Adobe recommended, but my previews are medium.

I've moved my cache to a separate drive and set it to 40GB. The only issue I have is that it takes too long to update the folder list in the left panel, and exports lock up other functions.

 

[privatebydesign]

I was comparing settings. I have no issues with the cache at 10 GB on my SSD boot drive. Its set to 10GB, which is What Adobe recommended, but my previews are medium.

I've moved my cache to a separate drive and set it to 40GB. The only issue I have is that it takes too long to update the folder list in the left panel, and exports lock up other functions.

I only went to 40GB cache after watching what it 'wanted', it is currently using 19GB and it is always good to leave plenty of headroom, I currently do zero video.

The folder list takes a second, maybe just under two, to repopulate after a big file move, don't know if that is within spec but considering it is moving tens of GB around on a FW800 linked drive I wasn't fussed.

As for Export, if you are going to export lots of files do it in smaller groups, they will all run at the same time and it will do the entire amount much faster. If you are going to export 200 files break it down to four lots of 50, it isn't quite four times the speed but it is much faster than one lot of 200. Though my Export doesn't lock up anything else.

I am not saying my setup is perfect, but for my drive and processing configuration I am able to work very effectively with zero hang time on a comparatively modest computer.

 

[privatebydesign]

My LR cache is set to 40GB on the second internal HDD (a 1TB 7200 2.5" HDD) and it is using 19 of them, my Catalog is 378GB and is also on the second internal and includes Smart Previews which are set to Discard:Never (setting up Smart Previews needs lots of space and several days!). My actual image files on the external are 1.4TB and it is connected via FW800, it contains in excess of 70,000 current images.

On this modest setup LR runs very nicely with no lag between panels, folders, collections or single images, even when set to display all 70,000+ images. I actually got the Mini as a stopgap when I sent a desktop in for repairs, it turned out to be a brilliant stand alone photography computer and has never made it to the intended long term use of the home media server.
......

378 "GB"? <-- Wondering about the GB part.. LOL
My cat is just 879 "MB" with currently 15,000+ images. 4.49GB in preview data..

BTW, also running a Mac Mini. L2012 i7 2.6Ghz Quad, 8GB DRAM, 256GB Samsung 830series SSD + 1TB Factory HDD (In a 1.25TB fusion array). If ONLY I had a better iGPU then this HD4000 mess......
I have a 2TB WD MyBook Mac for my main photos and youtube files, then a 3TB WD MyBook Mac as my Time Machine backup for it all.

As you can see from my screenshot, my catalog folder is 378GB, the previews are 377GB with 70,000+ images.

With your setup I'd stop the 'fusion drive' part, which I believe makes the two drives effectively one drive, I'd put the boot and apps to the SSD and run the catalog folder on the 1TB. Which is exactly what I did myself and the speeds are much better with effectively no hang time. The key is to have enough space for a full preview folder and never throw previews away so LR doesn't have to generate them on the fly.

Your computer is higher spec than mine and you have less images, i7 vs i5 and 15,000+ images vs 70,000+ images, you should have no hangups or preview delays.

 

[Mt Spokane Photography]

I was comparing settings. I have no issues with the cache at 10 GB on my SSD boot drive. Its set to 10GB, which is What Adobe recommended, but my previews are medium.

I've moved my cache to a separate drive and set it to 40GB. The only issue I have is that it takes too long to update the folder list in the left panel, and exports lock up other functions.

I only went to 40GB cache after watching what it 'wanted', it is currently using 19GB and it is always good to leave plenty of headroom, I currently do zero video.

The folder list takes a second, maybe just under two, to repopulate after a big file move, don't know if that is within spec but considering it is moving tens of GB around on a FW800 linked drive I wasn't fussed.

As for Export, if you are going to export lots of files do it in smaller groups, they will all run at the same time and it will do the entire amount much faster. If you are going to export 200 files break it down to four lots of 50, it isn't quite four times the speed but it is much faster than one lot of 200. Though my Export doesn't lock up anything else.

I am not saying my setup is perfect, but for my drive and processing configuration I am able to work very effectively with zero hang time on a comparatively modest computer.

I don't see any difference moving the cache to the 2nd drive and increasing it to 40 GB.

The file list is slow when I open LR. I never move files around, so I am only looking at the catalog. I do not pull up old images except rarely, so saving the previews is not a benefit to me, but I changed that as well.

My SSD is 2/3 full now, so I am going to clean some things up. Its a 500 GB SSD, so I could increase it to 1TB. They are cheap now. Usually, I wait until I replace the computer about every 1-2 years and get the larger one then. My current one is 1 year old, I've been thinking of a new dell XPS 8900, with I7 about $6XX thru my company. I have 6 pc's in use, the older ones are getting pretty out of date, and noticeably slower even though they have SSD's. Its the newer software that really hogs the cpu's, the computer itself has not gotten slower.

 

[privatebydesign]

I do reference old files all the time, so for me Delete:Never has made a big difference. But when I first did the 'build previews' thing it took four days of LR working 24hours to generate them all, I saw no difference in performance until I had done that and reopened.

But like I said, my setup has essentially been optimised for my setup, both hardware and my program use, and I am sure others will have equally good results going another route. What I like about my setup is it is a reasonably large sized catalog and very modest computer and yet it works fast with no hangups.

 

[tpatana]

I don't know why it shouldn't be on the boot/application drive, I was just told it shouldn't be, I think it was to do with LR's limited ability to multitask and the need to constantly read/write different actions to the same disc. My boot disc is my fastest too, but I still gained a lot of performance when I moved the cache to a different drive and made it a decent size.

I wonder if this tip is pre-dating SSD era. Old times you didn't want to be reading from multiple places on same HDD, or it really slowed down bad.

 

[privatebydesign]

I don't know why it shouldn't be on the boot/application drive, I was just told it shouldn't be, I think it was to do with LR's limited ability to multitask and the need to constantly read/write different actions to the same disc. My boot disc is my fastest too, but I still gained a lot of performance when I moved the cache to a different drive and made it a decent size.

I wonder if this tip is pre-dating SSD era. Old times you didn't want to be reading from multiple places on same HDD, or it really slowed down bad.

I doubt it, but maybe, I got my settings advice from several tutorials across the web and they were current so I expect they took widespread SSD's into account. Obviously I don't know specifically what part of my optimisation made the biggest improvement, but as it is set up now I am very happy with LR performance from a decent sized catalog with a modest computer. I tried the Smart Preview thing and didn't like it, I have tried various configurations of drives, image location etc, this seems to be a sweet spot.

 

[drmikeinpdx]

My Lightroom Database is about 3.3 Terrabytes divided between two mechanical hard drives. I have a small SSD that is designated as the cache drive.

I used to keep some of my photos on a hard drive that I never imported into the database, I would just occasionally import a particular photo I needed. Recently, I imported that drive and I like having it listed in Lightroom for quick access.

I feel my database works pretty well (fast) and have decided that I like having all my photos in it. I can't think of any good reason to have more than one database. You can have your photos distributed over multiple drives if you like and they can still be listed in the database.

 

[tpatana]

My Lightroom Database is about 3.3 Terrabytes divided between two mechanical hard drives. I have a small SSD that is designated as the cache drive.

I used to keep some of my photos on a hard drive that I never imported into the database, I would just occasionally import a particular photo I needed. Recently, I imported that drive and I like having it listed in Lightroom for quick access.

I feel my database works pretty well (fast) and have decided that I like having all my photos in it. I can't think of any good reason to have more than one database. You can have your photos distributed over multiple drives if you like and they can still be listed in the database.

I have currently 12 drives (+2 optical) on this PC, and only one of them doesn't hold photos. All duplicated though.

 

[tpatana]

To all of you who were holding your breath waiting for my results.

I changed the cache to 20GB. Also I updated Win7 -> Win10.

And now it works much faster switching between pictures. Most likely the cache setting did it, I don't think Win10 update helped on that.

Thanks for your tips!

 

[privatebydesign]

Good to hear!

Too often people just moan about performance without actually exploring the various ways to optimize for their particular needs, both in the hardware and the software we use.