Corrupted Images in Lightroom 5.3

[Rienzphotoz]

When I import images to Lightroom 5.3 (using "Add photos to catalog without moving them") some of my images are showing up with a weird color banding kinda thing at the bottom of the image (it only happens at the bottom of the image ... below is one for your reference ... you can also see a few more "damaged" images in the thumbnails at the bottom of the screen shot). However, when I download/check in Canon DPP, everything seems fine i.e. there is no image corruption/damage.
This is happening only with the images made with my 5D MKII and 70D ... images from my other cameras, (EOS-M, G1X, Nikon D610 & Sony a7) do not exhibit this problem ... and it happens regardless of the memory cards I use (CF & SHDC).
I import all my images to a Seagate Backup Plus USB3 Portable drive and access them from there ... but the same problem occurs even if I import directly to my MBP.
What do you think could be the problem?

 

[philmoz]

Can you upload one of the affected image files somewhere - I can try in my copy of LR and see if the same thing happens.
I can check on Windows and OS/X.

Might help narrow down if the problem is in the image file, or the LR software.

Phil.

 

[Rienzphotoz]

Can you upload one of the affected image files somewhere - I can try in my copy of LR and see if the same thing happens.
I can check on Windows and OS/X.

Might help narrow down if the problem is in the image file, or the LR software.

Phil.

Thanks for the reply ... below is a drop box link to the original file
https://www.dropbox.com/s/jpdp51b9hnfhb2f/IMG_1910.JPG

 

[philmoz]

Can you upload one of the affected image files somewhere - I can try in my copy of LR and see if the same thing happens.
I can check on Windows and OS/X.

Might help narrow down if the problem is in the image file, or the LR software.

Phil.

Thanks for the reply ... below is a drop box link to the original file
https://www.dropbox.com/s/jpdp51b9hnfhb2f/IMG_1910.JPG

That looks like a corrupted file. It shows with the bands at the bottom in the dropbox preview, and in image viewers on both Windows and OS/X.
LR on Windows and OS/X says the image is damaged.
DPP on OS/X shows the thumbnail ok (it's a small JPEG embedded in the file); but hangs when I try and open the image in the editor.

Do you still have the image on a CF or SD card?

Edit: I've seen something like this once with a faulty card reader. It would corrupt files when copying from the card (if I copied from the camera via USB the files were fine).

Phil.

 

[Rienzphotoz]

Can you upload one of the affected image files somewhere - I can try in my copy of LR and see if the same thing happens.
I can check on Windows and OS/X.

Might help narrow down if the problem is in the image file, or the LR software.

Phil.

Thanks for the reply ... below is a drop box link to the original file
https://www.dropbox.com/s/jpdp51b9hnfhb2f/IMG_1910.JPG

That looks like a corrupted file. It shows with the bands at the bottom in the dropbox preview, and in image viewers on both Windows and OS/X.
LR on Windows and OS/X says the image is damaged.
DPP on OS/X shows the thumbnail ok (it's a small JPEG embedded in the file); but hangs when I try and open the image in the editor.

Do you still have the image on a CF or SD card?

Edit: I've seen something like this once with a faulty card reader. It would corrupt files when copying from the card (if I copied from the camera via USB the files were fine).

Phil.

Yes, I still have the image in in the SD card and below is the link I've uploaded to dropbox (this one is straight from the SD card to dropbox using the MBP's internal card reader) and it does not exhibit the image corruption.
https://www.dropbox.com/s/qou36027ernjcuk/IMG_1910%202.JPG

This is random occurrence i.e. sometimes it happens when I use the external card reader, sometimes with the internal card reader and sometimes even when it is straight from the camera ... this is just driving me crazy.

 

[philmoz]

Can you upload one of the affected image files somewhere - I can try in my copy of LR and see if the same thing happens.
I can check on Windows and OS/X.

Might help narrow down if the problem is in the image file, or the LR software.

Phil.

Thanks for the reply ... below is a drop box link to the original file
https://www.dropbox.com/s/jpdp51b9hnfhb2f/IMG_1910.JPG

That looks like a corrupted file. It shows with the bands at the bottom in the dropbox preview, and in image viewers on both Windows and OS/X.
LR on Windows and OS/X says the image is damaged.
DPP on OS/X shows the thumbnail ok (it's a small JPEG embedded in the file); but hangs when I try and open the image in the editor.

Do you still have the image on a CF or SD card?

Edit: I've seen something like this once with a faulty card reader. It would corrupt files when copying from the card (if I copied from the camera via USB the files were fine).

Phil.

Yes, I still have the image in in the SD card and below is the link I've uploaded to dropbox (this one is straight from the SD card to dropbox using the MBP's internal card reader) and it does not exhibit the image corruption.
https://www.dropbox.com/s/qou36027ernjcuk/IMG_1910%202.JPG

This is random occurrence i.e. sometimes it happens when I use the external card reader, sometimes with the internal card reader and sometimes even when it is straight from the camera ... this is just driving me crazy.

Ok, that means it's not the camera(s) that are the problem.

What are the common factors?

You say you can get corruption with different card readers and also via direct USB copy from the camera, so that would eliminate the card reader as the problem.

Are you sure it's happening with different cards? Do you have another corrupt image where there is still a good copy on a CF or other SD card?

If it's not the memory cards then that only leaves the external hard drive or the MBP itself. You mentioned earlier that you've seen the problem copying directly to the MBP which would imply the problem lies in the Mac.

At this point I would start doing some controlled testing and take notes as you go.
Eliminate as many variables as possible and gradually change one thing at a time to see if you can isolate the problem.

I would also suggest the following in case the problem is with the MBP:
- run a memory and hardware check on the MBP with the Apple diagnostics (http://support.apple.com/kb/PH14291)
- do a disk check and permissions verification on the MBP hard disk, and a disk check on the external drive.

Phil.

 

[Rienzphotoz]

Can you upload one of the affected image files somewhere - I can try in my copy of LR and see if the same thing happens.
I can check on Windows and OS/X.

Might help narrow down if the problem is in the image file, or the LR software.

Phil.

Thanks for the reply ... below is a drop box link to the original file
https://www.dropbox.com/s/jpdp51b9hnfhb2f/IMG_1910.JPG

That looks like a corrupted file. It shows with the bands at the bottom in the dropbox preview, and in image viewers on both Windows and OS/X.
LR on Windows and OS/X says the image is damaged.
DPP on OS/X shows the thumbnail ok (it's a small JPEG embedded in the file); but hangs when I try and open the image in the editor.

Do you still have the image on a CF or SD card?

Edit: I've seen something like this once with a faulty card reader. It would corrupt files when copying from the card (if I copied from the camera via USB the files were fine).

Phil.

Yes, I still have the image in in the SD card and below is the link I've uploaded to dropbox (this one is straight from the SD card to dropbox using the MBP's internal card reader) and it does not exhibit the image corruption.
https://www.dropbox.com/s/qou36027ernjcuk/IMG_1910%202.JPG

This is random occurrence i.e. sometimes it happens when I use the external card reader, sometimes with the internal card reader and sometimes even when it is straight from the camera ... this is just driving me crazy.

Ok, that means it's not the camera(s) that are the problem.

What are the common factors?

You say you can get corruption with different card readers and also via direct USB copy from the camera, so that would eliminate the card reader as the problem.

Are you sure it's happening with different cards? Do you have another corrupt image where there is still a good copy on a CF or other SD card?

If it's not the memory cards then that only leaves the external hard drive or the MBP itself. You mentioned earlier that you've seen the problem copying directly to the MBP which would imply the problem lies in the Mac.

At this point I would start doing some controlled testing and take notes as you go.
Eliminate as many variables as possible and gradually change one thing at a time to see if you can isolate the problem.

I would also suggest the following in case the problem is with the MBP:
- run a memory and hardware check on the MBP with the Apple diagnostics (http://support.apple.com/kb/PH14291)
- do a disk check and permissions verification on the MBP hard disk, and a disk check on the external drive.

Phil.

The frustrating part about this is that the images in the CF/SD card are all fine (no damage/corruption), it only happens after I download/import it to Lightroom (weather it be my MBP or iMac or even the Windows 7 laptop) ... when I download them to DPP, there is absolutely no problem ... so far, I've been able to isolate the problem only to lightroom ... my workaround for this problem is trying at least 2 different imports/downloads i.e. try directly with the CF/SD card in the external card reader or internal card reader and finally directly from the camera to the computer ... usually one of them works with corruption free images. But this is just frustrating and time consuming

 

[philmoz]

The frustrating part about this is that the images in the CF/SD card are all fine (no damage/corruption), it only happens after I download/import it to Lightroom (weather it be my MBP or iMac or even the Windows 7 laptop) ... when I download them to DPP, there is absolutely no problem ... so far, I've been able to isolate the problem only to lightroom ... my workaround for this problem is trying at least 2 different imports/downloads i.e. try directly with the CF/SD card in the external card reader or internal card reader and finally directly from the camera to the computer ... usually one of them works with corruption free images. But this is just frustrating and time consuming

Can you outline the exact steps you take to transfer the files?

From your original post it sounded like you are copying the files to the hard drive outside of LR and them importing them into LR after transfer to the computer.

This latest post sounds like you are using the LR import dialog to transfer the files to the computer.

I can understand the frustration - these kind of intermittent / random problems can be difficult to solve.
The only reliable way to track down the problem is with methodical testing

If you are seeing the problem on multiple computers (and operating systems), multiple SD/CF cards, and multiple card readers then I'm really stumped, as that would seem to have eliminated all possible common elements. Unless you've found some very obscure Lightroom bug.

Phil.

 

[Rienzphotoz]

The frustrating part about this is that the images in the CF/SD card are all fine (no damage/corruption), it only happens after I download/import it to Lightroom (weather it be my MBP or iMac or even the Windows 7 laptop) ... when I download them to DPP, there is absolutely no problem ... so far, I've been able to isolate the problem only to lightroom ... my workaround for this problem is trying at least 2 different imports/downloads i.e. try directly with the CF/SD card in the external card reader or internal card reader and finally directly from the camera to the computer ... usually one of them works with corruption free images. But this is just frustrating and time consuming

Can you outline the exact steps you take to transfer the files?

From your original post it sounded like you are copying the files to the hard drive outside of LR and them importing them into LR after transfer to the computer.

This latest post sounds like you are using the LR import dialog to transfer the files to the computer.

I can understand the frustration - these kind of intermittent / random problems can be difficult to solve.
The only reliable way to track down the problem is with methodical testing

If you are seeing the problem on multiple computers (and operating systems), multiple SD/CF cards, and multiple card readers then I'm really stumped, as that would seem to have eliminated all possible common elements. Unless you've found some very obscure Lightroom bug.

Phil.

Over the past few years, this has been my regular workflow for transferring files:
1. Copy paste images from CF/SD cards (using an external Lexar Card reader) to an external hard drive
2. Start Lightroom to import using "Add photos to catalog without moving them"

I never had an issue with that workflow ... but a few months into using LR5 (in 2013), I noticed some of the images, from the CF/SD card in my 5D MK III with the same issue (no matter which computer I use i.e. MBP mid 2013 model, iMac early 2013 model, Windows 7 laptop (2012 model).

Since this was happening randomly, I thought it was the fault of SanDisk 64 GB CF or the SanDisk 64GB SDHC card, so I tried the SDHC card in the Sony a7, but did not have any problem with it ... so I thought it might be my camera's fault.

But it is also happening with my 5 days old 70D with a brand new 32GB SDHC card ... so I am assuming that it isn't the card or the camera(s) ... so I changed the external hard disk but still the same issue ... then I tried importing the images to to my computer's internal hard drive, yet the problem still continues ... what baffles me is that it only happens for just a few images and the majority of the images are fine. Luckily most of the images that get corrupted/damaged are the bad images (which would have ended up in the trash anyway) ... but occasionally a few very good images suffer from this problem ... that's when I get totally frustrated

 

[JustMeOregon]

Just a shot in the dark, but maybe try importing the files that are giving you problems into a new fresh Lightroom catalog. I've had a corrupt Lightroom catalog in the past and it resulted in a lot of strange intermittent errors.

 

[philmoz]

The frustrating part about this is that the images in the CF/SD card are all fine (no damage/corruption), it only happens after I download/import it to Lightroom (weather it be my MBP or iMac or even the Windows 7 laptop) ... when I download them to DPP, there is absolutely no problem ... so far, I've been able to isolate the problem only to lightroom ... my workaround for this problem is trying at least 2 different imports/downloads i.e. try directly with the CF/SD card in the external card reader or internal card reader and finally directly from the camera to the computer ... usually one of them works with corruption free images. But this is just frustrating and time consuming

Can you outline the exact steps you take to transfer the files?

From your original post it sounded like you are copying the files to the hard drive outside of LR and them importing them into LR after transfer to the computer.

This latest post sounds like you are using the LR import dialog to transfer the files to the computer.

I can understand the frustration - these kind of intermittent / random problems can be difficult to solve.
The only reliable way to track down the problem is with methodical testing

If you are seeing the problem on multiple computers (and operating systems), multiple SD/CF cards, and multiple card readers then I'm really stumped, as that would seem to have eliminated all possible common elements. Unless you've found some very obscure Lightroom bug.

Phil.

Over the past few years, this has been my regular workflow for transferring files:
1. Copy paste images from CF/SD cards (using an external Lexar Card reader) to an external hard drive
2. Start Lightroom to import using "Add photos to catalog without moving them"

I never had an issue with that workflow ... but a few months into using LR5 (in 2013), I noticed some of the images, from the CF/SD card in my 5D MK III with the same issue (no matter which computer I use i.e. MBP mid 2013 model, iMac early 2013 model, Windows 7 laptop (2012 model).

Since this was happening randomly, I thought it was the fault of SanDisk 64 GB CF or the SanDisk 64GB SDHC card, so I tried the SDHC card in the Sony a7, but did not have any problem with it ... so I thought it might be my camera's fault.

But it is also happening with my 5 days old 70D with a brand new 32GB SDHC card ... so I am assuming that it isn't the card or the camera(s) ... so I changed the external hard disk but still the same issue ... then I tried importing the images to to my computer's internal hard drive, yet the problem still continues ... what baffles me is that it only happens for just a few images and the majority of the images are fine. Luckily most of the images that get corrupted/damaged are the bad images (which would have ended up in the trash anyway) ... but occasionally a few very good images suffer from this problem ... that's when I get totally frustrated

Very strange.

You said that you also tried using the internal SD card reader in your MBP, and still seen the problem - is this correct?
Just want to confirm as it was a Lexar card reader I had trouble with (USB 3.0 dual CF/SD slot reader).

If this is the case then it would seem that Lightroom is the only common factor left.

Since you are importing the files in their existing location, LR would not normally write to the files unless you have something that is causing an update on import.
Are you applying any settings on import that might cause LR to update the files in some way? Even if you are it's hard to see how Lightroom could be causing this as it would normally only update the EXIF data.

Trying a new catalog as JustMeOregon suggested is worth a try.

You could also try resetting all the LR preferences and see if it helps (although I'd keep a backup if you've made a lot of setting changes).
http://www.lightroomforums.net/showthread.php?14226-Resetting-(or-Trashing-)-the-Lightroom-Preferences-file

I compared the two image files you uploaded and there is a small chunk of data missing (108 bytes) towards the end of the file, and the 404 bytes immediately before the missing chunk are all 0. Everything before and after this is identical.

Phil.

 

[privatebydesign]

Narrow down the possibilities.

Are the files on the HD good?

• Yes: Save your LR library somewhere , reload LR and then relink your old library file.
• No: You have an OS or hardware issue, as this happens across multiple drives it probably isn't hardware.

LR rarely touches the file, and never if you just "ADD" on import with or without a preset. The only time LR touches a file is when you move, copy, or export it, even then it doesn't write changes to the original, it just moves them, or copies them, or makes a new version.

If the files are good on the HD, I'd test open some problem images in DPP, then it is LR, just reload it.

 

[philmoz]

Narrow down the possibilities.

Are the files on the HD good?

• Yes: Save your LR library somewhere , reload LR and then relink your old library file.
• No: You have an OS or hardware issue, as this happens across multiple drives it probably isn't hardware.

LR rarely touches the file, and never if you just "ADD" on import with or without a preset. The only time LR touches a file is when you move, copy, or export it, even then it doesn't write changes to the original, it just moves them, or copies them, or makes a new version.

If the files are good on the HD, I'd test open some problem images in DPP, then it is LR, just reload it.

To summarise:

The images are JPEG and it appears that they are not corrupted after the initial copy to the hard disk (not using Lightroom).
After subsequent importing into Lightroom the image files become corrupted.

From what reinzphotoz has posted this happens on different computers, with different operating systems, different CF & SD cards, different external drives and different card readers (waiting on confirmation of this).

Lightroom will write back to newly imported JPEG files if you tell it to auto save changes to XMP files, and do something on import like add keywords. This should only update the EXIF data; but may change the file size causing a re-write of the entire file.

Phil.

 

[privatebydesign]

Missed that they were jpegs, my mistake I haven't used them since, forever!

I'd start by removing LR from my computer and reloading it.

 

[Rienzphotoz]

Missed that they were jpegs, my mistake I haven't used them since, forever!

I'd start by removing LR from my computer and reloading it.

The file I uploaded was JPEG, but I generally shoot RAW and this problem occurs also in RAW files as well.
I was thinking about reloading LR ... but am worried that it might be the catalog that is causing the problem ... but I'll reload it one of these days (anyway the catalog is safely backed up in several places. But if it is the bloody catalog, then I'd need to sort (around 40000+ images, which is what I shot since LR5 release) and re-edit ... now that is making me tired already.

 

[Rienzphotoz]

Just a shot in the dark, but maybe try importing the files that are giving you problems into a new fresh Lightroom catalog. I've had a corrupt Lightroom catalog in the past and it resulted in a lot of strange intermittent errors.

Thanks for the tip and I have done that on several occasions ... when I re-import the images into a separate catalog, the problem disappears ... but it is really annoying, because I need to re-import everything separately and work with separate catalogs and name them separately so I can remember which catalog belongs to which image files etc ... which is just time consuming and makes the process from being enjoyable experience to cumbersome experience.

 

[philmoz]

Missed that they were jpegs, my mistake I haven't used them since, forever!

I'd start by removing LR from my computer and reloading it.

The file I uploaded was JPEG, but I generally shoot RAW and this problem occurs also in RAW files as well.
I was thinking about reloading LR ... but am worried that it might be the catalog that is causing the problem ... but I'll reload it one of these days (anyway the catalog is safely backed up in several places. But if it is the bloody catalog, then I'd need to sort (around 40000+ images, which is what I shot since LR5 release) and re-edit ... now that is making tired already.

At this stage you might be better to contact Adobe.

Phil.

 

[Rienzphotoz]

Missed that they were jpegs, my mistake I haven't used them since, forever!

I'd start by removing LR from my computer and reloading it.

The file I uploaded was JPEG, but I generally shoot RAW and this problem occurs also in RAW files as well.
I was thinking about reloading LR ... but am worried that it might be the catalog that is causing the problem ... but I'll reload it one of these days (anyway the catalog is safely backed up in several places. But if it is the bloody catalog, then I'd need to sort (around 40000+ images, which is what I shot since LR5 release) and re-edit ... now that is making tired already.

At this stage you might be better to contact Adobe.

Phil.

I was planning on doing it, but I thought I'd give it a shot at CR to see if anyone else is facing the same problem and how they fixed it ... and if they did not get it fixed, then I was planning on contacting Adobe saying that "I've got few more people who are experiencing the same problem" ... but looks like, I'm alone on this one.
Will keep you posted on what Adobe has to say ... thanks for your replies.

 

[privatebydesign]

Missed that they were jpegs, my mistake I haven't used them since, forever!

I'd start by removing LR from my computer and reloading it.

The file I uploaded was JPEG, but I generally shoot RAW and this problem occurs also in RAW files as well.
I was thinking about reloading LR ... but am worried that it might be the catalog that is causing the problem ... but I'll reload it one of these days (anyway the catalog is safely backed up in several places. But if it is the bloody catalog, then I'd need to sort (around 40000+ images, which is what I shot since LR5 release) and re-edit ... now that is making me tired already.

If it is happening to RAW files it is 100% your catalog.

But fixing it isn't as onerous as all that. I'd start a new one, then split the old one and "Import from another Catalog....." do this in stages and you will narrow down where your issue is, you will save all your edits, ratings and keywords etc too.

 

[Rienzphotoz]

Missed that they were jpegs, my mistake I haven't used them since, forever!

I'd start by removing LR from my computer and reloading it.

The file I uploaded was JPEG, but I generally shoot RAW and this problem occurs also in RAW files as well.
I was thinking about reloading LR ... but am worried that it might be the catalog that is causing the problem ... but I'll reload it one of these days (anyway the catalog is safely backed up in several places. But if it is the bloody catalog, then I'd need to sort (around 40000+ images, which is what I shot since LR5 release) and re-edit ... now that is making me tired already.

If it is happening to RAW files it is 100% your catalog.

But fixing it isn't as onerous as all that. I'd start a new one, then split the old one and "Import from another Catalog....." do this in stages and you will narrow down where your issue is, you will save all your edits, ratings and keywords etc too.

Could you please elaborate on "split the old one and import form another catalog"?
Thanks in advance