DPP and Lightroom 5.5 JPG file export problem

[mpeeps]

Hello,
For some reason LR 5.5 exports unreadable jpg files by Digital Photo Professional 3. The message when trying to open (only recently it seems) exported jpg files is failure to decode. I had to go reinstall LR 5.4 to get back to that ability. Anyone else having same issues?

 

[dgatwood]

Hello,
For some reason LR 5.5 exports unreadable jpg files by Digital Photo Professional 3. The message when trying to open (only recently it seems) exported jpg files is failure to decode. I had to go reinstall LR 5.4 to get back to that ability. Anyone else having same issues?

I'm still on LR4, so I have no idea, but by any chance could you post a JPEG export of the same image from both versions (with the same export settings)? I'd be curious to compare them and see what the difference is. It might be something as silly as corrupted EXIF data or something.

 

[mpeeps]

Sorry...I already went back to LR 5.4. No problems now. The issue is only a problem when I want to open a single jpg file from My Pictures that I exported from LR5.5 as jpeg. If I exported file as Tif, no problem, but who wants thousands of tifs? I prefer opening individual files, both raw and jpg, with DPP editor module.

 

[mr_hyde]

The problem does not only exist in DPP. I have for instance a QNAP NAS which generates resized JPEGs for the photo station that is included. This process is not working with JPEGs generated with LR5.5. There is also other reported software that is not working with those JPEGs.

On the Adobe site there are already some bug reports. Just add another one or "+1" one of them.

 

[mpeeps]

Mr Hyde,
I followed you advice and reported problem to Adobe. Humorously, they though I was SHOUTING because I used caps on DPP and JPG!!

 

[Marsu42]

Humorously, they though I was SHOUTING because I used caps on DPP and JPG!!

Not without reason, Adobe seems to have gotten somewhat sloppy with their qa on production LR upgrades - back in the good ol' times there always were rc releases which they skipped this time. Producing jpegs that cannot be read by other standard apps is something that simply shouldn't happen. Adobe not releasing detailed changelogs for bugfixes and features also does not inspire confidence.

Having said that, imho downgrading is a bit over the top - if you export max iq jpeg then re-saving them for example with ImageMagick to correct the format won't hurt or maybe just running exiftool over them solves the problem. I've arrived at using IM for post-resizing, watermarking and so on anyway, much more flexible than LR.

 

[mpeeps]

Here is Adobe's response:

Do the JPEG files open in Photoshop?
If so, then the problem is most likely another bug in JPEG parsing in Canon's DPP code.

I was able to open the 5.5 exported file in PS. Not sure what that means. I've never had trouble with LR before and really do like the 5.X version.

 

[Marsu42]

Do the JPEG files open in Photoshop? If so, then the problem is most likely another bug in JPEG parsing in Canon's DPP code.

To get anyhwere with this issue you'll have to export the same shot in both LR5.5 and LR5.4 and then compare the xmp/iptc/exif/maker tags with exiftool - is there a difference? Of course Adobe will try to blame other apps, with "standards" such fuzzy as jpeg or tiff that's always possible until proven otherwise.

You can also simply strip all the tags from a "broken" LR5.5 image by running "exiftool -all= -Adobe:all= image.jpg" and then see if it opens in DPP or whatever.

 

[JEL]

To get anyhwere with this issue you'll have to export the same shot in both LR5.5 and LR5.4 and then compare the xmp/iptc/exif/maker tags with exiftool - is there a difference?

Here's an image exported in LR5.3 and LR5.5 for comparison purposes.

 

[Marsu42]

Here's an image exported in LR5.3 and LR5.5 for comparison purposes.

Hey, I didn't say *I* want to do this .. anyway, this is how the tags differ - There is a strange tag difference in the Adobe APP14Flags0 tag, maybe that's the cause.

Also attached is the LR55 jpeg with all tags stripped, check if this opens in non-Adobe apps - if not, the binary data is broken and it's an Adobe bug, if yes they can blame it on DPP.

--- LR53.txt	2014-07-09 23:22:40.108754500 +0200
+++ LR55.txt	2014-07-09 23:22:36.812749000 +0200
@@ -1,11 +1,11 @@
 [ExifTool] ExifToolVersion: 9.65
 [System] Directory: .
-[System] FileAccessDate: 2014:07:09 23:15:48+02:00
-[System] FileCreateDate: 2014:07:09 23:15:48+02:00
-[System] FileModifyDate: 2014:07:09 23:15:56+02:00
-[System] FileName: LR53.jpg
+[System] FileAccessDate: 2014:07:09 23:22:16+02:00
+[System] FileCreateDate: 2014:07:09 23:22:16+02:00
+[System] FileModifyDate: 2014:07:09 23:22:18+02:00
+[System] FileName: LR55.jpg
 [System] FilePermissions: 666
-[System] FileSize: 227148
+[System] FileSize: 263092
 [File] BitsPerSample: 8
 [File] ColorComponents: 3
 [File] CurrentIPTCDigest: 491e51fded5c122d01e757c357157923
@@ -20,11 +20,11 @@
 [IFD0] Copyright: JEL
 [IFD0] Make: Canon
 [IFD0] Model: Canon EOS 5D Mark III
-[IFD0] ModifyDate: 2014:01:05 09:12:46
+[IFD0] ModifyDate: 2014:07:07 21:21:31
 [IFD0] ResolutionUnit: 2
-[IFD0] Software: Adobe Photoshop Lightroom 5.3 (Windows)
-[IFD0] XResolution: 240
-[IFD0] YResolution: 240
+[IFD0] Software: Adobe Photoshop Lightroom 5.5 (Windows)
+[IFD0] XResolution: 300
+[IFD0] YResolution: 300
 [ExifIFD] ApertureValue: 1.20000007832525
 [ExifIFD] ColorSpace: 1
 [ExifIFD] CreateDate: 2014:01:01 02:17:34
@@ -58,16 +58,16 @@
 [ExifIFD] WhiteBalance: 0
 [IFD1] Compression: 6
 [IFD1] ResolutionUnit: 2
-[IFD1] ThumbnailLength: 9365
+[IFD1] ThumbnailLength: 9035
 [IFD1] ThumbnailOffset: 986
 [IFD1] XResolution: 72
 [IFD1] YResolution: 72
 [Photoshop] DisplayedUnitsX: 1
 [Photoshop] DisplayedUnitsY: 1
 [Photoshop] IPTCDigest: 491e51fded5c122d01e757c357157923
-[Photoshop] PhotoshopThumbnail: (Binary data 9365 bytes, use -b option to extract)
-[Photoshop] XResolution: 240
-[Photoshop] YResolution: 240
+[Photoshop] PhotoshopThumbnail: (Binary data 9035 bytes, use -b option to extract)
+[Photoshop] XResolution: 300
+[Photoshop] YResolution: 300
 [IPTC] ApplicationRecordVersion: 4
 [IPTC] By-line: [email protected].
 [IPTC] CodedCharacterSet: .%G
@@ -117,9 +117,9 @@
 [ICC-meas] MeasurementObserver: 1
 [XMP-x] XMPToolkit: Adobe XMP Core 5.5-c002 1.148022, 2012/07/15-18:06:45
 [XMP-xmp] CreateDate: 2014:01:01 02:17:34.00
-[XMP-xmp] CreatorTool: Adobe Photoshop Lightroom 5.3 (Windows)
-[XMP-xmp] MetadataDate: 2014:01:05 09:12:46+01:00
-[XMP-xmp] ModifyDate: 2014:01:05 09:12:46+01:00
+[XMP-xmp] CreatorTool: Adobe Photoshop Lightroom 5.5 (Windows)
+[XMP-xmp] MetadataDate: 2014:07:07 21:21:31+02:00
+[XMP-xmp] ModifyDate: 2014:07:07 21:21:31+02:00
 [XMP-dc] Creator: [email protected]: JEL248
 [XMP-dc] Format: image/jpeg
 [XMP-dc] Rights: JEL
@@ -136,14 +136,14 @@
 [XMP-photoshop] DateCreated: 2014:01:01 02:17:34.00
 [XMP-xmpMM] DerivedFromDocumentID: 185070B51AB83644A38502E862A348A9
 [XMP-xmpMM] DerivedFromOriginalDocumentID: 185070B51AB83644A38502E862A348A9
-[XMP-xmpMM] DocumentID: xmp.did:781592f3-572b-c643-be24-7d1671d63c02
+[XMP-xmpMM] DocumentID: xmp.did:18c1f38b-c9f2-9f4c-baba-1a9ba35c6d82
 [XMP-xmpMM] HistoryAction: derived, saved
 [XMP-xmpMM] HistoryChanged: /
-[XMP-xmpMM] HistoryInstanceID: xmp.iid:781592f3-572b-c643-be24-7d1671d63c02
+[XMP-xmpMM] HistoryInstanceID: xmp.iid:18c1f38b-c9f2-9f4c-baba-1a9ba35c6d82
 [XMP-xmpMM] HistoryParameters: converted from image/x-canon-cr2 to image/jpeg, saved to new location
-[XMP-xmpMM] HistorySoftwareAgent: Adobe Photoshop Lightroom 5.3 (Windows)
-[XMP-xmpMM] HistoryWhen: 2014:01:05 09:12:46+01:00
-[XMP-xmpMM] InstanceID: xmp.iid:781592f3-572b-c643-be24-7d1671d63c02
+[XMP-xmpMM] HistorySoftwareAgent: Adobe Photoshop Lightroom 5.5 (Windows)
+[XMP-xmpMM] HistoryWhen: 2014:07:07 21:21:31+02:00
+[XMP-xmpMM] InstanceID: xmp.iid:18c1f38b-c9f2-9f4c-baba-1a9ba35c6d82
 [XMP-xmpMM] OriginalDocumentID: 185070B51AB83644A38502E862A348A9
 [XMP-crs] AlreadyApplied: True
 [XMP-crs] AutoLateralCA: 0
@@ -182,7 +182,7 @@
 [XMP-crs] HueAdjustmentYellow: 0
 [XMP-crs] LensManualDistortionAmount: 0
 [XMP-crs] LensProfileChromaticAberrationScale: 100
-[XMP-crs] LensProfileDigest: 639283345D6001793A602558C753423F
+[XMP-crs] LensProfileDigest: 1B9204B25A98B63F2DFD385A244207BE
 [XMP-crs] LensProfileDistortionScale: 100
 [XMP-crs] LensProfileEnable: 0
 [XMP-crs] LensProfileFilename: Canon EOS 5D Mark II (Canon EF 85mm f1.2L II USM) - RAW.lcp
@@ -241,17 +241,22 @@
 [XMP-crs] SplitToningShadowHue: 215
 [XMP-crs] SplitToningShadowSaturation: 50
 [XMP-crs] Tint: +12
+[XMP-crs] ToneCurve: 0, 0, 255, 255
+[XMP-crs] ToneCurveBlue: 0, 0, 255, 255
+[XMP-crs] ToneCurveGreen: 0, 0, 255, 255
+[XMP-crs] ToneCurveName: Linear
 [XMP-crs] ToneCurveName2012: Linear
 [XMP-crs] ToneCurvePV2012: 0, 0, 255, 255
 [XMP-crs] ToneCurvePV2012Blue: 0, 0, 255, 255
 [XMP-crs] ToneCurvePV2012Green: 0, 0, 255, 255
 [XMP-crs] ToneCurvePV2012Red: 0, 0, 255, 255
-[XMP-crs] Version: 8.3
+[XMP-crs] ToneCurveRed: 0, 0, 255, 255
+[XMP-crs] Version: 8.5
 [XMP-crs] Vibrance: 0
 [XMP-crs] VignetteAmount: 0
 [XMP-crs] WhiteBalance: As Shot
 [XMP-crs] Whites2012: 0
-[Adobe] APP14Flags0: 16384
+[Adobe] APP14Flags0: 49152
 [Adobe] APP14Flags1: 0
 [Adobe] ColorTransform: 1
 [Adobe] DCTEncodeVersion: 100

 

[JEL]

Hey, I didn't say *I* want to do this

Haha sorry ;D
I just used your quote to direct attention to why I posted the 2 images, but they're really for who-ever wants to do the detective-work

They're also talking about it over on the adobe-forum, but it's a bit too advanced tech-stuff for my skill-level
https://forums.adobe.com/message/6537585#6537585

 

[mr_hyde]

ok,

i have exported the same image with LR5.4 and 5.5 and did a comparison with "Compare". there seems to be also an x/y-offset if you are using the same settings in 5.4 and 5.5. I have uploaded both images and the diff image to adobe and asked them "friendly" not to blame something immediately to other apps but change the customer service attitude and ask for samples instead. Not sure where this ends, but it is worth a try. I need to see if I also have the exiftool installed on my NAS, then I will also do the exif comparison of these images and upload it to adobe......

EDIT:
just tried the LR5.5 and stripped LR5.5 on the NAS photostation.... Too bad. both images do show the same result.

and the same is true for DPP3 and DPP4.

 

[mr_hyde]

look at this:

https://forums.adobe.com/message/6538313#6538313

The plugin solves the issue. It works at least for my application.

 

[dgatwood]

So not an EXIF problem; it's a problem with the JPEG data itself. The most obvious difference between these two files is that the order of the JPEG markers is very different. If a JPEG parser made naive assumptions about the order of some of the segments, it could barf.

LR 5.3:

SOI (Start of Image)
Adobe DCT file header
Define quantization table
SOF (Start of frame)
Define restart interval
Define Huffman table
Start of scan


LR 5.5:

SOI
Define quantization table
Define restart interval
Adobe DCT file header
SOF
Define Huffman table
SOS


Many of those changes are Adobe-specific, but their decision to move the restart interval before the start-of-frame marker could very easily break parsers that assume the SOF is part of the frame, not part of the file as a whole. As an experiment, I reordered the sections of the file with dd, to move the restart interval marker and value below the start of frame marker to match the LR 5.3 ordering. See if the attached file works.

 

[philmoz]

So not an EXIF problem; it's a problem with the JPEG data itself. The most obvious difference between these two files is that the order of the JPEG markers is very different. If a JPEG parser made naive assumptions about the order of some of the segments, it could barf.

LR 5.3:

SOI (Start of Image)
Adobe DCT file header
Define quantization table
SOF (Start of frame)
Define restart interval
Define Huffman table
Start of scan


LR 5.5:

SOI
Define quantization table
Define restart interval
Adobe DCT file header
SOF
Define Huffman table
SOS


Many of those changes are Adobe-specific, but their decision to move the restart interval before the start-of-frame marker could very easily break parsers that assume the SOF is part of the frame, not part of the file as a whole. As an experiment, I reordered the sections of the file with dd, to move the restart interval marker and value below the start of frame marker to match the LR 5.3 ordering. See if the attached file works.

This file does not load in DPP.

Phil.

 

[mr_hyde]

Pretty interesting, just saw the release notes for the new adobe camera raw 8.6 Release Candidate:



"Fix issue with JPEG images saved by Camera Raw not readable in some external applications."



I guess there will be a fix for LR5.5.

http://blogs.adobe.com/lightroomjournal/2014/07/camera-raw-8-4-rc-and-dng-converter-8-4-rc-now-available-on-adobe-labs.html?scid=social_20140717_28031436