Opinion: Love it or Hate it, Digital Correction is here to Stay

[DocInfoSci]

I wish there was a way for manufactures to share their lens correction data properly.

Possible an open source file format?

This way Lightroom etc. can benefit from the exact correction data to use in their lens profiles. Also, this would work in other ways. It would allow third party lens manufacturers to give their lens correction data to Canon, Nikon, etc. for use with in camera correction.

I suspect that gives away too much of the secret sauce. An interesting point, big pharma companies often will give away their source code as part of articles and general demonstrations of research thrust, but they seldom give away in-house data — because the in-house data would communicate a lot of trade secrets, hard-won insights, tiny little things that will add up to big competitive differences in drugs, etc. Ditto here for Canon's DLO corrections, which would probably communicate very interesting things about material behaviour, light behaviour between and through lenses, etc. Canon is simply making everyone else do the reverse engineering and math that Canon achieved during the design, manufacturing, and test processes themselves.

 

[DocInfoSci]

Or point out the problem in the examples that I posted above, because I don’t see it.

I have to agree that your samples are very uniform. I see nothing wrong with that noise, and if preserved in that way after "reasonable" edits then I'd be happy to hand if off as final work.

But, you mentioned the use of DXO and it has a reputation for achieving nice outcomes. What about pre-DXO? What about RAW?

In general Canon has been continuously working to improve their noise, which is why in part the higher ISO values become more usable. I'm not sure where in the pipeline that noise is adjusted or refined (if refined, depending on the model), but I wouldn't be surprised if Canon uses their a) knowledge of the sensor, b) knowledge of the lens behaviour (DLO data), and C) a pipeline to make a final trained-AI adjustment to the noise in the image. That could be done as part of generating the RAW image (a phase 1 adjustment to the sensor read) and in generating a DLO-derived TIFF, HEIC, or JPEG (a phase 2 output image adjustment as part of the DLO system). If any part of that is, in fact, true then that would go a long way to explaining the stellar VCM performance in the example image graciously provided by @neuroanatomist.

To @zardoz's point, I have witnessed myself bad artifact generation / deterioration when boosting aspects of an image and making other nudges (like correcting a building perspective). So I know it happens in scenarios, and when I first saw @zardoz's post it made sense to me that the noise might be presented in a manner that would lend itself as described. But, I don't own an image circle compromised lens (in the physical sense) so without driving to my local store and grabbing some test photos I can't see it for myself (and I don't generally trust Internet posts on these topics these days).

But if it would otherwise happen in the VCM world then Canon or DXO is doing a bang-up job of creating a final image that hides the issue to some reasonable degree. If that same process would generate a TIFF or DNG for use in other apps, either via Canon's software or a direct import by, say, Photoshop then bravo to Canon for going the extra step!

 

[neuroanatomist]

I have to agree that your samples are very uniform. I see nothing wrong with that noise, and if preserved in that way after "reasonable" edits then I'd be happy to hand if off as final work.

But, you mentioned the use of DXO and it has a reputation for achieving nice outcomes. What about pre-DXO? What about RAW?

Here's the same image, still in DxO but with all corrections turned off (so as close to RAW as you can get, more so than Canon because DPP will force the distortion correction and DxO does not), pushed 3 stops. No NR. Below it are the top left and bottom left corners at 100%.



Very noisy, but the noise still looks even without any 'lattice, moire, etc.' artifacts.

In general Canon has been continuously working to improve their noise, which is why in part the higher ISO values become more usable. I'm not sure where in the pipeline that noise is adjusted or refined (if refined, depending on the model), but I wouldn't be surprised if Canon uses their a) knowledge of the sensor, b) knowledge of the lens behaviour (DLO data), and C) a pipeline to make a final trained-AI adjustment to the noise in the image. That could be done as part of generating the RAW image (a phase 1 adjustment to the sensor read) and in generating a DLO-derived TIFF, HEIC, or JPEG (a phase 2 output image adjustment as part of the DLO system). If any part of that is, in fact, true then that would go a long way to explaining the stellar VCM performance in the example image graciously provided by @neuroanatomist.

I don't use Canon's software to convert RAW images, except on the few occasions when I have a camera that's too new for DxO to support (and in that case, I usually shoot RAW+JPG so I can have usable images immediately, then I go back and properly convert them later. I loathe DPP's workflow.

DxO is using AI for the noise reduction, but I have no idea what's going on under the hood (if anything) in terms of the interaction of NR algorithms with lens correction profiles.

To @zardoz's point, I have witnessed myself bad artifact generation / deterioration when boosting aspects of an image and making other nudges (like correcting a building perspective). So I know it happens in scenarios, and when I first saw @zardoz's post it made sense to me that the noise might be presented in a manner that would lend itself as described. But, I don't own an image circle compromised lens (in the physical sense) so without driving to my local store and grabbing some test photos I can't see it for myself (and I don't generally trust Internet posts on these topics these days).

I've certainly seen fixed pattern noise artifacts when pushing images from older Canon sensors (5DII, for example). Once Canon updated their lithography (at the time of the 5DIV, IIRC) the FPN issues pretty much went away (as did most of the internet complaints about Canon's poor low ISO DR). I can't say I've seen artifacts from perspective correction of architecture shots, but I usually do those with a TS-E lens and not software.

But if it would otherwise happen in the VCM world then Canon or DXO is doing a bang-up job of creating a final image that hides the issue to some reasonable degree. If that same process would generate a TIFF or DNG for use in other apps, either via Canon's software or a direct import by, say, Photoshop then bravo to Canon for going the extra step!

I use DxO PhotoLab, but they also have PureRAW that offers similar corrections more easily integrated into other workflows (e.g. with plugins for LR and PS).

 

[AlanF]

DxO is using AI for the noise reduction, but I have no idea what's going on under the hood (if anything) in terms of the interaction of NR algorithms with lens correction profiles.

DxO used to do its superb noise reduction by old-fashioned number crunching, and added ML only recently. I wonder how much AI they actually use?

By the way, I don't like those vertical purple fringes.

 

[P-visie]

I'm not sure where in the pipeline that noise is adjusted or refined (if refined, depending on the model), but I wouldn't be surprised if Canon uses their a) knowledge of the sensor, b) knowledge of the lens behaviour (DLO data), and C) a pipeline to make a final trained-AI adjustment to the noise in the image. That could be done as part of generating the RAW image (a phase 1 adjustment to the sensor read) and in generating a DLO-derived TIFF, HEIC, or JPEG (a phase 2 output image adjustment as part of the DLO system).

This is what Canon’s neural network image processing tool does.
See: https://app.ssw.imaging-saas.canon/app/en/nnipt.html

Halfway down the page is a link to a whitepaper which describes the concepts.
See: https://app.ssw.imaging-saas.canon/...aper_Deep_Learning_Upscaling_Technology_E.pdf

 

[DocInfoSci]

Here's the same image, still in DxO but with all corrections turned off (so as close to RAW as you can get, more so than Canon because DPP will force the distortion correction and DxO does not), pushed 3 stops. No NR. Below it are the top left and bottom left corners at 100%.

Thank you!

Very noisy, but the noise still looks even without any 'lattice, moire, etc.' artifacts.

Yeah, wherever it might be an issue that context is not this context. Unless maybe someone just takes Canon's RAW and does their own thing.

I don't use Canon's software to convert RAW images, except on the few occasions when I have a camera that's too new for DxO to support (and in that case, I usually shoot RAW+JPG so I can have usable images immediately, then I go back and properly convert them later. I loathe DPP's workflow.

I do use DPP. And what you said...

DxO is using AI for the noise reduction, but I have no idea what's going on under the hood (if anything) in terms of the interaction of NR algorithms with lens correction profiles.

I've certainly seen fixed pattern noise artifacts when pushing images from older Canon sensors (5DII, for example). Once Canon updated their lithography (at the time of the 5DIV, IIRC) the FPN issues pretty much went away (as did most of the internet complaints about Canon's poor low ISO DR). I can't say I've seen artifacts from perspective correction of architecture shots, but I usually do those with a TS-E lens and not software.

The TS is the one lens series that I'd love to add to my collection. Yet to justify it past want, but I'm looking for excuses.

I use DxO PhotoLab, but they also have PureRAW that offers similar corrections more easily integrated into other workflows (e.g. with plugins for LR and PS).

It wouldn't surprise me if DXO takes the time to do the math and refinement.

I use DPP to start with Canon's "math" applied and then I start my edits. No point in recreating their work, and since I bought into their ecosystem to start with... well, would be silly to avoid it on some principle. I think using something like DXO or other high quality purveyor of transformations is the same thing, just a more refined experience.

But in the context of this thread, for the examples given it seems that Canon's process has mitigated any issue that might have arisen from the stretching of the image circle to a rectangle. That should probably be quite comforting.

Unless it AI-slopped into existence the person obsessing with their smart phone in the corner. In which case, concerning... and truly funny.

 

[neuroanatomist]

DxO used to do its superb noise reduction by old-fashioned number crunching, and added ML only recently. I wonder how much AI they actually use?

Introducing DxO DeepPRIME 3: The New Standard in RAW Image Processing - DxO

Discover a new generation of its AI-based denoising DeepPRIME technology for both Bayer and X-Trans sensors

www.dxo.com

"Trained on a dataset ten times larger than its predecessor and leveraging over 100,000 DxO Modules, DeepPRIME 3 sets a new benchmark for RAW processing."

Well, nice but...ten times larger than what?

By the way, I don't like those vertical purple fringes.

Yeah, makes me really miss my first Canon prime, the EF 85/1.8.

 

[Klendothir]

This is what Canon’s neural network image processing tool does.
See: https://app.ssw.imaging-saas.canon/app/en/nnipt.html

Halfway down the page is a link to a whitepaper which describes the concepts.
See: https://app.ssw.imaging-saas.canon/...aper_Deep_Learning_Upscaling_Technology_E.pdf

LR user here (with DxO PureRaw plugin), currently undergoing a switch DxO PhotoLab 9. DxO doesn't support the RF 14 yet. But I also subscribed to Canon's NnIP for a year, since there's a 30 days trial period included. I use the latest version of the software (1.5.10) and it doesn't apply DLO on RF 14 RAWs. It indicates this with a specific output message btw. I will check this weekend how it handles my RF 20 files since I don't know if it's deliberate or a lack of proper lens profiles... I will also check how DPP handles these...

 

[PhotonShark]

Canon is simply making everyone else do the reverse engineering and math that Canon achieved during the design, manufacturing, and test processes themselves.

The problem is that their included software is just terrible by modern standards and I would guess that the overwhelming majority of users are editing their RAW files in other software.

When Canon sells you a lens which is designed to be digitally corrected, they should include (directly, or indirectly) the means by which you can get the image that they sold you. In other words, the lens plus the correction data.

As I also mentioned, this works for non-native lenses too. Where the camera could thus use the third party correction data for in camera .jpeg.

 

[neuroanatomist]

When Canon sells you a lens which is designed to be digitally corrected, they should include (directly, or indirectly) the means by which you can get the image that they sold you. In other words, the lens plus the correction data.

One could argue that they do just that, since with RF lenses the correction data for DLO are stored in the lens itself and transmitted to the camera.

6 Significant Features of RF Lenses

 

[AlanF]

One could argue that they do just that, since with RF lenses the correction data for DLO are stored in the lens itself and transmitted to the camera.

6 Significant Features of RF Lenses

I think he is implying that the correction data should be readable by other RAW converters like DxO, PS etc without their having to reverse engineer it.

 

[neuroanatomist]

I think he is implying that the correction data should be readable by other RAW converters like DxO, PS etc without their having to reverse engineer it.

Yes, that was clear. But…do we know those data aren’t readable by other developers? Have DxO, Adobe, etc., come out and stated that they cannot use Canon's in-lens correction profiles that are presumably written into the .CR3 files? Those developers have established workflows for building lens profiles that work for all manufacturers' lenses. I can certainly see those developers deciding that modifying their software and processes for Canon RF lenses alone would not be a worthwhile investment of resources, given that the current processes in place have been working and continue to work for them.

 

[AlanF]

Yes, that was clear. But…do we know those data aren’t readable by other developers? Have DxO, Adobe, etc., come out and stated that they cannot use Canon's in-lens correction profiles that are presumably written into the .CR3 files? Those developers have established workflows for building lens profiles that work for all manufacturers' lenses. I can certainly see those developers deciding that modifying their software and processes for Canon RF lenses alone would not be a worthwhile investment of resources, given that the current processes in place have been working and continue to work for them.

It would not surprise me if they were encrypted. Canon have now introduced a subscription model for using their higher level add-ons to DPP, and they like putting restrictions on 3rd parties.

 

[roby17269]

It would not surprise me if they were encrypted. Canon have now introduced a subscription model for using their higher level add-ons to DPP, and they like putting restrictions on 3rd parties.

have they for real? Who's going to pay to use DPP? those add-ons must be really enticing....

 

[AJ]

There is one advantage to files shot with barrel-distorted lenses I'd like to mention here, and it is this: there is more leeway for rotating images to correct leveling. For files without distortion, any rotation will result in having to crop. In contrast, distorted files can often be rotated quite a bit before additional cropping is necessary.

 

[AJ]

I wish there was a way for manufactures to share their lens correction data properly.

I wish that lens manufacturers, including Canon, would share this information readily and freely. You'd think it would be a competitive advantage to have corrections ready to go at dxo, lightroom, darktable, and whatnot, the very day a new lens hits the street. Yes there is DPP, and out-of-camera jpeg, but photographers are looking for lenses that fit their streamlined workflow.

As an example of how things can go south, there was a widely circulated "review" of the RF 14/1.4 recently whereby the authors used the 16/2.8 profile because they didn't have a profile for the new lens.

Lenses these days are a two-part system: the physical lens, and the software/profiles needed for mandatory corrections. It seems to me that for manufacturers, the value is in the lens and not the software or profiles. If anything, giving away the profiles adds value. Having companies like dxo retro-engineer the corrections seems counterproductive to me.

I'd even go as far as to say that when corrections are free and are easily and efficiently applied, checking a box with a single click of the mouse, without having to complicate the workflow, you may pull in people from the "hate distortions" camp to the "love it" camp.

 

[neuroanatomist]

I wish that lens manufacturers, including Canon, would share this information readily and freely. You'd think it would be a competitive advantage to have corrections ready to go at dxo, lightroom, darktable, and whatnot, the very day a new lens hits the street. Yes there is DPP, and out-of-camera jpeg, but photographers are looking for lenses that fit their streamlined workflow.

As an example of how things can go south, there was a widely circulated "review" of the RF 14/1.4 recently whereby the authors used the 16/2.8 profile because they didn't have a profile for the new lens.

I'd say that's down to the reviewer, though, since DPP is free, available, and uses the profile stored on the lens.

Having said that, I certainly agree with the idea that 3rd party support for new lenses is better sooner than later. But I'll also say that while Canon's DLO sounds impressive (using the mathematical model of the lens to correct the image), in practice when I compare DxO to DPP I find the former yields better image quality even in the extreme corners of a lens like the RF 14-35/4, where the most stretching is occurring.

Specifically regarding the RF 14/1.4, DxO will reportedly support that lens with their April update (was mentioned here and I confirmed that a DxO employee provided that information). That's pretty fast (faster than some prior lenses), but I'll still have the lens before DxO supports it because according to FedEx my copy is on a truck that just turned onto my street.

 

[AlanF]

have they for real? Who's going to pay to use DPP? those add-ons must be really enticing....

There certainly is:

https://www.canon.co.uk/pro/infobank/neural-network-technology/

I have asked twice here if anyone has paid to use it, but with no response.

 

[Klendothir]

There certainly is:

https://www.canon.co.uk/pro/infobank/neural-network-technology/

I have asked twice here if anyone has paid to use it, but with no response.

I've just started a subscription There’s a 30 days free trial period included, so I've decided to try it out.

 

[AlanF]

I've just started a subscription There’s a 30 days free trial period included, so I've decided to try it out.

Let us know the results.