DXOMark concludes that the Canon EOS R3 is the ‘best low light performer’

AlanF

Stay at home
CR Pro
Aug 16, 2012
9,100
12,823
Where are you getting a 0.42 stop difference in SNR? That would correspond to a ~2.5dB difference in DxO's measurements when what they've presented is more like a 0.5dB difference.
Go back to my post number 18 where DxOmark's numbers are in the image. The R3 has a low light iso of 4086 calculated by DxOmark from their S/N data, the R5 is 3042. The difference in stops (ev) is log(base2) of 4086/3042 = 0.4256.
Note the iso for the R6 is 3394, only 20% poorer than that of the R3, and for which I calculated 0.27 ev. I think my calculations must be consistent with DxOmark as they write: "A difference in low-light ISO of 25% equals 1/3 EV and is only slightly noticeable." https://www.dxomark.com/dxomark-camera-sensor-testing-protocol-and-scores/#sports
 

tron

EOS-1D X Mark III
CR Pro
Nov 8, 2011
5,006
1,371
Many thanks Alan for this. It seems more obvious if someone decides to put round numbers like 1000 and 4000 iso.

Log(base2) (4000/1000) = Log(base2) 4 = Log(base2) 2^2 = 2 stops <--- It's a nice way to explain to others and it kind of puts things into perspective. For example using your formula I found out that my R5 is about 1/5 stop better at low light than my D850 = practically equal.
 
  • Like
Reactions: RaPhoto

raptor3x

EOS RP
Jan 26, 2012
626
112
State College, PA
whumber.com
Go back to my post number 18 where DxOmark's numbers are in the image. The R3 has a low light iso of 4086 calculated by DxOmark from their S/N data, the R5 is 3042. The difference in stops (ev) is log(base2) of 4086/3042 = 0.4256.
Note the iso for the R6 is 3394, only 20% poorer than that of the R3, and for which I calculated 0.27 ev. I think my calculations must be consistent with DxOmark as they write: "A difference in low-light ISO of 25% equals 1/3 EV and is only slightly noticeable." https://www.dxomark.com/dxomark-camera-sensor-testing-protocol-and-scores/#sports
That's what I thought, unfortunately it's an incorrect reading of the DxO low light score and really goes to show why the low light score is such a poor metric. The SNR is a component of the DxO low light score, but that score is determined by interpolating out the highest ISO setting that will satisfy a minimum value for three metrics: SNR, DR, and color depth. It's really the color depth one that tends to throw things off. Like I said earlier, your estimate of ~0.42 stops improvement in SNR between the R3 and R5 from the score is way off. DxO actually provides the SNR data and the reality is that the difference between the R5 and R3 is some where between 0.5-1dB, which corresponds to 1/12th to 1/6th of a stop. Nowhere close to 1/2 a stop.
 
  • Like
Reactions: AlanF

dtaylor

Canon 5Ds
Jul 26, 2011
1,764
1,346
To it into perspective, if it does mean anything the way it is defined by them, the R3 is 0.27 stops superior to the R6 and 0.42 stops to the R5 at the lowest light levels. And, for the overall score, DxO doesn't include the number of Mpx.
This coincides with the test samples at DPReview. I can't see a dime's worth of difference between the R3, R5, and R6 at high ISO looking at areas of solid tone (like the color chart). In areas of detail the R5 comes off a little better, naturally because it's got 45mp to work with. But noise wise they look the same. This is scaled to the same view size of course.

I don't doubt that DxO is detecting the small difference they are reporting, it's just difficult to observe outside testing software because it's <0.5 stop across the board.

This doesn't detract from the R3. It's a fantastic camera with a sharp sensor and killer features for the sports/action niche. It just continues the trend we saw in the 2010's: high ISO gains are small because for over a decade high ISO has been dominated by shot noise.
 
  • Like
Reactions: docsmith and AlanF

AlanF

Stay at home
CR Pro
Aug 16, 2012
9,100
12,823
That's what I thought, unfortunately it's an incorrect reading of the DxO low light score and really goes to show why the low light score is such a poor metric. The SNR is a component of the DxO low light score, but that score is determined by interpolating out the highest ISO setting that will satisfy a minimum value for three metrics: SNR, DR, and color depth. It's really the color depth one that tends to throw things off. Like I said earlier, your estimate of ~0.42 stops improvement in SNR between the R3 and R5 from the score is way off. DxO actually provides the SNR data and the reality is that the difference between the R5 and R3 is some where between 0.5-1dB, which corresponds to 1/12th to 1/6th of a stop. Nowhere close to 1/2 a stop.
My calculations from the DxOmark's sports iso data are done exactly the same way as they do theirs and give the same results. My gut reaction is that you are probably right that the difference is only 1/12th to 1/6th of a stop. The best way to check is by experiment. @privatebydesign showed from dpr's comparison tool that the noise in RAW files at iso12800 from the R3 seemed no better than those from the R5, R6 and 1DXIII when viewed at the same size. I've done the same at at iso51k, to check at very high iso where tiggy had some data, and the same as pbd found seems true. It certainly looks like your estimates of less than 1/12th stops difference seems very reasonable and close to thee truth. My gut reaction was that I found it difficult to believe that under conditions when the noise is primarily generated from just the statistical fluctuations in photon flux that sensors can differ greatly in S/N when images are viewed at the same output size. Maybe you should take it up with DxOmark that their low light score and their interpretation is a misleading metric.
Screenshot 2021-12-07 at 16.36.42.png
 
  • Like
Reactions: tron

AlanF

Stay at home
CR Pro
Aug 16, 2012
9,100
12,823
This coincides with the test samples at DPReview. I can't see a dime's worth of difference between the R3, R5, and R6 at high ISO looking at areas of solid tone (like the color chart). In areas of detail the R5 comes off a little better, naturally because it's got 45mp to work with. But noise wise they look the same. This is scaled to the same view size of course.

I don't doubt that DxO is detecting the small difference they are reporting, it's just difficult to observe outside testing software because it's <0.5 stop across the board.

This doesn't detract from the R3. It's a fantastic camera with a sharp sensor and killer features for the sports/action niche. It just continues the trend we saw in the 2010's: high ISO gains are small because for over a decade high ISO has been dominated by shot noise.
My post overlapped in the ether with yours, and with a similar message. Frankly, those small differences would be hardly noticeable, and I think @raptor3x is right that it is even less. Anyway, the pluses of the R3 should be in other features, like its AF and speed of sensor readout, not IQ.
 
  • Like
Reactions: tron and dtaylor

raptor3x

EOS RP
Jan 26, 2012
626
112
State College, PA
whumber.com
Maybe you should take it up with DxOmark that their low light score and their interpretation is a misleading metric.
This is exactly the issue. Your way of calculating the relative performance between the R5 and R3 makes perfect sense but it only works if the DxO lowlight/sports score tells us something meaningful. I've been harping on them about this for a good 10 years now but the way they have that metric defined it's essentially worthless in any general sense. That said, the underlying data they collect that goes into it is quite good.
 
  • Like
Reactions: RaPhoto and AlanF

Fischer

EOS RP
Mar 17, 2020
319
230
My post overlapped in the ether with yours, and with a similar message. Frankly, those small differences would be hardly noticeable, and I think @raptor3x is right that it is even less. Anyway, the pluses of the R3 should be in other features, like its AF and speed of sensor readout, not IQ.
This is also why we can expect that even a very high MPIX Canon body (80-90 MPIX) would not carry any noticeable high iso penalty - if any at all. Just like with the 5DsR.
 

AlanF

Stay at home
CR Pro
Aug 16, 2012
9,100
12,823
Joules has started another thread about data from photons to photons, and the shadow improvement charts etc bear out what we have concluded here, not much difference up to iso about 40-50k, where the R3 draws ahead. The R5 is isoinvariant from iso 800 (or lower) whereas the R3 from iso 1273.
 
  • Like
Reactions: tron

AlanF

Stay at home
CR Pro
Aug 16, 2012
9,100
12,823
50k iso underexposed by one stop is equivalent to iso 100k. The official iso range of the R5 is up to 51k wheres the R3 and the R6 both go up to 102k, so you really are taking this to beyond the extreme range of the R5. It would be interesting to see how the R3 does vs the R6. If you are working at these very high isos, then the resolution of a high density sensor is wasted in any case as the noise wipes out fine detail.
The results in the previous post bear this out: the R5 is designed to work up to 51k and the R3 to 102k. Though not many would want to go that high!
 
  • Like
Reactions: tron