DXOMark tests the Canon EOS R image sensor, scores it at 89

When I found this out I immediately wrote them off entirely.
Same thing for me when I looked closer how they measure resolution. They invented some 'perceptual megapixels' but didn't disclose the exact measurement methods. That rings a queen of all bells. DxO measurements cannot be verified and reproduced so you can call them unscientific.
 
Upvote 0
Jan 29, 2011
10,675
6,121
I think anybody with half a brain can reasonably discount DXO's conclusions, anybody who has spent time here knows I am no fan of theirs or their obvious bias. But where they are useful and generally accurate (with some glaring mistakes accepted in the normal magnanimous French way :rolleyes:/end sarcasm) is if you look at specific comparisons, Compare the R to the 5D MkIV in real life and you'll see the newer camera is not quite as able in the shadows at lowest iso's. How useful and how often that minuscule difference is is entirely moot. However their findings align with other sites like Photonstophots http://www.photonstophotos.net/Charts/PDR.htm#Canon EOS 5D Mark IV,Canon EOS R

The truth is the R is not as capable at lowest ISO's, by a very small margin, as the 5D MkIV. Same with the 6D MkII and 6D.
 
  • Like
Reactions: 1 user
Upvote 0
I made an earlier comment about shadow pushing being a component of the "Sports low-light ISO" score that wasn't quite accurate, after some further reading it seems that the "Sports low-light ISO" should actually be called:

Low light color depth OR low light dynamic range OR noise, whichever degrades past our threshold first.

The way this data is presented with the SNR 18% chart shown under the "Sports" section one might think at first glance the sports score is determined by noise, that's the way I always understood it ( incorrectly ). But " We have therefore defined low-light ISO as the highest ISO setting for a camera that allows it to achieve a SNR of 30dB while keeping a good dynamic range of 9 EVs and a color depth of 18bits. "

In the case of the Canon R it seems the low-light ISO score is actually determined by the color depth score even though they talk about shadow pushing being the distinguishing factor in their analysis.

Here's my serious question: Has anyone ever noticed "wow the colors at ISO 3200 just look drab on X camera vs Y camera? "
 
  • Like
Reactions: 1 user
Upvote 0
I'm one of the rare few in this crowd that consider DXO scores to be very informative. Having owned many different Canon bodies and the current owner of a 70D, 5D3, A7r and a7r3 I think the broad scores provided by DXO are very accurate to my own experiences. I have found many quirks in the operation and useability of the Sonys that isnt published anywhere online and which makes the playing field much more level in reality than the scores suggest. However, simply in terms of IQ, the Sonys are well beyond the Canons in very much the levels and areas given by the DXO figures.
 
  • Like
Reactions: 1 user
Upvote 0
Jul 21, 2010
31,099
12,863
That's literally how they test the lenses: on the bodies. I don't know how you think they test lenses otherwise.
Incidentally, there's a perfectly good way to test a lens' performance that doesn't require a camera body. It's called an optical bench, and it's what is used to test lenses by LensRentals, Zeiss, and anyone else who wants to empirically determine a lens' MTF and a variety of other performance metrics.

 
  • Like
Reactions: 1 users
Upvote 0

Jethro

EOS R
CR Pro
Jul 14, 2018
996
1,037
Still, the silver lining here for me is that this means they’re close to EOS R support for DxO PhotoLab, which is my RAW converter of choice. I actually went to RAW+JPG for my first outing with the EOS R, since DxO support for .CR3 isn’t slated until April.
EOS R is supported in the latest (as of a week or so ago) update to PhotoLab - it works fine now.
 
Upvote 0
I'm one of the rare few in this crowd that consider DXO scores to be very informative. Having owned many different Canon bodies and the current owner of a 70D, 5D3, A7r and a7r3 I think the broad scores provided by DXO are very accurate to my own experiences
...
simply in terms of IQ, the Sonys are well beyond the Canons in very much the levels and areas given by the DXO figures.

Sorry for the following barrage of questions, my intentions aren't hostile, given your experience I'm hoping you can give some real world feedback.

Do you notice the color depth difference between the A7R and the 5D iii in a blind test? ( you should be able to if the dxo mark score is useful, color depth seems weighted heavily in their tests)

Do you notice a high iso noise difference between the A7R and the 5D iii in a blind test ? ( A7r wins 2746 to. 2293 low-light scores, but they have virtually the same SNR measurements so you they should look the same in terms of noise)

The overall sensor score is 95 to 81 for the A7R over the 5D iii, would you say that is good indication of how much better the A7R is over the 5D iii in terms of image quality? Do you immediately pick out shots from the 5D iii and say "I wish I would have shot that with the A7R" ?

The R scored 2742 on the high iso test while the a7 iii scored 3730. At first glance based on those scores you'd assume the sony high iso performance trounces the canon. But then in their own writeup they state:
" the values are so close at higher ISOs that it’s unlikely you’d be able to distinguish among them"

I'm legitimately wondering how these scores translate to the real world.
 
Last edited:
Upvote 0
Jul 21, 2010
31,099
12,863
I'm legitimately wondering how these scores translate to the real world.
Somewhere they state (or used to state) that a difference of 5 points was essentially meaningless.

How applicable their scores to the real world depends on your world. That applies to their sensor scores, but it’s easiest to see with their lens scores. Their lens score tests are all done in 150 lux illumination with a shutter speed fixed at 1/60 s. So if your world is a dimly lit warehouse, the scores are useful. That explains why, even though the 600/4 II outperforms the 50/1.8 on every optical measurement, the 50/1.8 gets a higher score...if I was going to shoot portraits in dim warehouse lighting, I’d pick the 50/1.8, too.
 
  • Like
Reactions: 1 user
Upvote 0

dtaylor

Canon 5Ds
Jul 26, 2011
1,805
1,433
DxO's dynamic range scores appear to be internally consistent. I don't think they perfectly match classic transmission step wedge tests, but they do give you a good idea of how cameras relate. That said, I think DR is way overhyped. Most people truly do not maximize the cameras they have, or understand what a stop difference means in practical use. Hearing it discussed over and over again is exhausting. Even the YouTube e-celebs who originally championed it, back when it was the 5D3 with banding vs. the D800, have started to say the differences don't matter now.

DxO's sports score is next to useless for the reasons digitalride pointed out.

The portrait or color depth score is absolutely worthless, as is the overall sensor score which ignores one of the most important aspects of IQ: resolution.

Their lens scores are observably wrong and are reported using pseudo-scientific terminology. (Megapixels is a statement of sensor sampling frequency, not resolved detail. "Perceptual megapixels" makes no sense and has no perfect reference point for any given MP rating.)

tl;dr - I ignore pretty much everything DxO claims.
 
Upvote 0
Mar 25, 2011
16,848
1,835
When I got my R, I put it on my light table, and compared a series of photos using the same lens shot from the same place under as identical conditions as possible. That meant using a EF lens on the R, so the lenses would be the same.

I saw a very slightly poorer quality image from the R, but it was very slight and impossible to determine if it was due to Lightroom, or the sensor. For all practical purposes, they were identical. I'm not certain if I kept the images.
 
Upvote 0
Sorry for the following barrage of questions, my intentions aren't hostile, given your experience I'm hoping you can give some real world feedback.

Do you notice the color depth difference between the A7R and the 5D iii in a blind test? ( you should be able to if the dxo mark score is useful, color depth seems weighted heavily in their tests)

Do you notice a high iso noise difference between the A7R and the 5D iii in a blind test ? ( A7r wins 2746 to. 2293 low-light scores, but they have virtually the same SNR measurements so you they should look the same in terms of noise)

The overall sensor score is 95 to 81 for the A7R over the 5D iii, would you say that is good indication of how much better the A7R is over the 5D iii in terms of image quality? Do you immediately pick out shots from the 5D iii and say "I wish I would have shot that with the A7R" ?

The R scored 2742 on the high iso test while the a7 iii scored 3730. At first glance based on those scores you'd assume the sony high iso performance trounces the canon. But then in their own writeup they state:
" the values are so close at higher ISOs that it’s unlikely you’d be able to distinguish among them"

I'm legitimately wondering how these scores translate to the real world.

No worries, I'll try my best to answer them.

Color depth is a tough one for me for two reasons. Firstly I dont feel I shoot massive areas of similar colour sufficiently often in real world scenarios to have a full appreciation for the limitations of either system in this area (my photos tend to be fairly 'busy') and secondly because I dont have a clue what color depth actually refers to (reading up about it doesnt help either!). I will say I have encountered serious banding at base ISO with my Canon 5D3 (and even made a post about it many years ago) but have not had the same issue with the a7r but then maybe because my post processing workflow improved. In short I cant make a solid statement about color depth.

High ISO noise I would say is similar between the two systems but a7r wins out when downsized to 5D3 sizes. Having owned a 50D, 5D3, 7D and 70D I would say the ISO score DXO gives has been consistent in reflecting my own maximum tolerance ISO levels for each of those cameras. I'm also not sure if this is the right place to mention it but the processing latitude the a7r gave over the 5d3 was in a different league. The slightest adjustment to shadows in the 5d3, even at low ISOs, would introduce pretty horrible, patterned banding. The amount I could manipulate the a7r files was/is miles ahead.

I would say 95 to 81 is an accurate indication from my experience. The images are just so much crisper and more detailed from the a7r (of course the higher resolution has an impact on this). For every shot I have ever taken I wish I had taken it with the highest resolution medium format camera in existence so I'm not quite sure how to answer the second part of that question. Yes I wish I had taken every shot on the 5D3 on my a7r because the quality is better and the resolution higher, but as I mentioned in my post the a7r has usability quirks which the 5D3 doesnt have, plus at the time i was heavily invested in Canon lenses (and still am) so this also meant for anything other than landscapes I had to use my Canon as the a7r simply wouldnt have coped.

I cant comment on the R as I've never seen it nor taken photos with it. I would add that what I write above is purely based on a simple image quality comparison and doesnt go into the real world useability of the Sonys versus the Canons, for which the a7r was extremely limited (laggy, terrible battery etc). The a7r3 is massively improved and suitable for any type of photography but still has operational quirks (turning itself off at high temps, write lags, poor interface) that I've never had to deal with in Canon.

Hope this makes some sense of my position.
 
Upvote 0