DXO claim 1DX2 new class leader

Now, as the DR war seems to be solved some declare the war on DXO? Seems fighting is the important thing here, regardless against what.

DXO measures quite reasonable, the only problem for some is to understand the numbers. The values are sensorbased, only. And it's only meaningful to compare specific graphs with eachother, not the final number of "88" against "88".

Canon made a big step forward, showing what the nonsense ISO-specs of the D5 really are: marketing-BS, only.

Now as the 1DX2 has more DR@BaseISO than the Nikon dD5, DXO must be wrong, paid or insignificant. Funny, as they were the holy bible in the last few Sensorbashing-years.
 
Upvote 0
Halfniak said:
Well done Canon. ISO Sports score: 3207 !!

But... but... according to 'unbiased' DPReview:

"Noise performance falls slightly behind the Nikon D5 (and even the Sony a7R II when normalized) at very high ISOs."
- http://www.dpreview.com/news/8090146652/canon-eos-1d-x-mark-ii-studio-tests/2

Paging Rishi Sanyal. Rishi Sanyal.

;D ;D
 
Upvote 0
macVega said:
Finally.. Canon decides to give DxO some money, and that is a good thing because many people actually believe the crap DxO write on their site, Nikon has surely benefited from a sponsorship for a very long time now.. ???

nah.. Canon just designed a better sensor. Nothing biased or false about DXO's scoring.
 
Upvote 0
neuroanatomist said:
AlanF said:
neuroanatomist said:
9VIII said:
ahsanford said:
Of course, trying to understand DXO's overall score metric is tantamount to hitting oneself in the head with a hammer.

#ibuprofenplease

- A

I have no doubt that there is no formula.

I've seen an approximation:

DxOMark_Sensor_Score = 59 + 4.3*(ColorDepth-21.1) + 3.4*(DynamicRange-11.3) + 4.4*log2(ISO/663) -0.2

59-0.2 = 58.8 so shouldn't the eqn be:

DxOMark_Sensor_Score = 58.8 + 4.3*(ColorDepth-21.1) + 3.4*(DynamicRange-11.3) + 4.4*log2(ISO/663)

Yes, or there's a missing set of parentheses around a term. Not my formula, found in an article on DxO's website (reposted with permission, the author was not affiliated with them). The same author also suggested that the coefficients for the three sub scores might not be constant across all sensors.
I haven't found the reference Neuro mentions, but I believe the approximation formula was published in an essay on Luminous Landscapes by Peter van der Hamer, a Dutch physicist/engineer. He believes the reference point is the Lecia M8 which had color depth of 21.1, dynamic range of 11.3, and ISO of 663. Various weighting factors are applied and sensors which are better than the M8 score higher, poorer performing sensors score lower. The [ -0.2] term remains a mystery as it appears to be a typo in the original paper. The claim is that the formula is within 1 or 2 points of all DxO scores posted as of 2012 when the formula was published.

The trouble is that current generation sensors are all so good that measurement error (which is never acknowledged by DxO just like the real formula is never stated!) times these weighting factors may be just as significant as any real differences between sensors. Bad Science for sure!
 
Upvote 0
Woody said:
Halfniak said:
Well done Canon. ISO Sports score: 3207 !!

But... but... according to 'unbiased' DPReview:

"Noise performance falls slightly behind the Nikon D5 (and even the Sony a7R II when normalized) at very high ISOs."
- http://www.dpreview.com/news/8090146652/canon-eos-1d-x-mark-ii-studio-tests/2

Paging Rishi Sanyal. Rishi Sanyal.

;D ;D

DxO's ISO 'Sports' measure takes SNR *and* DR into account. I can't remember the exact details but they are available, for once, on DxO's website.

DPR on the other hand only talk about noise level (i.e. related to DxO's SNR measurements), not DR. At least I assume so.

So if the D5 has better SNR performance but worst DR at higher ISOs (which seems consistent with everything I've read about these cameras, but then again I have zero direct experience with them), it's perfectly possible for the D5 to do worse on the DxO metric, and yet have its praises sung by DPR.
 
Upvote 0
In the continuing debate of DxO being biased or honest, I give you this image. Nobody will convince me they're honest...I think the numbers they find are truthful inside their graphs, but their overall scores and main scores are entirely made up or use a rigged algorithm. These 2 APS-C sensors really fall behind the 1DX, yet the overall score would give the impression that they're competitive...to me, that's dishonest.
 

Attachments

  • image.png
    image.png
    815.3 KB · Views: 169
Upvote 0
NorbR said:
DxO's ISO 'Sports' measure takes SNR *and* DR into account. I can't remember the exact details but they are available, for once, on DxO's website.

DPR on the other hand only talk about noise level (i.e. related to DxO's SNR measurements), not DR. At least I assume so.

So if the D5 has better SNR performance but worst DR at higher ISOs (which seems consistent with everything I've read about these cameras, but then again I have zero direct experience with them), it's perfectly possible for the D5 to do worse on the DxO metric, and yet have its praises sung by DPR.

Except that when you look at DxO's measurements, you see that the 1D X II has better SNR than the D5 from ISO 400 all the way up to ISO 409600, which is the highest available setting on the 1D X II. In fact, the D5 has slightly more DR from ISO 3200 to 102400, and they're equal beyond that.

So, nope...DPR is just seeing what they want to see – bias.
 

Attachments

  • D5-1DXII-SNR.png
    D5-1DXII-SNR.png
    60 KB · Views: 2,127
Upvote 0
NorbR said:
So if the D5 has better SNR performance but worst DR at higher ISOs (which seems consistent with everything I've read about these cameras, but then again I have zero direct experience with them), it's perfectly possible for the D5 to do worse on the DxO metric, and yet have its praises sung by DPR.

Yet, on DPReview page where the comparison is done, I have examined every detail in the test scene and cannot tell where the D5 appears to have better SNR than 1DX2. The detail resolved and noise levels are so close it's not funny. Therein lies the mystery... ;D
 
Upvote 0
Woody said:
NorbR said:
So if the D5 has better SNR performance but worst DR at higher ISOs (which seems consistent with everything I've read about these cameras, but then again I have zero direct experience with them), it's perfectly possible for the D5 to do worse on the DxO metric, and yet have its praises sung by DPR.

Yet, on DPReview page where the comparison is done, I have examined every detail in the test scene and cannot tell where the D5 appears to have better SNR than 1DX2. The detail resolved and noise levels are so close it's not funny. Therein lies the mystery... ;D

I remember last time Rishi was in here... When he couldn´t convince people of the A7RII being superior at higher ISO, in a visual way, he started arguing with maths and formulas. How math is relevant in a visual comparison is yet to be explained... I´m sure bias couldn't have anything to do with it? ::)
 
Upvote 0
dilbert said:
DxO have performed tests in their labs that are repeatable and measurable to come to these conclusions.

What tests have you yourself done with all three cameras to show otherwise?

That's the problem, Dilbert. They're the only show in town* that is distilling sensor performance into numbers.

Unlike with lenses, where each lens that gets tested has 3-4 other places effectively doing the same test, DXO stands alone here, so they are the loudest voice on the subject. Everyone else just posts IQ samples, noise samples, etc. and lets photographers judge for themselves -- and that model of judgment can't be re-posted on Petapixel, DPR, etc. like DXO's work can.

- A

(*Yes, there are other places that do this but they get 1/100th the traffic because they tend to publish reports that resemble electrical engineering lab tests. Rank and file photographers can't process that information quickly or compare it to others in a simple web-driven interface.)
 
Upvote 0
dilbert said:
Larsskv said:
...
I remember last time Rishi was in here... When he couldn´t convince people of the A7RII being superior at higher ISO, in a visual way, he started arguing with maths and formulas. How math is relevant in a visual comparison is yet to be explained... I´m sure bias couldn't have anything to do with it? ::)

Eyes are easily deceived, hence optical illusions. The brain is also bad when it comes to "remembering."

You've seen the pictures with different shades of grey where the same color in one place looks dark than in another just because of what's around it? You could swear black and blue that one patch is darker but in actual fact it is not.

In the end, the best proof of something as absolute is math and science - not your eyes or brain. Ask any scientist about how reliable "eye witness" statements are vs something that is measurable/recordable.

Sorry, Dilbert. You failed in cinvincing me that I should stop using my eyes, and in stead rely on DPR maths, when determining the tecnical quality of a picture.
 
Upvote 0
Larsskv said:
dilbert said:
Larsskv said:
...
I remember last time Rishi was in here... When he couldn´t convince people of the A7RII being superior at higher ISO, in a visual way, he started arguing with maths and formulas. How math is relevant in a visual comparison is yet to be explained... I´m sure bias couldn't have anything to do with it? ::)

Eyes are easily deceived, hence optical illusions. The brain is also bad when it comes to "remembering."

You've seen the pictures with different shades of grey where the same color in one place looks dark than in another just because of what's around it? You could swear black and blue that one patch is darker but in actual fact it is not.

In the end, the best proof of something as absolute is math and science - not your eyes or brain. Ask any scientist about how reliable "eye witness" statements are vs something that is measurable/recordable.

Sorry, Dilbert. You failed in cinvincing me that I should stop using my eyes, and in stead rely on DPR maths, when determining the tecnical quality of a picture.

Eye witness statements aren't usually anything to do with science, they are usually to do with law cases or claims about UFOs etc. As a scientist who has spent most of his career trying to "prove" mechanisms, I can assure you that direct observation is usually the closest we can get to "proof" because the evidence is real and not indirect from deduction. Indirect evidence can always be interpreted in different ways by different theories. All of this doesn't mean I don't accept measurements - I use them all the time, and the more accurate the better - it's that experimental observation is how you test different theories.
 
Upvote 0
AlanF said:
Larsskv said:
dilbert said:
Larsskv said:
...
I remember last time Rishi was in here... When he couldn´t convince people of the A7RII being superior at higher ISO, in a visual way, he started arguing with maths and formulas. How math is relevant in a visual comparison is yet to be explained... I´m sure bias couldn't have anything to do with it? ::)

Eyes are easily deceived, hence optical illusions. The brain is also bad when it comes to "remembering."

You've seen the pictures with different shades of grey where the same color in one place looks dark than in another just because of what's around it? You could swear black and blue that one patch is darker but in actual fact it is not.

In the end, the best proof of something as absolute is math and science - not your eyes or brain. Ask any scientist about how reliable "eye witness" statements are vs something that is measurable/recordable.

Sorry, Dilbert. You failed in cinvincing me that I should stop using my eyes, and in stead rely on DPR maths, when determining the tecnical quality of a picture.

Eye witness statements aren't usually anything to do with science, they are usually to do with law cases or claims about UFOs etc. As a scientist who has spent most of his career trying to "prove" mechanisms, I can assure you that direct observation is usually the closest we can get to "proof" because the evidence is real and not indirect from deduction. Indirect evidence can always be interpreted in different ways by different theories. All of this doesn't mean I don't accept measurements - I use them all the time, and the more accurate the better - it's that experimental observation is how you test different theories.

Just to add, I've spent a spent a significant amount of time engaged in analysis of scientific images. While measurement is certainly important from a quantitative standpoint, in my experience if you can't see a qualitative difference with your eyes, a measurable quantitative difference is very unlikely to be meaningful in real world terms.
 
Upvote 0