DxOMark vs. Reality

[well_dunno]

hmmm, funny how this thread has developed - Neuro's original post was stating DxO has been and is showing scores that suggest sensors in Nikon bodies are better however the reality, that is the economic reality, suggests Canon has been selling more and presumably having better business. Out of this, the logical inferences should be something like the below:

- Those who purchase Canon cams do not look at or do not care about DxO scores
- Those who purchase Canon cams do value DxO scores but other parts of the cameras in which Canon is better are more important to them
-  They are too heavily invested in Canon gear to change brand (this would naturally not explain any increasing lead for Canon) etc etc
if we put the rational choice theory aside the list can be made longer I assume, though I cannot see any discussion about to what extent DxO scores reflect image quality from the sensors  in Neuro's opening. The thread seems to have come off topic...

Those of you who defend DxO scores, why do you have Canon gear?

Cheers!

[PackLight]

So, this discussion, which one has mounted the biggest d1ck... sorry, sensor, in a camera case, is total bu11shit. I think, none of you has ever come into a situation, when he thought stuff like "crap, the dynamic range in this picture is too low. The customer will be very unpleased. I wish, i had a Nikon."

Yes and No

No customer as I am not a Pro Photog

And

Yes, many times a higher DR would have been nice.

[elflord]

hmmm, funny how this thread has developed - Neuro's original post was stating DxO has been and is showing scores that suggest sensors in Nikon bodies are better however the reality, that is the economic reality, suggests Canon has been selling more and presumably having better business. Out of this, the logical inferences should be something like the below:

- Those who purchase Canon cams do not look at or do not care about DxO scores
- Those who purchase Canon cams do value DxO scores but other parts of the cameras in which Canon is better are more important to them
-  They are too heavily invested in Canon gear to change brand (this would naturally not explain any increasing lead for Canon) etc etc
if we put the rational choice theory aside the list can be made longer I assume, though I cannot see any discussion about to what extent DxO scores reflect image quality from the sensors  in Neuro's opening. The thread seems to have come off topic...

Those of you who defend DxO scores, why do you have Canon gear?

See my post elsewhere in this thread.

http://www.canonrumors.com/forum/index.php?topic=10020.msg181164#msg181164

The sensor is an important piece, but it isn't everything. (essentially (2) on your list). I started with Canon because I knew other Canon users. Even then, I was familiar with review sites like snapsort that take DxO scores into account, so Canon needed to deliver a respectable performance.

When I went to upgrade, I could have switched (I had a rebel and two non-L lenses). However, for my application, high ISO performance is much more important than dynamic range at base ISO. So there's a very substantial benefit to having a full frame camera, but as far as I'm concerned, a D800 would not be any better than a 5DIII (indeed, not noticably better than my 5DII) I also think Canon have some compelling advantages over other brands (see my other post)

[neuroanatomist]

give one example there Canons sensor is better

Ok, you like wide DR.  I need to shoot at high ISO.  If I need to shoot at ISO 12800, which sensor offers a wider DR - the D800 or the 5DIII?

do you   understand the benefit and  the choiche of underexposure and moving middle grey down 1-4 levels and bring in highligts far above 3.5 stops .

Yes, it's one of several ways to extract the maximum amount of information from a RAW file.  But the bottom line is that you can't extract more information than is there to begin with.

Oly OM-D, because DxOMark says it's better than a 7D anyway, right?  ) . 

OM-D sensor could well be better on some measurements (it's a Sony sensor), but if ergonomics with long lenses, and AF performance are a factor, it's a slamdunk for the 7D. Also, since you mentioned shooting with a 300mm lens -- on Canon, you could shoot with the 300mm f/4, the 300mm f/2.8, the 70-300L, the Sigma 120mm-300, or the Canon 100mm-400mm to name a few. I'm not sure what lens you'd use on the olympus.

Which gets back to other points made in this thread -- the system consists of much more than a sensor.

Exactly.  Picking the Civic over the Corvette (was that analogy even in this thread?!?) makes sense if fuel economy or small parking spaces are your biggest needs, picking the Corvette over the Civic makes sense if you engage in street racing, and neither of them make sense if you need to haul your 2-ton yacht to the harbor.

Those of you who defend DxO scores, why do you have Canon gear?

Excellent question! 

Maybe it's a nuance, but speaking for myself, I defend DxO's measurements (usually), but not their scores.  Their scores do not affect my buying decisions because I understand how they generate them (mostly, within the limits of their published methods, which lack some details), and I know how to interpret their underlying raw data for myself and draw my own conclusions based on them.  I also use DxO software for RAW conversions (and I'm pleased as punch that as of today it supports the 1D X, because now I can get away from the kludgy DPP user interface).

Some of the reasons I have Canon gear are that they have lenses that better meet my needs, I prefer the ergonomics of the Canon bodies and menus, and I detest the color yellow.

[Meh]

I love Canon cameras and lenses.  There's a certain sexiness to Canon.  Nikon seems so utilitarian... they get the job done but I don't get the same feeling from it.  I need that feeling.  I don't get that feeling from DR at ISO100. 

[ankorwatt]

Guys, you've got a real problem, when you assess camera sensors basing on some stupid numbers like DxO.

If you're a professional photographer (a skilled one) then none of your customers will ever see the difference in the IQ/dynamic range/whatever between a Nikon and a Canon DSLR.

So, this discussion, which one has mounted the biggest d1ck... sorry, sensor, in a camera case, is total bu11shit. I think, none of you has ever come into a situation, when he thought stuff like "crap, the dynamic range in this picture is too low. The customer will be very unpleased. I wish, i had a Nikon."

Get a life.

A skilled photographer is not the same as  he or she  is a good  copyist and know how a digital work flou works.  Have you  anytime worked in a darkroom with a negative film  then you will understand what I mean and likewise it is with a raw files.

A skilled photographer with also skills in the digital "dark room" kan do so much more than a photographer with no  photoshops skills.  And even more with a camera who has 14 stops DR.
best regards , pro since 30 years back.

[ankorwatt]

Ok, you like wide DR.  I need to shoot at high ISO.  If I need to shoot at ISO 12800, which sensor offers a wider DR - the D800 or the 5DIII?


At 6400 they are equal, at 12800 I need to rezise the d800 size to 5dmk3 size and they are very similar, above 12800 there is small advatage to 5dmk3 BUT with little more NR they look both OK
As you can se here blue d800  red 5dmk3

[LetTheRightLensIn]

@tnargs, if anything, that just makes the main point of post #1 even stronger.  If DxO truly means that you can only use their scores to compare sensors of similar resolution, that make their results even more meaningless in the real world.  Furthermore, that begs the question - why normalize at all, if you can only compare sensors of simlar resolutions, normalization is moot. 

Exactly.

When I open up Road & Track and compare braking, 1/4 mile, and 0-60, 0-100 for a Ferrari versus a Hyundai, those tests stand up.

When I want to compare CPUs, I can use PassMark to see a plethora of different criteria and I can compare a Intel Celeron M 600Mhz to an Intel Core i7 3960X and QUANTITATIVELY see performance.

Granted in the CPU realm, Motherboard throughput will play a roll, but the speed of the calculations, etc is measurable, definable and COMPARABLE across generations.  So if I can compare the power of a 8 core CPU to a CPU from 8 years ago and measure the difference, how can I not DEFINITIVELY MEASURE a 36 MP sensor to a 10 MP sensor?

If you can't, then the TEST is MEANINGLESS.

The more I read the more I see just how flawed the DxOMarks scores are.  Anyone can produce DATA.  But data is not information. 

What DxOMarks lacks is RELEVANCE

You must be blind then if you can't see the real world difference the extra DR can make in real world properly exposed shots taken with D800 vs 5D3. It's plenty relevant for some shots.

[LetTheRightLensIn]

DxOMarks are useless and only made to entertain hobby photographers.

Yay! I love to run around bragging about how ignorant I am! Yay!

[LetTheRightLensIn]

I do agree that it's best to look at their plots and RAW numbers and not there overall scores since they may not weigh various factors the way you would plus there isn't really any a way to look at so many different aspects at once and come up with one universal score to give meaning to it all at once.

I also agree that their lens testing appears to leave MUCH to be desired. Lens testing is done by different people there and it's much trickier to carry out well. I ignore their lens tests but their sensor data generally seems to be pretty solid.

OK maybe DxO tests have not crashed the market for Canon yet, but as you say they, and general more over the top crippling of silly little things, sure have made their user base restless as you can tell but peeking at any forum on the web. Switching systems is a big deal for many (it's a pain, costs some money, some don't like the Nikon UI as much, Nikon doesn't have 70-200 f/4 IS and such they are mostly all huge fast stuff or consumer stuff) and it can take a while for things to build up to make many switch, maybe they did not get there quite yet, but they could be heading there. Also the bulk of sales are probably in the Rebel market. Also the 5D2 was aided a lot by the video guys nabbing them like mad and Nikon hasn't had Exmor in FF smaller, more reasonably priced bodies until recently, etc.

And whoever markets things better often matters even more. Apple IIe and IBM PC were utter trash as was Windows for a long time (and even now to an extent) and yet that stuff sold and the far more advanced stuff, sadly, did not thanks to horrible management and a computer press often too easily bought and paid for.

[ankorwatt]

Oh boy, what a thread.

yes and mybe some have learn something
1 there is noting like a right exposure
2.How you expose depends on the motive and the  motives DR and what camera you use
3.With a camera and 14 stop DR you can cover more from the topp to the bottom, and adjust with curves and other tools and  even mark an area and do a selektive adjustments to show more information from shadows  for example.

[ankorwatt]

This is 5dmk2 and d800 shooten against sun and clouds, exposed so the sky and clouds are visible,
same time, f-stop and 100iso
both images  lifted  in the same way in CR and then some work in photoshop so we should be able to see country landscape.

DR   and freedom from banding/pattern noise is what I wish  from my next Canon

[PackLight]

This is 5dmk2 and d800 shooten against sun and clouds, exposed so the sky and clouds are visible,
same time, f-stop and 100iso
both images  lifted  in the same way in photoshop so we should be able to see country landscape.

DR   

Sorry but this set is bull. Not a believable comparison at all.

[Meh]

Oh boy, what a thread.

yes and mybe some have learn something
1 there is noting like a right exposure
2.How you expose depends on the motive and the  motives DR and what camera you use
3.With a camera and 14 stop DR you can cover more from the topp to the bottom, and adjust with curves and other tools and  even mark an area and do a selektive adjustments to show more information from shadows  for example.

It's a nice thought you have that somebody might have learned something... but sadly the evidence suggests that somebody hasn't learned a thing.

[neuroanatomist]

At 6400 they are equal, at 12800 I need to rezise the d800 size to 5dmk3 size and they are very similar, above 12800 there is small advatage to 5dmk3 BUT with little more NR they look both OK
As you can se here blue d800  red 5dmk3

Wait, I thought we were talking about DxO's oh-so-reliable-even-with-normalization data?