December 08, 2016, 01:02:13 AM

Author Topic: DxOMark scores for 5DMkIII out - total score 81, 5DMkII had 79  (Read 83203 times)

peederj

  • Canon 70D
  • ****
  • Posts: 293
Re: DxOMark scores for 5DMkIII out - total score 81, 5DMkII had 79
« Reply #75 on: April 19, 2012, 12:07:00 PM »
Given neuro's explanation, quite a bit of the 5d3 advantage over the 5d2's DxO score would be similarly the increase from 21 to 22MP...something we would find photographically irrelevant but would be picked up on this test.

And given that effect, Canon's sensor technology didn't exactly improve on a Moore's law rate. The Jpeg in-camera processing did though.

If the lenses were freely interchangeable a lot of the hysteria would go away. I am happy enough with the 5d3 for stills. I look forward to more quantification of the 5d3 vs others.

canon rumors FORUM

Re: DxOMark scores for 5DMkIII out - total score 81, 5DMkII had 79
« Reply #75 on: April 19, 2012, 12:07:00 PM »

bornshooter

  • 5D Mark III
  • ******
  • Posts: 702
  • Love L series glass
    • my flickr
Re: DxOMark scores for 5DMkIII out - total score 81, 5DMkII had 79
« Reply #76 on: April 19, 2012, 12:15:19 PM »
DXO is run by nikon fanboys take this with a very very very small pinch of salt!

Ahhh...and you have proof to support that serious allegation?

Honestly, everyone should take a step back and consider what the DxOMark scores mean - and don't mean.  They are quite transparent about their scoring metrics and methods, and there's no reason to suspect there's collusion or favoritism occurring.  At the same time, it's important to remember that the sensor score is just that - a score of the sensor itself, not a 'camera score'. 

Furthermore, keep in mind that all of their scores are based on reducing the image to 8 MP - thus, the greater the starting resolution the more downsampling, which means lower apparent noise.  That lower apparent noise means apparently better ISO performance and apparently more dynamic range (lower apparent noise lowers the 'floor' for the DR estimate).  If you like, you can substitute 'artifically inflated' for 'apparently'. 

They have a page describing the mathematics of the normalization to 8 MP, and on that page, there's the following statement:

What should be remembered is that doubling the resolution adds:
3dB to the normalized SNR
0.5 bit to the normalized DR
0.5 bit to the normalized TR
1.5 bit to the normalized CS.


So, compared to 8 MP the D800 is 4.5-fold higher (just over two doublings), whereas the 5DIII is 2.75-fold higher (just over one doubling).  What that means, mathematically, is that the D800 has approximately one extra doubling of resolution relative to DxOMark's 8 MP normalized value - that accounts for all of the differences in the scores for both ISO and Color Depth, and part of the difference in dynamic range. 

Note that DxOMark does provide the non-normalized data, they just don't use those data to calculate the overall scores, the rationale being that normalizing to 8 MP allows appropriate comparisons.  In one sense, it does - if you're going to print 8x10" images all the time, then their scores actually apply pretty well.

Let me give a specific example for color sensitivity, which is the basis for DxOMark's Portrait Score.  That's one area where the D800 with 25.3 bits 'beats' the 5DIII with 24-bits.  Below is the comparison without normalizing to 8 MP, where you can see that the D800's advantage pretty much entirely disappears.
serious allegation lol yeah take me to court hahaha liste neuro you go play with your charts im going out shooting!

bornshooter

  • 5D Mark III
  • ******
  • Posts: 702
  • Love L series glass
    • my flickr
Re: DxOMark scores for 5DMkIII out - total score 81, 5DMkII had 79
« Reply #77 on: April 19, 2012, 12:20:22 PM »
anyway if people here are so disappointed with the results sell all your gear and go to nikon we wont miss you here let me know where you advertise your second hand stuff :D

dswatson83

  • EOS M3
  • ****
  • Posts: 243
Re: DxOMark scores for 5DMkIII out - total score 81, 5DMkII had 79
« Reply #78 on: April 19, 2012, 12:24:59 PM »
All the DXO score tells us (if you even believe it) is that at 8MP in perfect lighting, the Nikon D800 SENSOR is a bit better in certain areas. This test should make DXO rethink their testing though as a result of the Nikon resulting in higher sports high iso results when there isn't a single photographer that would ever say that. The D800 may be impressively close, but it isn't better in photos at high ISO speeds.

What I really want to know is how the Nikon competes at 36MP...that is going to tell me what I want to know which is if it was better for Nikon to put a 36MP chip into the camera or stick with the smaller 16MP chip in the D4. The DXO result is like saying a Ferrari is worse at high speeds to a Lexus because while running the test at a top speed of 40mph, the Lexus ran smoother compared to the Ferrari. I bet my lexus would be better at towing than a F150 if you ran a test towing 20lb worth of bicycles. Try towing 3000lb and tell me which one wins. Stupid DXO tells me nothing I want to know.
« Last Edit: April 19, 2012, 12:26:55 PM by dswatson83 »

PhilDrinkwater

  • Guest
Re: DxOMark scores for 5DMkIII out - total score 81, 5DMkII had 79
« Reply #79 on: April 19, 2012, 12:30:40 PM »
It will be interesting to see just how the 1D X sensor fares when it is tested.  I've got a sneaking suspicion that Canon really haven't done an awful lot to the 21MP sensor design to produce the 22MP version in the 5D MkIII.  If true, one would wonder whether this is a deliberate ploy to protect the 1D X or simply concentrating resources on other projects.
I still think the 5d3 sensor has been improved "significantly" - just not as significantly as the d800. The high ISO DR is better. Pattern noise has been reduced.

I do wonder if you're right though - maybe they left some of the improvements for the 1dx. That may prove to have been a mistake.

Regardless of the fact that I think the 5d3 is an amazing camera, I do think the market will be more impressed with the d800 'cos they don't really look at the camera as a whole. In that regard, Canons marketing has messed up.

ctmike

  • Guest
Re: DxOMark scores for 5DMkIII out - total score 81, 5DMkII had 79
« Reply #80 on: April 19, 2012, 12:37:37 PM »
The numbers are the numbers; the approach seems reasonable... for the specific conditions they test. It's still related to the sensor only, and it doesn't take away from the generally awesome performance of the MkIII.

Canon does have a significant marketing problem on their hands now, though. People are going to read today and for the foreseeable future that : Nikon Sensor > Canon Sensor, Nikon 36MP > Canon 24MP, and D800 costs less than MkIII. Most people aren't going to get in to many philosophical debates about what the DxOMark scores actually mean, they are going to see the above and buy Nikon. Quite frankly, Canon is going to have this problem at the low end as well, with whatever they release to compete against the D3200.

I love Canon, but they got caught with their pants down, marketing-wise. Doesn't mean that their cameras take bad pictures. But taking great pictures is only part of the story when people decide what to buy.

kozakm

  • Guest
Re: DX0 Mark Canon 5D MkIII Review
« Reply #81 on: April 19, 2012, 12:54:55 PM »
The only test i've seen banding and color blotches in, they took a shot underexposed, then added 4-5 stops of exposure to show the banding & blotches. I'm fine in most cases with pixel peeping, but seriously, there is no real world example to when you would need to push the exposure that far. Those kinds of tests are rubbish.

Not true. You definitely don't need to add 4 to 5 stops. And real world example? Shooting at the bright sunlight with deep shadows. I also tried several Nikon raw files and din't notice behavior like this.

canon rumors FORUM

Re: DX0 Mark Canon 5D MkIII Review
« Reply #81 on: April 19, 2012, 12:54:55 PM »

skitron

  • Canon 7D MK II
  • *****
  • Posts: 517
Re: DxOMark scores for 5DMkIII out - total score 81, 5DMkII had 79
« Reply #82 on: April 19, 2012, 12:57:21 PM »
Personally, I think DXO provides a valuable service in posting their findings.

That said, two things: first, don't expect anything meaningful out of a single number (i.e. DXO mark) and second, when you DO look at the details, it's a good idea to keep things in perspective (i.e. the scale of the graphs matters):

http://www.dxomark.com/index.php/Cameras/Compare-Camera-Sensors/Compare-cameras-side-by-side/(appareil1)/795%7C0/(brand)/Canon/(appareil2)/792%7C0/(brand2)/Nikon/(appareil3)/767%7C0/(brand3)/Nikon

As we can see from the Measurements:

1) ISO sesitivity - no meaningful differences except the total range (5D3 beats D800 and D4 beats 5D3)

2) SNR 18% - no meaningful difference except high ISO (5D3 beats D800 and D4 beats 5D3)

3) Dynamic range - some meaningfull differences where D4 wins it all, D800 beats 5D3 below ISO 2400, 5D3 beats D800 above ISO 2400

4) Tonal range - some slightly meaningfull differences where D4 wins it all, D800 seems to beat 5D3 below ISO 2400, 5D3 seems to beat D800 above ISO 2400 and of course 5D3 extends the total range over D800

5) Color sensitivity - some meaningfull differences where D4 loses to D800 below ISO 360 and 5D3 beats D800 above ISO 16000

Bottom line is three fantastic cameras, just take your pick and be happy!

5D3, 70-200 f/2.8L IS II, 100L, 24-105L, Sigma 50/1.4 DG, Rokinon 14mm Canon TC 1.4x III

AJ

  • Canon 7D MK II
  • *****
  • Posts: 490
Re: DxOMark scores for 5DMkIII out - total score 81, 5DMkII had 79
« Reply #83 on: April 19, 2012, 12:57:29 PM »
well, here, how I look at it.

The camera: it is what it is.

The dxo score: hopefully it'll put some pressure on for Canon to lower the price, sooner rather than later.

Fishnose

  • Rebel T6i
  • ****
  • Posts: 104
Re: DxOMark scores for 5DMkIII out - total score 81, 5DMkII had 79
« Reply #84 on: April 19, 2012, 01:18:54 PM »
I love Canon, but they got caught with their pants down, marketing-wise.

Yes, success leads to complacency.
Just like the big trouble Nokia is in now. They had 37% of the whole cellphone market a couple of years ago, now they're #2 after Samsung and losing speed fast.

Situation:
2 Canon guys, from R&D and Sales, are sitting out in the sun, taking a (really long) break.
Something yellow and black comes by at tremendous speed, so fast they can't really tell what it was.
"A bumblebee, maybe?" says one.
"Yeah, I guess."
And they go on relaxing.

What was it?
Nikon, going really fast.

neuroanatomist

  • CR GEEK
  • ************
  • Posts: 20033
Re: DxOMark scores for 5DMkIII out - total score 81, 5DMkII had 79
« Reply #85 on: April 19, 2012, 01:22:42 PM »
1) ISO sesitivity - no meaningful differences except the total range (5D3 beats D800 and D4 beats 5D3)

Just to call this one out, DxOMark's ISO Sensitivity measurement isn't measuring what most people think it's measuring.  It has no direct bearing on ISO noise performance (but an indirect one, see below).  The name is perhaps misleading, maybe better to call it 'ISO Accuracy' or 'ISO Fidelity'.  What this test measures is the real ISO value (benchmarked against the actual International Organization for Standardization's criteria) vs. the ISO setting on the camera, or to put it another way, it measures how much the camera lies to you when you pick a given ISO setting. 

In the plot you can see that for ISO 50 and ISO 100, the dots for the 5DIII, D800, and D4 are all stacked on top of each other, and they're all at ISO 75 for both settings.  What that means is all three cameras are lying to you in exactly the same way - whether you set ISO 50 or ISO 100 for your shot, the exposure is actually at around ISO 75 and then pushed or pulled by the camera as needed, although the ISO value you selected is what's actually recorded in the metadata.  This lying is not new or unique - both Canon and Nikon do it routinely for fast lenses, where the incident angle of the light exceeds the refracting capability of the microlenses and exposure is 'secretly' boosted to compensate (i.e. about 1/2 stop of the light coming in at f/1.2-1.4 is not detected by a digital sensor, so the camera boosts the ISO half a stop - meaning half a stop more noise - without telling you).

Perhaps of a bit more significance is the way this plays out - if you compare just the D800 to the 5DIII and look at the ISO Sensitivity plot, you can see that the D800 is a little further off the nominal value at all the settings, with more separation at the higher ISOs.  In other words, the D800 lies to you a little more than the 5DIII (but the 5DIII is still lying).  For example, when you set both cameras to ISO 6400, the D800 is actually shooting at ISO 4211 (it's lying by 2/3-stop), whereas the 5DIII is actually shooting at ISO 5179 (it's lying by only 1/3-stop).  Translation - artificial advantage for the D800 because it's shooting at a lower actual ISO than the 5DIII for a given setting.

Bottom line is three fantastic cameras, just take your pick and be happy!

This is the take home message!
EOS 1D X, EOS M2, lots of lenses
______________________________
Flickr | TDP Profile/Gear List

skitron

  • Canon 7D MK II
  • *****
  • Posts: 517
Re: DxOMark scores for 5DMkIII out - total score 81, 5DMkII had 79
« Reply #86 on: April 19, 2012, 01:25:53 PM »
1) ISO sesitivity - no meaningful differences except the total range (5D3 beats D800 and D4 beats 5D3)

Just to call this one out, DxOMark's ISO Sensitivity measurement isn't measuring what most people think it's measuring.  It has no direct bearing on ISO noise performance (but an indirect one, see below).  The name is perhaps misleading, maybe better to call it 'ISO Accuracy' or 'ISO Fidelity'.  What this test measures is the real ISO value (benchmarked against the actual International Organization for Standardization's criteria) vs. the ISO setting on the camera, or to put it another way, it measures how much the camera lies to you when you pick a given ISO setting. 

In the plot you can see that for ISO 50 and ISO 100, the dots for the 5DIII, D800, and D4 are all stacked on top of each other, and they're all at ISO 75 for both settings.  What that means is all three cameras are lying to you in exactly the same way - whether you set ISO 50 or ISO 100 for your shot, the exposure is actually at around ISO 75 and then pushed or pulled by the camera as needed, although the ISO value you selected is what's actually recorded in the metadata.  This lying is not new or unique - both Canon and Nikon do it routinely for fast lenses, where the incident angle of the light exceeds the refracting capability of the microlenses and exposure is 'secretly' boosted to compensate (i.e. about 1/2 stop of the light coming in at f/1/2-1.4 is not detected by a digital sensor, so the camera boosts the ISO half a stop - meaning half a stop more noise - without telling you).

Perhaps of a bit more significance is the way this plays out - if you compare just the D800 to the 5DIII and look at the ISO Sensitivity plot, you can see that the D800 is a little further off the nominal value at all the settings, with more separation at the higher ISOs.  In other words, the D800 lies to you a little more than the 5DIII (but the 5DIII is still lying).  For example, when you set both cameras to ISO 6400, the D800 is actually shooting at ISO 4211 (it's lying by 2/3-stop), whereas the 5DIII is actually shooting at ISO 5179 (it's lying by only 1/3-stop).  Translation - artificial advantage for the D800 because it's shooting at a lower actual ISO than the 5DIII for a given setting.

Bottom line is three fantastic cameras, just take your pick and be happy!

This is the take home message!


Oh, I fully agree. And as you pointed out, it further drives home the idea that you have to pay attention to what was actually posted.
5D3, 70-200 f/2.8L IS II, 100L, 24-105L, Sigma 50/1.4 DG, Rokinon 14mm Canon TC 1.4x III

bp

  • EOS M3
  • ****
  • Posts: 169
Re: DxOMark scores for 5DMkIII out - total score 81, 5DMkII had 79
« Reply #87 on: April 19, 2012, 01:32:02 PM »
Nikon, going really fast.

"Nikon, going real fast"?

The DxO just tests the sensor.  Sony makes the D800 sensor, not Nikon.  Shouldn't all this fanboi love be giving Sony the credit?  All Nikon did was stick it in a camera "system", that by all other accounts is far inferior.

So... Way to go Sony!  Too bad Nikon f#@%ed it up by putting it in that body, but very nice job on the sensor.
5D2 / A7Rii | 24L II | 85L II | 135L | 24-105L | 24-70L II | 70-200 IS II | Shorty Forty | Bower 14 | Rokkor 58mm f/1.2 | Sony Macro 90 G | Zeiss Batis 85 | Zony 55 f/1.8

canon rumors FORUM

Re: DxOMark scores for 5DMkIII out - total score 81, 5DMkII had 79
« Reply #87 on: April 19, 2012, 01:32:02 PM »

Fishnose

  • Rebel T6i
  • ****
  • Posts: 104
Re: DxOMark scores for 5DMkIII out - total score 81, 5DMkII had 79
« Reply #88 on: April 19, 2012, 01:33:55 PM »
1) ISO sesitivity - no meaningful differences except the total range (5D3 beats D800 and D4 beats 5D3)

Just to call this one out, DxOMark's ISO Sensitivity measurement isn't measuring what most people think it's measuring.  It has no direct bearing on ISO noise performance (but an indirect one, see below).  The name is perhaps misleading, maybe better to call it 'ISO Accuracy' or 'ISO Fidelity'.  What this test measures is the real ISO value (benchmarked against the actual International Organization for Standardization's criteria) vs. the ISO setting on the camera, or to put it another way, it measures how much the camera lies to you when you pick a given ISO setting. 

In the plot you can see that for ISO 50 and ISO 100, the dots for the 5DIII, D800, and D4 are all stacked on top of each other, and they're all at ISO 75 for both settings.  What that means is all three cameras are lying to you in exactly the same way - whether you set ISO 50 or ISO 100 for your shot, the exposure is actually at around ISO 75 and then pushed or pulled by the camera as needed, although the ISO value you selected is what's actually recorded in the metadata.  This lying is not new or unique - both Canon and Nikon do it routinely for fast lenses, where the incident angle of the light exceeds the refracting capability of the microlenses and exposure is 'secretly' boosted to compensate (i.e. about 1/2 stop of the light coming in at f/1.2-1.4 is not detected by a digital sensor, so the camera boosts the ISO half a stop - meaning half a stop more noise - without telling you).

Perhaps of a bit more significance is the way this plays out - if you compare just the D800 to the 5DIII and look at the ISO Sensitivity plot, you can see that the D800 is a little further off the nominal value at all the settings, with more separation at the higher ISOs.  In other words, the D800 lies to you a little more than the 5DIII (but the 5DIII is still lying).  For example, when you set both cameras to ISO 6400, the D800 is actually shooting at ISO 4211 (it's lying by 2/3-stop), whereas the 5DIII is actually shooting at ISO 5179 (it's lying by only 1/3-stop).  Translation - artificial advantage for the D800 because it's shooting at a lower actual ISO than the 5DIII for a given setting.

No Sir, this is incorrect. Your technical explanations so far have been stellar, but here you're wrong. The 'ISO accuracy' you menton is a different measurement altogether.

Quote from DxOMark concerning what their 'Sports (Low-Light ISO)' measures:

"Sports & action photography: Low-Light ISO

....Photojournalists and action photographers often struggle with low available light and high motion. Achieving usable image quality is often difficult when pushing ISO.

When shooting a moving scene such as a sports event, action photographers’ primary objective is to freeze the motion, giving priority to short exposure time. To compensate for the lack of exposure, they have to increase the ISO setting, which means the SNR will decrease. How far can they go while keeping decent quality? Our low-light ISO metric will tell them.

The SNR indicates how much noise is present in an image compared to the actual information (signal). The higher the SNR value, the better the image looks, because details aren't drowned by noise. SNR strength is given in dB, which is a logarithmic scale: an increase of 6 dB corresponds to doubling the SNR, which equates to half the noise for the same signal.

An SNR value of 30dB means excellent image quality. Thus low-light ISO is the highest ISO setting for a camera that allows it to achieve an SNR of 30dB while keeping a good dynamic range of 9 EVs and a color depth of 18bits."

psolberg

  • Canon 7D MK II
  • *****
  • Posts: 550
Re: DxOMark scores for 5DMkIII out - total score 81, 5DMkII had 79
« Reply #89 on: April 19, 2012, 01:42:03 PM »
Nikon, going really fast.

"Nikon, going real fast"?

The DxO just tests the sensor.  Sony makes the D800 sensor, not Nikon.  Shouldn't all this fanboi love be giving Sony the credit?  All Nikon did was stick it in a camera "system", that by all other accounts is far inferior.

So... Way to go Sony!  Too bad Nikon f#@%ed it up by putting it in that body, but very nice job on the sensor.

apple designs the iphone, foxconn makes it. Nikon designed the sensor, sony makes it.

canon rumors FORUM

Re: DxOMark scores for 5DMkIII out - total score 81, 5DMkII had 79
« Reply #89 on: April 19, 2012, 01:42:03 PM »