December 06, 2016, 05:20:36 AM

Author Topic: DxOMark scores for 5DMkIII out - total score 81, 5DMkII had 79  (Read 83159 times)

LetTheRightLensIn

  • Canon EF 400mm f/2.8L IS II
  • *********
  • Posts: 4753
Re: DxO results out for 5D3
« Reply #255 on: April 20, 2012, 03:04:44 PM »
And to not bubble burst too much so once again it is awesome they gave it 1 series AF and increased fps and all, it's quite a beast and handles way better than the 5D2 from what I read, and well much of that is simply fact, 6fps is faster than 4fps and the faster shutter response etc are fact, AF is a better on paper, didn't try it yet myself, reports sound very, very good so far though. So for non-liveview tripod shooting it should work anywhere from a little bit better to vastly better depending what you are doing with it.

And the SNR is tied for best ever which is all that we could have hoped for there (at expense of a bit of color-blindness which is why DxO scores it lower for high iso, although their weighting probably exaggerates the usable difference for SNR/DR alone, sorting out the color-blindess effects is really tricky, anyway DxO just weighs that in heavily as an incentive so that one makers doesn't just totally thin it out while others don't and make the ones who don't look bad, you can argue what the weighting should be).

The DR was already known to be coming in no better than the 5D2. So nothing new here.

The SNR part of results here are something positive though since there were some questions about exactly where it would end up. The early talk, in this case, was just best guesstimates. Some had said it might only be 1/4 to 1/3 stop better, i.e. nothing doing, so the results here showing it to be around 1/2 to 2/3+ better depending upon the ISO are pleasant.

It was nice to see it ended up at the high end of the early predictions and matching the D800/D4/D3s.

(and real world, since the D3s has much lower resolution I think you can say it would often look worse at high ISO than the others, if you ignore DR where it has a full stop better than D800 and 5D3, so it's darker shadow areas will look better at high iso and you won't feel quite so clipped in with teh dynmaic range, but the lower res means clumpier larger sized noise and less detail which tends to look uglier and doesn't take advanced NR as well so overall for scenes not having tons of deep shadows it probably wouldn't look quite as good as the others at high iso actually, although for certain scenes it might look better than the D800 and 5D3)

The people suggesting the lesser SNR improvement for the 5D3 over the 5D2 were failing to take into account that the 5D3 rates ISOs more conservatively than the 5D2 does I think.

Not that 1/2 to 3/4 stop is insanely OMG but it's realistically all we could have hoped for (nobody has done better yet in any typical consumer camera). And the high iso banding is also lesser in the 5D3 than the 5D2. You also get 1/2-2/3 stop more DR at the much higher ISOs than the 5D2.

Anyway so long as you are not a 100% tripod/liveview shooter the 5D3 certainly improves tons of things compared to the 5D2.








But it would be nice if Canon finally paid some attention to low ISO DR and image quality. They haven't done a single improvement there since the 1Ds3 (which is arguably still the best Canon, by a tiny trace, at ISO100).
(I saw a tiny bit of talk from some with bit of Canon connection say that they got the impression that Canon was apparently taken by surprise that Nikon/Sony decided to push their FF exmor stuff out the door so quickly and they didn't think they had to push their FF sensors yet and though they could focus on improving body handling mostly for now. As some have been saying, Canon management was acting way too complacent and king of the hill resting on their laurels. I also saw some disturbing talk that they seem to think it was only the MP that are exciting about the D800 and they still don't get the dynamic range stuff, not sure if that is true, but at least a few seem to have that unfortunate, IMO, point of view and that they, or at least a few (few? some? many? who knows) might consider a MP count higher than the D800 a huge success even if it had the same old 2007 dynamic range again. Anyway that is somewhat hearsay so who really knows exactly what they are thinking.)


canon rumors FORUM

Re: DxO results out for 5D3
« Reply #255 on: April 20, 2012, 03:04:44 PM »

straub

  • Guest
Re: DxO results out for 5D3
« Reply #256 on: April 20, 2012, 03:17:28 PM »
Normalization. I do not think it means what you think it means.  ;D

Since you seem to be the self-proclaimed go-to guy regarding normalization, you can probably easily answer this. Suppose you have a sensor that captures 1Gpix of data with DR of 2 stops. You expose the highlights to LV+15 and thus record luminosities between LV+13 and LV+15. How do you suppose "normalizing" this data will somehow generate luminosity values below LV+13 and/or above LV+15?

LetTheRightLensIn

  • Canon EF 400mm f/2.8L IS II
  • *********
  • Posts: 4753
Re: DxO results out for 5D3
« Reply #257 on: April 20, 2012, 03:27:58 PM »
Normalization. I do not think it means what you think it means.  ;D

Since you seem to be the self-proclaimed go-to guy regarding normalization, you can probably easily answer this. Suppose you have a sensor that captures 1Gpix of data with DR of 2 stops. You expose the highlights to LV+15 and thus record luminosities between LV+13 and LV+15. How do you suppose "normalizing" this data will somehow generate luminosity values below LV+13 and/or above LV+15?

Why would the sensor suddenly not record any data lower than 2 stops below LV+!5? It would. It would be a mess of noise at 1GP scale though but it would still grab the data.

straub

  • Guest
Re: DxO results out for 5D3
« Reply #258 on: April 20, 2012, 03:34:27 PM »
Normalization. I do not think it means what you think it means.  ;D

Since you seem to be the self-proclaimed go-to guy regarding normalization, you can probably easily answer this. Suppose you have a sensor that captures 1Gpix of data with DR of 2 stops. You expose the highlights to LV+15 and thus record luminosities between LV+13 and LV+15. How do you suppose "normalizing" this data will somehow generate luminosity values below LV+13 and/or above LV+15?

Why would the sensor suddenly not record any data lower than 2 stops below LV+!5? It would. It would be a mess of noise at 1GP scale though but it would still grab the data.

Ah, I should have also mentioned that the camera would have 2-bit ADC. Which then means that in this case anything below LV+13 is recorded and interpreted as LV+13. It is a silly hypothetical camera, but the principle is the same for any N-bit ADC--you simply don't have any data beyond the range of the ADC. And no amount of normalization is going to change that.

LetTheRightLensIn

  • Canon EF 400mm f/2.8L IS II
  • *********
  • Posts: 4753
Re: DxO results out for 5D3
« Reply #259 on: April 20, 2012, 03:47:09 PM »
Normalization. I do not think it means what you think it means.  ;D

Since you seem to be the self-proclaimed go-to guy regarding normalization, you can probably easily answer this. Suppose you have a sensor that captures 1Gpix of data with DR of 2 stops. You expose the highlights to LV+15 and thus record luminosities between LV+13 and LV+15. How do you suppose "normalizing" this data will somehow generate luminosity values below LV+13 and/or above LV+15?

Why would the sensor suddenly not record any data lower than 2 stops below LV+!5? It would. It would be a mess of noise at 1GP scale though but it would still grab the data.

Ah, I should have also mentioned that the camera would have 2-bit ADC. Which then means that in this case anything below LV+13 is recorded and interpreted as LV+13. It is a silly hypothetical camera, but the principle is the same for any N-bit ADC--you simply don't have any data beyond the range of the ADC. And no amount of normalization is going to change that.

But the way you are reading it out doesn't make sense since it wouldn't store LV 13-15 from scene.



« Last Edit: April 20, 2012, 03:55:02 PM by LetTheRightLensIn »

neuroanatomist

  • CR GEEK
  • ************
  • Posts: 19998
Re: DxO results out for 5D3
« Reply #260 on: April 20, 2012, 03:50:55 PM »
Ah, I should have also mentioned that the camera would have 2-bit ADC. Which then means that in this case anything below LV+13 is recorded and interpreted as LV+13. It is a silly hypothetical camera, but the principle is the same for any N-bit ADC--you simply don't have any data beyond the range of the ADC. And no amount of normalization is going to change that.

Clearly, you have a dizzying intellect, but you fail to understand normalization.  Let's define it:

Normalization (ˌnôrmələˈzā sh ən): n. A magical process by which certain data are removed from images and other, nonexistent data are added to images, the net result of which is that those images can be used to support any conclusion in which I firmly believe.

See...it all makes sense, you just had to see it in context.
EOS 1D X, EOS M2, lots of lenses
______________________________
Flickr | TDP Profile/Gear List

straub

  • Guest
Re: DxO results out for 5D3
« Reply #261 on: April 20, 2012, 03:53:14 PM »
Ah, I should have also mentioned that the camera would have 2-bit ADC. Which then means that in this case anything below LV+13 is recorded and interpreted as LV+13. It is a silly hypothetical camera, but the principle is the same for any N-bit ADC--you simply don't have any data beyond the range of the ADC. And no amount of normalization is going to change that.

Clearly, you have a dizzying intellect, but you fail to understand normalization.  Let's define it:

Normalization (ˌnôrmələˈzā sh ən): n. A magical process by which certain data are removed from images and other, nonexistent data are added to images, the net result of which is that those images can be used to support any conclusion in which I firmly believe.

See...it all makes sense, you just had to see it in context.

Ah magick, now I understand!

canon rumors FORUM

Re: DxO results out for 5D3
« Reply #261 on: April 20, 2012, 03:53:14 PM »

jrista

  • Canon EF 600mm f/4L IS II
  • **********
  • Posts: 5334
  • EOL
    • Nature Photography
Re: DxO results out for 5D3
« Reply #262 on: April 20, 2012, 03:53:45 PM »
Right, sorry. I know LTRLI's formula is not a downsampling formula, what I was trying to say is I don't think it takes into account the realities of downsampling properly. To put it another way, I think it produces a result without context, which is why it seems to produce much higher numbers for DR than would seem realistic when computing DR for a downsampled image. In its own right, taken at face value (without any context) its certainly not an invalid formula...but do the results produced by it have real-world applicability? Do they represent something real, or is the formula too simplistic? I used to believe that DxO's Print DR numbers, which are derived the same way, were simply demonstrating that once you eliminate noise, you are then realizing the sensors full potential. Kind of tough to keep believing that, though, when the D800's Print DR numbers are BETTER than the sensors full potential...its not quite like a star trek warp drive...you don't get 120% of maximum out of a 14-bit sensor. ;)

Imagine you have 32MP sensor and you then averaged every 16 pixels into 1 and got 2MP output. And you then measure the Std Dev of the black frame. It's measure lower from the 2MP version than from the 32MP version but the white frame max saturation will measure the same in both cases. Plug into the DR formula and it goes up. It's not magic because the 2MP file also shows a heck of a lot less detail than the original 32MP file did so you traded that detail away for the higher score on the other aspect in a sense, what you are really trying to do it just compare the two files at the same scale. Even if the simplistic normalization didn't quite match in practice and over-stated something a little, the thing is in practice used use advanced NR to equalize them which would be doing better anyway, so there would be no problem there at all.

I understand the nature of scaling, that wasn't in question. Perhaps we just need to qualify what we mean by DR when we discuss these things. When it comes to "hardware DR"...I consider that a fixed constant, and an intrinsic attribute of the camera. If a camera has 14-bit hardware and X amount of overhead, it is going to be capable of a fixed, unchanging amount of "hardware DR"...or lets call it "true DR". If you saturate pixels beyond the True DR, say 15000 ADU, no amount of software-based tuning will be able to recover what you lost.

When we normalize two source images to a common image size, were affecting what I'd like to term "perceptual DR", or perhaps "virtual DR". You are not actually changing the characteristics of the hardware...there is still a hard limit on what a 14-bit camera can do. Virtual DR can only help you fully realize what that hardware is capable of, up to, but not exceeding the limits of, the hardware itself. I don't have a problem with the idea that with normalization of LOTS of pixels into FEW pixels, you might be able to fully realize 13.2 stops of DR, and possibly get a little bit more than that because your mitigating noise. I wouldn't be surprised if Virtual DR for a 13.2 stop sensor ended up being 13.8 stops, and if you had some REALLY GOOD algorithms to handle your normalization for you, I'd be ok, albeit surprised, if your normalized results achieved exactly 14.0 stops...but ok, fine...your making more effective use of data in the "noise floor" of your dynamic range. To achieve 14.4 tops, your somehow creating information...additional levels of luminance (over 5000 of them) to achieve a higher dynamic range than your sensor is capable of. Your adding information above and beyond the maximum number of ADU the ADC is capable of producing for a single pixel at maximum saturation.

Saying the D800 is capable of 14.4 stops, even though that is based on "normalized" data, is misleading. The camera itself is NOT capable of 14.4 stops...its capable of 13.2. Sure, it still wins the DR contest even with 13.2, but were were talking about over a full stop here...thats not within the margin of error. There is something wrong with the methodology here...something extremely UNrealistic about the way DR is being calculated from normalized data. Its not even remotely realistic when your claiming the D800 is capable of more than DOUBLE the dynamic range it ACTUALLY IS capable of at a hardware level. This also begs the question why a mere 36.3mp sensor from Nikon gains SO MUCH MORE than say a 60mp sensor from Hasselblad when downscaling. According to the approach you and DxO use, the more megapixels you start out with, the better the ultimate results when you downscale to an 8mp image. There is obviously a serious disconnect when a FF DSLR sensor with half the resolution as a MF sensor achieves several stops better results, when if the key factor that determines normalized results is how much information you started with. The hassy should win hands down every time, even against a D800, if that was the case...and in visual comparisons even a 40mp hasselblad offers far more highlight recovery room and far better color rendition than a D800. There is a disconnect somewhere in the approach DxO uses and real-world results...and its not an insignificant disconnect.... I'm not sure if I'm portraying my point well enough here... I think sarangiman gets it, but I'm not sure if anyone else does.

As for SNR stuff they want to compare at the same noise scale otherwise you might be comparing a 32MP file that has lots of high frequency noise to a 2MP file that all the higher powers of noise automatically clipped away so it's not fair to compare the two images as if they were at the same noise scale. So they are really just normalizing the power scale of the noise between the two to not unfairly make the higher MP cameras look worse.  You are reducing noise and signal so it's not like the image really improves any, it's actually a worse output really, yeah but you are just trying to do relative noise comparisons here and to compare it to the other file fairly that already had all the high frequency noise clipped away otherwise you are comparing the total noise over a larger frequency range than for the other one where the upper end of the range was hard clipped off which isn't fair. To do it well (not talking in terms of processing images for use but just for this relative comparison) you want to first gaussian blur away the high freq noise and then downsample to the scale you blurred it away to. or maybe more simply upscale the lower MP cam to the match the higher. Or you could just look at both at 100% on a monitor but stand back farther from the screen for the higher MP camera whatever distance is needed to make it's detail the same as what the lower one can capture. i think i garbled that a bit

I get your meaning there. I think, however DxO is doing it, that their "normalization" overcompensates for higher resolution and undercompensates for lower resolution, thereby unfairly making the lower MP cameras look worse than they really are. You effectively apply noise reduction to the higher MP camera, and none to the lower MP camera. I completely understand the value of normalized comparisons, but is the process of normalization valid? Scaling a smaller image up is actually not the same as scaling a larger image down. The former would actually leave noise characteristics largely in-tact for both images, without giving either of them a clear advantage. The problem with that is its harder to keep things consistent, as with each new higher resolution sensor, you need to reprocess old data to the new larger size and recompute the results. I get that, however I also believe it would still be more accurate...it would eliminate a lot of bias towards higher megapixel sensors (assuming that is the only bias...as I mentioned before, if the normalization process caters to higher resolution sensors because your factoring multiple source pixels into each output pixel of an 8mp target image, then medium format cameras should be trouncing everything else...as they well should...but there is clearly something else far less obvious going on with the D800's results.)

Do you think the D700 has much better ISO1600 than the 5D2? Because if you don't normalize, and you now seem to say that normalization doesn't make sense and isn't fair, then the D700 will stomp your 5D2, when the reality is it isn't fair to say that it does.

At a hardware level, I would say the D700 and 5D II have roughly the same performance at ISO 1600. I think the D700 wins by about 1/4 of a stop. That doesn't surprise me, and sensitivity and dynamic range are strongly correlated with larger pixel area. Both cameras are full-frame, so the D700 certainly has a lot more area per pixel than the 5D II. I would EXPECT the D700 to offer more DR at any given ISO setting, since DR is closely linked to maximum saturation and standard deviation of read noise. (Barring any other improvements, making a pixel's total area larger will always allow for a larger maximum saturation level, and assuming all other things remain equal, I would expect the noise characteristics of two identical sensors with differing pixel sizes to be the same.)

Now, from a NOISE standpoint, which is obviously something we regularly correct with software, normalizing the two images to the same dimensions makes sense, and will help the 5D II's standing from a post-processing standpoint...we all use software to clean up noise after we've imported our photos from our camera. But any side effect in terms of improved DR or color fidelity or anything else that you might get, at least according to the log(base2)(maxSaturation-stdevReadNoise) formula, is not reflecting a hardware capability in any way. Actually, the improvement in noise achieved by downsampling is also not reflecting a hardware capability in any way. None of the results gleaned from a software-normalized 8mp image will tell you anything about what the hardware itself is physically capable of...all it will tell you is what the software your using to perform the image normalization is capable of. DxO Analyze is apparently capable of doing amazing things with Nikon RAW files...but the fact that DxO Analyze is capable of doing amazing things with Nikon RAW files doesn't tell me anything about the Nikon cameras itself...its just telling me that DxO Analyze is amazing for processing Nikon RAW files.

straub

  • Guest
Re: DxO results out for 5D3
« Reply #263 on: April 20, 2012, 03:58:57 PM »
If you had a 2 bit monitor would you set 0=bright gray and have the backlight shining through a lot even when fed signal 0? You'd still set 0= liquid crystals block backlight as much as they can = black and then dither to build up lots of intermediate tones and if it had some huge resolution like that you'd build lots of tones.

You can certainly dither between LV+13 and LV+15 in the example system. Not above and not below though. As far as the sensor in this case is concerned, LV+13 *is* black. It is zero in the ADC output scale of [0,3]. Anything below that has ceased to exist after the ADC has digitized it.

jrista

  • Canon EF 600mm f/4L IS II
  • **********
  • Posts: 5334
  • EOL
    • Nature Photography
Re: DxO results out for 5D3
« Reply #264 on: April 20, 2012, 04:10:00 PM »
Something that should also be pointed out is that image comparisons on a normal basis only make sense if that is actually how you are going to use the images. If you only print at 8x10 size, then comparing all images at that size makes sense. However one of the primary reasons you buy a higher resolution camera is to help facilitate the need to print at a higher resolution, or crop to a greater degree and print at a lower resolution. In either of those cases, comparing normalized results at 8x10 size is not going to tell you much about how the camera will perform for your needs, which might require you to print at 13x19, 17x22, or even higher. If you regularly scale your images up to immense sizes, say 30x40 or greater, then the IQ of your camera at its native resolution will be paramount, and not properly  reflected by the 8x10 results.

LetTheRightLensIn

  • Canon EF 400mm f/2.8L IS II
  • *********
  • Posts: 4753
Re: DxO results out for 5D3
« Reply #265 on: April 20, 2012, 04:19:06 PM »
Something that should also be pointed out is that image comparisons on a normal basis only make sense if that is actually how you are going to use the images. If you only print at 8x10 size, then comparing all images at that size makes sense. However one of the primary reasons you buy a higher resolution camera is to help facilitate the need to print at a higher resolution, or crop to a greater degree and print at a lower resolution. In either of those cases, comparing normalized results at 8x10 size is not going to tell you much about how the camera will perform for your needs, which might require you to print at 13x19, 17x22, or even higher. If you regularly scale your images up to immense sizes, say 30x40 or greater, then the IQ of your camera at its native resolution will be paramount, and not properly  reflected by the 8x10 results.

Yes and if you want to know how it might look if you maintain max detail from the same viewing distance then you look at 100%, but when comparing one camera against another and asking what has better noise performance you don't do that since it it not fair.

For their SNR tests they are just comparing to a middle gray signal and featureless gray slab that no information other than it's shade and looks the same at any resolution.

For the 14bit limit stuff, yeah if you calculate it using 14bit math and store the normalized result in 14bits you are stuck at 14bits max but who says you should stick to 14bit math, you need to go to a higher bit space to carry it out.

sarangiman

  • Canon 6D
  • *****
  • Posts: 375
Re: DxO results out for 5D3
« Reply #266 on: April 20, 2012, 04:39:04 PM »
I tend to agree w/ the idea that your DR should never exceed the max bit-depth of your ADC. Simply b/c at best your stdev of pixels in a black frame can at best be 1ADU, in which case the DR formula will give you, in EV, the bit-depth of your ADC. But someone somewhere mentioned that you can theoretically get 1.2EV, or something like that, more than the bit-depth of your ADC... somewhere on dpreview forums, can't remember where.

jrista: While I understand your points, I'm not sure I see this 'bias' you mention re: normalization for different brands. LTRLI I think rightly points out that it's just a matter of the magnitude of the effect increasing the greater the normalization needed (i.e. higher MP cameras will see better gains). For example, here are some normalizations (screen DR on left -> print DR on right):

11.37->12.5 (HD39)
11.35->12.7 (HD50)

11.85->12.2 (D700)
11.16->11.9 (5DII)

So you can see that the 39MP Hasselblad got a 0.8EV boost upon normalization, & the 50MP one got a 1.2EV boost, much like the Nikon D800 (13.2->14.4). The 5DII got a bigger boost than the D700, b/c of its higher MP count. So even though this formula may not be perfect, it seems somewhat consistent. But, yes, I still don't understand a DR greater than the bit-depth of the ADC.

You mention the highlight headroom of the Hasselblad MF cameras. While this may be true, if the read noise is high, then the DR is limited by the poor performance on the lower end. Put another way, sure the headroom of the Hasselblad may be higher than that of the D800; however, with the D800, you just underexpose your image (say by 2 stops), and then pull detail out of the shadows b/c the read noise is so low (not the case for the Hasselblad, or Canon sensors, hehe).

So while it's weird to hear that the D800 has higher DR than a MF camera, it's theoretically possible, IF the read noise on the MF camera is significantly poorer than that on the D800. Yes?

What I'm saying is: I'm allowing for the possibility that DXO's scores *are* valid :) I just wish they published their protocols explicitly.

Is it true that the D800 uses Exmor technology but the D4 doesn't? That'd be weird.

jrista

  • Canon EF 600mm f/4L IS II
  • **********
  • Posts: 5334
  • EOL
    • Nature Photography
Re: DxO results out for 5D3
« Reply #267 on: April 20, 2012, 10:05:08 PM »
I tend to agree w/ the idea that your DR should never exceed the max bit-depth of your ADC. Simply b/c at best your stdev of pixels in a black frame can at best be 1ADU, in which case the DR formula will give you, in EV, the bit-depth of your ADC. But someone somewhere mentioned that you can theoretically get 1.2EV, or something like that, more than the bit-depth of your ADC... somewhere on dpreview forums, can't remember where.

jrista: While I understand your points, I'm not sure I see this 'bias' you mention re: normalization for different brands. LTRLI I think rightly points out that it's just a matter of the magnitude of the effect increasing the greater the normalization needed (i.e. higher MP cameras will see better gains). For example, here are some normalizations (screen DR on left -> print DR on right):

11.37->12.5 (HD39)
11.35->12.7 (HD50)

11.85->12.2 (D700)
11.16->11.9 (5DII)

So you can see that the 39MP Hasselblad got a 0.8EV boost upon normalization, & the 50MP one got a 1.2EV boost, much like the Nikon D800 (13.2->14.4). The 5DII got a bigger boost than the D700, b/c of its higher MP count. So even though this formula may not be perfect, it seems somewhat consistent. But, yes, I still don't understand a DR greater than the bit-depth of the ADC.

You mention the highlight headroom of the Hasselblad MF cameras. While this may be true, if the read noise is high, then the DR is limited by the poor performance on the lower end. Put another way, sure the headroom of the Hasselblad may be higher than that of the D800; however, with the D800, you just underexpose your image (say by 2 stops), and then pull detail out of the shadows b/c the read noise is so low (not the case for the Hasselblad, or Canon sensors, hehe).

So while it's weird to hear that the D800 has higher DR than a MF camera, it's theoretically possible, IF the read noise on the MF camera is significantly poorer than that on the D800. Yes?

What I'm saying is: I'm allowing for the possibility that DXO's scores *are* valid :) I just wish they published their protocols explicitly.

Is it true that the D800 uses Exmor technology but the D4 doesn't? That'd be weird.

Thanks for the numbers for each of those cameras. I guess I'm still not entirely convinced though. The HD50 has 50% more pixels (which is a LOT!) than the D800, and it has half the read noise of the 5D II and III. I'd expect it to realize better gains than the D800, despite the fact that its read noise is higher, yet it realizes apparently the same exact gain from downsampling. In all honesty, I don't really see the same bias for most of the rest of Nikon's cameras...its primarily the D800 that seems to be so out of wack. The D4 seems to be pretty spot on with what I expected before the cameras were tested by DxO, the 5D III is a touch worse than I expected and about a stop worse than I hoped for (I was hoping for 12.7 stops at ISO 100, but it seems Canon opted for SNR improvements rather than DR improvements.)

I find the results for the D800 to be very odd, and the Print DR results, which seem to be what DxO camera scores are primarily computed from, seem to be way off. The DR seems literally impossible at 0.4 stops higher than the hardware is capable of, and the low light ISO rating seems to be significantly skewed relative to that of the 5D III, when the actual data for both cameras doesn't seem to actually indicate that much of a difference...and visual comparisons of the 5D III at higher ISO definitely seem to give it the edge (even when normalized.) I'm suspicious that is the result of bonus points (a ridiculous concept, if your trying to produce objective results) offered to the D800 for barely falling within a certain region while the 5D III barely falls without, and perhaps its just their rating system that is skewing the results, not intentional bias.

Either way...the D800 score and results seem to be unduly skewed, making pretty much all the competition (including $40,000 digital medium format cameras) sound "crappy" in comparison, when the reality is so very far from that.

canon rumors FORUM

Re: DxO results out for 5D3
« Reply #267 on: April 20, 2012, 10:05:08 PM »

sarangiman

  • Canon 6D
  • *****
  • Posts: 375
Re: DxO results out for 5D3
« Reply #268 on: April 20, 2012, 10:45:06 PM »
Quote
(I was hoping for 12.7 stops at ISO 100, but it seems Canon opted for SNR improvements rather than DR improvements.)

Oh yeah. That reminds me. How does the 5D Mark III have SNR improvements (due to better QE/microlenses?), but worse DR than the 5D Mark II?

Those kinds of conclusions really make me wonder if they're actually doing DR measurements or just reading the black/white files... the Mark III this time suffering because it's black level has been raised from 1024 to 2048... Which I don't think should impact DR much...

And I agree that ISO sensitivity ratings don't make much sense either between the 5D Mark III, D800, & D4.

jrista

  • Canon EF 600mm f/4L IS II
  • **********
  • Posts: 5334
  • EOL
    • Nature Photography
Re: DxO results out for 5D3
« Reply #269 on: April 20, 2012, 11:29:08 PM »
Quote
(I was hoping for 12.7 stops at ISO 100, but it seems Canon opted for SNR improvements rather than DR improvements.)

Oh yeah. That reminds me. How does the 5D Mark III have SNR improvements (due to better QE/microlenses?), but worse DR than the 5D Mark II?

The QE improvements are part of it, which was facilitated by gapless microlenses. I think the other factor is weaker color filters in the CFA for red and blue pixels. Both pixels now let in more green light than they used to, and in the case of red, it lets in ever so slightly less red light. The filters were effectively made "more translucent" than they were previously. Greater transparency to light means more light overall, so you can achieve a higher SNR. I think the CFA thing is kind of a cheap trick, and I'm not really sure why the opted to go that route.

That said, Canon must REALLY have some major kind of roadblock to improving the noise efficiency of their CMOS sensor design. I checked sensorgen.info a short while ago, and the 5D III has even HIGHER ISO 100 read noise (33.1e-) than the 5D II, which in its own right had ridiculously high ISO 100 read noise (27.8e-). The 5D III has a higher saturation point, which is why at higher ISO's, SNR is better (read noise drops to 3-4e- beyond ISO 400 most of the time for most cameras), but Canon seems to have literally gone backwards in regards to low ISO.

Until I read the sensorgen.info data (which is derived from DxO RAW data), I wasn't too worried about 5D III low ISO...it seemed pretty much status quo (which would be fine...at least status quo is not a step backwards). And even despite the higher read noise, the 5D III images still appear consistently better than the 5D II images at all ISO's (nose appears far more random than it did before)...but its still a rather odd result, for Canon's latest and greatest to have that much read noise at a low ISO. Its a total difference of 5.3e-, or a 19% increase. Thats not just a margin of error thing there, thats a major mistake! In that respect, I'm entirely in line with LTRLI...Canon is seriously missing the ball in regards to low ISO performance. While I applaud their efforts to increase the range of usable native high ISO settings (which certainly caters to the kind of photography I do most of the time), and they have definitely produced a camera capable of some stellar high ISO performance, doing so at a 20% loss of efficiency on the other end is unacceptable. I figured things would stay pretty much the same and hoped they would get maybe 10% better...but now I'm actually concerned. If Canon releases a big-mp camera next year (something I have been looking forward to since I also do landscape photography alongside my wildlife and bird photography), that has the same low ISO read noise issues as the 5D III, then I might seriously have to consider a D800e and a 14-24mm lens for my landscape work.

Going 20% in the wrong direction is unacceptable, totally in line with you there, LTRLI! Canon needs to get their asses in gear and solve their read noise problem...which at this point seems to be a problem growing at an unmanageable rate. I was willing to forgive Canon for some things because they made so many other improvements to the 5D III, but now that the actual results are in, I have no option but to capitulate and agree...the low-ISO trend is very concerning. (And to that end, I intend to open up direct dialog with Canon about the issue, I as just discussing it on this forum is not going to actually have any kind of real impact on the issue at Canon.)

Those kinds of conclusions really make me wonder if they're actually doing DR measurements or just reading the black/white files... the Mark III this time suffering because it's black level has been raised from 1024 to 2048... Which I don't think should impact DR much...

And I agree that ISO sensitivity ratings don't make much sense either between the 5D Mark III, D800, & D4.

Well, they make sense if arbitrary "bonus points" are awarded to the ISO category. ;P I mean, the sensor doesn't ACTUALLY get bonus ISO points in reality, so such bonus points are 100% pure subjectivity in a test that is supposed to be objective. You don't reward hardware for being a "good boy". :P ISO is ISO...it shouldn't be changed in any way as part of computing a "score" or "rating".

canon rumors FORUM

Re: DxO results out for 5D3
« Reply #269 on: April 20, 2012, 11:29:08 PM »