DxOMark scores for 5DMkIII out - total score 81, 5DMkII had 79

[Maui5150]

Re: DxO results out for 5D3

DxO is about as relevant as Ken Rockwell and their numbers similarly make as much sense.

After all, according to their numbers the PhaseOne and D4 are pieces of crap compared to d800

 

[Arun]

Re: DxO results out for 5D3

Suppose the 5D3 beat the D800 hollow. Or whatever. Even so, there would still be a lot of people taking photographs with "lesser" cameras. Like the 7D. Like the photos on display here:
http://www.fredmiranda.com/forum/topic/1097338

Nikon/Sony's sensor definitely open up new possibilities, and that is a good thing. But it is still only a small part of the art of the photograph.

 

[Michael7]

Re: DxO results out for 5D3

Thanks for posting the results. There's no way in heck I'm buying the 5D III now. The D800 is the camera to have it seems.

 

[thepancakeman]

Re: DX0 Mark Canon 5D MkIII Review

Someone on here said it best, "When all this hype has died down in two years 90% of pros will be shooting with the 5D3 over the D800, just like the 5D2 over and D700 and all Canon over Nikons"

Looking at the Amazon best seller list I predict you'll be very likely wrong:

http://www.amazon.com/Best-Sellers-Electronics-Digital-SLR-Cameras/zgbs/electronics/3017941/ref=pd_ts_zgc_e_3017941_morl?pf_rd_p=1270018122&pf_rd_s=right-5&pf_rd_t=101&pf_rd_i=507846&pf_rd_m=ATVPDKIKX0DER&pf_rd_r=0JABKZ0CEFB8XTBV2D7S

Nikon occupied the first three spots for the past month and a half (since the D800 come out), and now with the D3200 also the 4th is gone.

Because most of the pros get their stuff off of Amazon, right?

 

[seekn]

Re: DxO results out for 5D3

Patting yourself on the back is not attractive.
Anyway, the one thing that I just dont understand on these forums is why soooo much emphasis is placed on one aspect of a camera? I totally agree that the 5dmarkiii DR is subpar to the 800 but what about the other million aspects of the camera? Do they not count also?
Cameras are works of art if I may say so - but there has to be balance in the composition. If one side is too heavy, something else has to give. Canon and Nikon are just different companies with different emphasis. This OP is just concerned about the sensor and making it seem like Canon is hopeless when they have in fact turned out a wonderful camera.
I really question the DXO results - how in the world can there be such a huge separation between high iso action shots? From all the other tests I have seen they are nearly equal or canon has the edge. These tests seem way to overblown to me - but just a feeling.
Either way - so many people on these forums seem to be ignorant to the aspect that not everything is all about the sensor. I have seen more than one test show that you get more "keepers" with the mark iii during action shots than the 800 - primarily due to the AF and FPS. The canon is said to have a faster, more responsive AF which surely results in capturing that fleeting moment.
Do I wish that canon has better low ISO DR? Heck yeah! Do I truly sweat about it when I am looking over my shots of the day? Not really to be honest. I am more concerned about my lighting, my composition and the overall look and feel of the shot. 99.9% of my viewers or clients will never complain to me that they cannot see details in the darkest parts of my pictures. Honestly most of them are too busy looking to see what they look like in the shot than to examine how much DR there is!
Either way, kudos to the OP for being right but that is in no way going to make me switch platforms or make me regret buying a mark iii. Its a great camera plain and simple. People jumping ship are going to be sorely disappointed when their bottom line doesnt increase just from switching. Anyway, Im finished with work and its time for me to head home and get my hands on my mark iii. Seriously I have shot every day with it and I already have had some Nikonites drooling at my photos! lol. Grass is always greener I guess.
And to the above poster - if you are going to buy the 800 just based on that I think you are making a mistake. You should test each out, see which feels better to you in hand and also see what images are more pleasing to you. Also, what camera suits your needs the best.

 

[sarangiman]

Re: DxO results out for 5D3

I'm sorry, but downscaling is not a mystical tool that will magically fabricate additional DR. If you blow the highlights in-camera, no amount of downscaling or upscaling or scaling in any other form is going to RECOVER pixels that reached maximum saturation and then-some. I think such facts need to be disputed, not really for the good of Canon shooters (where generally all sticking with Canon gear)...but more for the good of potential camera buyers.

jrista: But downsampling also reduces noise, so since DR is calculated by taking into account where SNR drops to 1 (that's how DXO does it), the DR would increase upon downsampling b/c SNR would increase for any given dark patch. Hence a darker patch may yield a higher SNR (e.g. of 1) after downsampling than prior to downsampling... effectively increasing your calculated DR. You're right that highlights can't be retrieved by dowsampling; i.e. think the increased DR upon normalization is due to lower noise in shadows upon downsampling.

Alternatively, if you're calculating DR using LTRLI's method of DR = log(base 2)[max signal/read noise], read noise goes down upon downsampling, so your calculated DR goes up, yes?

That being said, I just don't understand how DXO rates the D800 has having higher DR than the D4 (both normalized & un-normalized), or the D800 having better ISO performance than the 5DIII.

I mean, just looking at these curves, there's something really fishy about ISO 100 on the D800:

Furthermore, LTRLI: you keep claiming how it's cool that your calculation of DR based off of black/white RAW files from the camera match DXO's lab tests (where they actually shoot patches of varying brightness).

What I find interesting is that no one seems to question that methodology of calculating DR from black/white RAW files! I know jrista had issues with the source files LTRLI was using, but what of the methodology?

Why does it match DXO so well? I find that interesting/weird/fascinating... wish some of you would comment on it, especially LTRLI!

Also, FWIW, I've been shooting quite a bit w/ the 5DIII lately, & preliminarily, IQ-wise, I have to say:
[list type=decimal]
[*]Every time I try to lift shadows of ISO 100 images in LR (no I'm not talking about +100) it's a banding mess. So there's really not that much leeway in terms of screwing up your exposure or single-shot HDR
[*]I tried averaging dark frames to do a dark frame subtraction on ISO 100 images. No luck yet b/c the FPN does change somewhat from day to day. Could try averaging frames from multiple days to only try & subtract out the FPN that does persist strongly
[*]At least there's not much banding in high ISO shots. Sometimes strange horizontal banding shows up at really high ISOs. Again, no surprise here, b/c when you add enough noise to FPN, you effectively mask the FPN with all the other random noise
[/list]

That being said, I really don't think we should be bashing people or going off-topic by talking about the other aspects of the camera that make it good. That's not the point of LTRLI's thread, & I think all the other talk dilutes those of us with a semi-academic interest, as well well as those of us interested in optimizing our setup. Yes, I agree, the AF has been amazing so far, & the 600EX-RT flashes have been reliable & great; in fact, those are the reasons I'm sticking with Canon for now b/c I now do more people photography than landscape photography. But heck, I really would've liked it if the Canon served me as well as Nikon during those times I do revert to landscape shooting. Then again, I used to love shooting slides so one may wonder why I'm complaining In all seriousness, though, if you're into optimizing, you're into finding the tool that best serves whatever needs you have... and sometimes we have multiple needs/interests.

Therefore these tests are relevant to me, as are discussions of the validity of these tests. Let's try & not bash anyone.

 

[V8Beast]

Re: DX0 Mark Canon 5D MkIII Review

Because most of the pros get their stuff off of Amazon, right?

Damn straight. Amazon is the vendor of choice for all serious photographers, followed closely by overstock.com and Wal-Mart.

 

[jrista]

Re: DxO results out for 5D3

I'm sorry, but downscaling is not a mystical tool that will magically fabricate additional DR. If you blow the highlights in-camera, no amount of downscaling or upscaling or scaling in any other form is going to RECOVER pixels that reached maximum saturation and then-some. I think such facts need to be disputed, not really for the good of Canon shooters (where generally all sticking with Canon gear)...but more for the good of potential camera buyers.

jrista: But downsampling also reduces noise, so since DR is calculated by taking into account where SNR drops to 1 (that's how DXO does it), the DR would increase upon downsampling b/c SNR would increase for any given dark patch. Hence a darker patch may yield a higher SNR (e.g. of 1) after downsampling than prior to downsampling... effectively increasing your calculated DR. You're right that highlights can't be retrieved by dowsampling; i.e. think the increased DR upon normalization is due to lower noise in shadows upon downsampling.

Alternatively, if you're calculating DR using LTRLI's method of DR = log(base 2)[max signal/read noise], read noise goes down upon downsampling, so your calculated DR goes up, yes?

I guess I would call that making existing DR (in these cases, in the shadows) more usable, where as before downsampling it may be less usable. It can't increase DR beyond the capabilities of the hardware, however, regardless of how you work the math. There are physical attributes of the electronics in a sensor and in an ADC that limit the actual range. If the D800 has a 14bit sensor, then it has a theoretical maximum limit of 2^14 discrete levels of luminance, and a theoretical maximum limit of 14 stops of dynamic range. When you factor in overhead and losses (some of which might be mitigated with better hardware...i.e. CDS), you arrive at the physical limitations of the hardware, which both from a saturation point (maximum discrete luminance levels) and DR standpoint, will be less than the theoretical maximums. Downsampling might be able to recover some of what was lost, possibly generating a normalized result that has slightly better characteristics than the hardware, but you still can't gain more than the theoretical maximum as limited by the hardware. You can't really gain anything at all...your just better realizing the potential of what you already have, up to physical or at best mathematical limits.

I'm not sure if a simpler strait forward equation like what LTRLI uses is indicative of real-world downsampling, either. All forms of downsampling are variations of averaging and filtering algorithms, which sample multiple source pixels to produce some kind of cross-processed result pixel. If you have a range of input shadow samples from 0-5, the only way you could gain DR would be if the output was always 0...however an averaging algorithm will rarely always result in the best output for DR from a given set of input. If you have 0, 2, 1, 2, 5, 3, 4, 1, 0, 1, 5 as input samples, the unweighted average would be 2.1818 or ~= 2.2, which would result in an output pixel of 2. You actually lose DR from a real-world standpoint, be it a simple nearest neighbor sampling or something more advanced like bicubic or possibly a fractal algorithm. The original DR for shadows was 5 levels, where as the final DR for shadows is 1. The same thing would occur for the entire DR of an input image, from shadows to highlights, so for any real-world downsampling algorithm that most photographers would have access to, I don't see how you can really gain anything from the act of downsampling. I think at best you can maintain, and at worst lose a minor amount, of overall dynamic range, even if you improve the characteristics of noise. You might also improve the perception of the image overall...it'll appear less noisy and probably sharper...but mathematically, in actuality, those improvements are perceived out of a loss. Perception is by definition a subjective measure, though, hence the issues I have with using DxO Print DR as a primary measure of IQ.

That being said, I just don't understand how DXO rates the D800 has having higher DR than the D4 (both normalized & un-normalized), or the D800 having better ISO performance than the 5DIII.

I mean, just looking at these curves, there's something really fishy about ISO 100 on the D800:

Furthermore, LTRLI: you keep claiming how it's cool that your calculation of DR based off of black/white RAW files from the camera match DXO's lab tests (where they actually shoot patches of varying brightness).

What I find interesting is that no one seems to question that methodology of calculating DR from black/white RAW files! I know jrista had issues with the source files LTRLI was using, but what of the methodology?

I have a LOT of issues with DxO's methodology. I've commented about them on other threads in more detail...but that is essentially what I'm arguing. Statistically, DxO results seem to be accurate relative to one another, but the whole model seems to be flawed in some way that skews results and doesn't accurately reflect real-world results, and with each new Nikon camera, the gap between what is well-known in the real-world as EXCELLENT seems to...theoretically...fall farther and farther behind Nikon cameras specifically (not even Sony cameras, which use the same Sony sensors as Nikon cameras do, seem incapable of keeping pace with Nikon cameras for some reason...which is VERY ironic.) The fact that digital MF sensors, which produce some of the most astounding output I've ever seen (particularly of portraiture, but also for landscapes), their sensor ratings on DxO are rather "sucky", particularly in relation to Nikon cameras. I'm called a fanboy for calling such discrepancies out...but its a relevant issue, in my opinion, just as relevant as the nature of Canon's DxO ratings or quirky results like 14.4 stops of DR for the D800. Just too many odd discrepancies relative to the actual real-world performance of these sensors that begs questioning the results, and possibly the entire approach.

Also, FWIW, I've been shooting quite a bit w/ the 5DIII lately, & preliminarily, IQ-wise, I have to say:
[list type=decimal]
[*]Every time I try to lift shadows of ISO 100 images in LR (no I'm not talking about +100) it's a banding mess. So there's really not that much leeway in terms of screwing up your exposure or single-shot HDR
[*]I tried averaging dark frames to do a dark frame subtraction on ISO 100 images. No luck yet b/c the FPN does change somewhat from day to day. Could try averaging frames from multiple days to only try & subtract out the FPN that does persist strongly
[*]At least there's not much banding in high ISO shots. Sometimes strange horizontal banding shows up at really high ISOs. Again, no surprise here, b/c when you add enough noise to FPN, you effectively mask the FPN with all the other random noise
[/list]

[/quote]

I think you may be encountering banding, which is different from FPN (which by definition is fixed and cannot vary.) Banding, according to a couple who have posted on this forum and over on DPR, is often the result of electronic interference...possibly even from the camera's own battery interface, possibly from other sources. I've seen a lot of sample shots from new 5D III owners...some exhibit definite banding at low ISO in the deep shadows, where as some don't seem to exhibit banding at all. It may just be related to manufacturing tolerance, and you might be able to resolve the issue by swapping your body for another one (although you might have to do that a few times to get one that doesn't exhibit banding...assuming its a sample issue at all...you may just have some electronic or radio frequencies in the areas you've been shooting that are strong enough to interfere with any inadequately shielded electronics in the camera.)

 

[sarangiman]

Re: DxO results out for 5D3

Thanks jrista for your reply. A couple of things:

I guess I would call that making existing DR (in these cases, in the shadows) more usable, where as before downsampling it may be less usable. It can't increase DR beyond the capabilities of the hardware, however, regardless of how you work the math.

The DR the camera can capture/record is (partially) hard limited by the ADC, yes?

In other words, an 8-bit ADC would necessarily limit your DR to 8EV? Want to make sure I'm understanding this correctly. For example, if full-well capacity is 65,536e-, & you use a gain of 256e-/ADU, but now your blacks are crushed even if the read noise of your sensor is, say, 30e-. Even though your sensor may be differentiating blacks, a lot of blacks will be binned to an ADU of 1, correct?

If so, then, yes I totally see your point.

I'm not sure if a simpler strait forward equation like what LTRLI uses is indicative of real-world downsampling, either.

I wasn't talking about a downsampling equation... I don't remember LTRLI ever providing one. I was talking about his calculation of DR:

• DR = log(base 2)[max pixel value in white file/read noise in ADU]

I understand that's a general way of determining PDR, but my issue, which I raised a number of times before, is that to relate this calculation to the DR the camera is capable of recording requires a number of assumptions that I do not know are valid! I even provided a thought experiment that, if implemented into the signal processing pipeline, would make Nikon appear to have greater dynamic range even though it may not (basically: if you bin certain low signals to 0, rather than allow both positive & negative variation around a black point of, like 2048, you can reduce the stdev of pixels in a resultant black frame).

I don't see how you can really gain anything from the act of downsampling.

All I'm saying is: take a black frame from your camera, load it up in IRIS, measure the stdev of pixels. Now downsample that file 4x or something; now measure the stdev of pixels. I just did for a black frame from my 5DIII, & here are my results:

[list type=decimal]
[*]100%: stdev = 5.9ADU | DR=11.4EV
[*]25% (bilinear downsampling): stdev = 1.61ADU | DR=13.2EV
[/list]

Wow! ~2 stops more DR! Kinda makes you question that DR formula... what say you?

(not even Sony cameras, which use the same Sony sensors as Nikon cameras do, seem incapable of keeping pace with Nikon cameras for some reason...which is VERY ironic.)

That is interesting & to me points to the signal processing pipeline in Nikon cameras doing something different that yields better results (real world? DXO? both?... I can't comment yet b/c I haven't yet done my 5DIII vs. D800 transmission wedge test, which I'll do as soon as my friend gets his D800).

 

[briansquibb]

Re: DxO results out for 5D3

Since all this hooha started about DxO figures for DR I have been deliberately been trying to get the maximum from my 1DS3 which according to DxO should be about 12 stops

Now I dont have all the fancy testing gear and nor do I care enough either so I just monitored the RAW histogram in DPP which seemed to me to show the DR as the scale. I dont really care whether it is accurate or not - it is the relative DR that I was interested in.

These are the findings from my pictures which are pretty varied in terms of content and lighting

1. Most of my pictures sit between 8 and 10 at iso100 and these are in quite contrasty situations

2. Of all the thousands of pictures I only got a handfull over 10 and a couple approaching 12 - and these were theatre shots which one could visually see went from pure black to pure white

3. On the >10 I could see no noise, and they gave beautiful prints - as did all the others taken at iso100

4. Use of flash on the whole gave better looking pictures, but reduced the DR

I have come to the conclusion that I dont need DR >12 - In fact 10 would probably be sufficient for 99.9% of my taken shots. I am not sure that losing a little DR would make any difference to either the screen shots or prints

In view of the wide variety of shots I take at iso 100/200 I would suggest that there would be very few circumstances where the shot where it would be possible to tell the difference in DR between 12 stop DR or 14 stop DR bodies because there are so few images in real life that require it

I believe the real issue lies in what happens to DR at higher iso. Below is one of the pictures that pushed the DR to the max at iso1600, registering a DR of 10 - there is a little dr clipping in the blacks - but I would suggest it doesn't matter as the blacks are not a integral element of the image.

So at iso1600 I would say the DR is OK - but what of the trend towards the curent low light favourites - iso12800 and higher. On my 1D4 the DR is way down, the images are clean, reasonably sharp but visually lacking in DR. This is the area where I feel more DR would be useful (on the 1D4 at 6400/12800). I VERY rarely shoot at these isos - only as a last resort. At weddings (with the 5DII) I avoided anything over 1600 by using flash - yes you can use flash in the reception - with all the iphones going off then a man sized flash doesn't attract attention. Or set up a picture booth - you will be surprised at the length of the queue that you will get, especially after the beer starts flowing

My bottom line is that:

- arguing over the DxO testing methods is largely irrelevant
- monitor your own DR requirements using DPP histogram
- take test shots and see how they match up to your eye, pleasing images are far more important than any lab testing
- avoid high iso wherever possible ('high' being body dependant)
- personally I dont believe that fraction of a point with high DR are relevant to field taken images for the VAST MAJORITY OF PEOPLE for ALL their images

What I am saying here in this thread context is that both the 5DIII and the D800 seem to have more than adequate DR to cope with all of the pictures for the majority. Certainly DR is not going to be a deal breaker for either.

Several posters here have mentioned that good technique is key to good images, and for DR best practice is to keep iso in that 'golden' part of the curve (for the 1D4 and 1Ds3 that seems to be iso1600 or less) where of course coincides with the golden part of the curve for least obtrusive noise. The responsibility for highest IQ is down to the shooter to use best practice and with todays technology you will be rewarded with great images.

 

[briansquibb]

Re: DX0 Mark Canon 5D MkIII Review

According to DxO, the 5DII and 1DsIII are turds, but they dominated the studio scene for quite some time. I can see the D800 cutting into this advantage, but I still don't think the gap is big enough for many working pros to consider switching systems.

The 1DS3 has moved into the sports area now and is hosing the D800 with its massive 25% advantage in fps

 

[q3chap]

Till now, i am very happy with my MII.

Clearly you haven't been understanding this thread--all those pictures you've taken that you thought you loved now look much worse and you hate them.

hehe .. yeah .. but if the pictures looks bad, than it's because of me and not of the camera ...

But I miss the new AF in my MII, but it is not 1500 euro worth. Maybe 400 euro, but not 1500, but I think, in a year or two, the price of MIII will be 2500 euro instead of 3300 euro.

 

[briansquibb]

- The popup flash on the D800 is NOT meant for use as a flash per se. It is meant to be used as master for controlling other flashes in the CLS (Creative Lighting System), which beats the hell out of Pocket Wizard and other primitive 'on or off' systems.

I guess this is a comment made by someone who has no significant expence with PW/Odins/Pixel King.

But hey it is a good thow away remark often repeated by like minded people

 

[Matthew Saville]

Is it just me, or didn't this happen once before? When DXO published a test, and then changed it? Or was that a different review site? (I know Luminous Landscape got itself into trouble once or twice, but I could swear that DXO also "changed it's mind" too once...)

Just wondering. I'm a Nikon owner, and I don't believe the findings lol. The 5D 3 should perform much better in low light, from the images I've seen. I dunno, maybe it's chroma noise that is bringing the 5D mk3 RAW score down? I never consider color noise because it's so dang easy to remove, but Canon usually does have more color noise in RAW files even if the "properly NR'd" files end up being better... *shrug*

(I do post-production for a living, so I spend all day staring at D700 12 MP files and 5D 2 sRAW1 files. The 5D mk2 cleans up really well in sRAW1, and beats the D700 "in the real world", ...but the un-NR'd 21 MP files are what kill the DXO score.

=Matt=

 

[Aglet]

Re: DxO results out for 5D3

LTRLI - pretty hard to feel vindicated here, eh?

This thread goes all over the map. Only noteworthy SENSOR difference I'm really seeing is low ISO DR from DxO's results. Mine are showing the noise structures.

Numbers don't matter for those who may not fully appreciate their significance and their limitations. They're a tool, like any other, to help us differentiate the things they describe. They're certainly more valuable if we know the complete methodology behind them, tho.

I finally had a chance to load the screen results for 5d2, 5d3 and d800 as I couldn't earlier.
They're all pretty darn close at everything except low ISO DR, as expected. whether screen or print, the significance is considerable and disappointing that Canon didn't do better. I still find the screen numbers more believable and useful since I'm not interested in the downscaling benefits as their normalizing doesn't match mine. With screen numbers I have a better idea what to expect when comparing different camera hardware, printing drivers and issues are removed.

Given that, I don't think the D800 deserves a total score as high as it received, nor the 5d3 as low as it received. Not knowing how they weight that final total score means, to me, that it is about the only irrelevant part of DxO's presentation.

OTOH, everything else about the 5D3 has been significantly improved, going by all the numbers and features. It also should be capable of slightly more pushing in post because of reduced banding vs the 5d2, so that raises its subjective IQ and DR a little more than the numbers might suggest.

if I were in the market for the 5D3, at least for stills, I've have no hesitation in buying one. I don't give a rodent's sphincter about video tho, so whatever those results are have no sway in my decision.

As it is, there's not much compelling me to upgrade from my 5D2 unless I get rid of my 7D too and trust the 5d3 can cover both of those bases adequately. Losing the extra "reach" of the 7D is a worthy trade for me considering how horridly bandy mine is. It's good for high speed snapshots, not something I can really massage in post like the 5d2 or D800.

Not having seen more images from the 5d3, or having used it myself, I'll stay with the devils I know for the moment. Well, not counting the new Devil800 I ordered to augment my toolkit because I know I could put faith in some of those abstract numbers describing it.

Now for some visual aids.

images below are from my +4 EV push test. scaled 1/8 linear using bicubic-sharper and then equally fudged in GraphicConverter to quickly bring up the levels just to really show the large-scale banding structure differences. cropped to the upper-left 600x400 so as not to suffer any uncontrolled scaling losses on this forum.

Now, could some lumberjocks please tell me again how much you like plaid and that it just doesn't matter? ;D

Some of us really don't like plaid, I don't want my prints to wear it either.
So I won't be using more bandy-sensored Canons, much as i love using them otherwise, when I need to do some single shot, high-DR imaging.
Use the best tool for the job. (or hire one )

 

[LetTheRightLensIn]

Re: DxO results out for 5D3

But you come in and trash anyone to pieces if they make a sensor thread and if they mention that any aspect of a sensor is not the best in the world.

No...I trash the idea that Canon not having the best sensor in the world matters.

I think some people dispute the accuracy of DxO's results when they claim that the D800 is capable of more dynamic range than the hardware is rated for.

that is only because you don't understand normalization

I fully understand normalization. Normalization doesn't create anything that you didn't start out with, though. It REMOVES things...such as noise, but it does not create. One way or another, scaling down or scaling up, you are destroying original, pure information and diluting it. You cannot create dynamic range you did not start out with. The only thing that might potentially increase is perception...but thats a subjective matter that cannot be objectively evaluated in a scientific test like those DxO runs.

That said, DxO's screen DR numbers still show that the D800 wins on the low-ISO DR front. It still gets 13.2 EV of native hardware DR. I have no interest in disputing that. Sony and Nikon made some brilliant moves with better technology, and they deserve to take the win!

OK, well you sure bashed and insulted and trashed me (and indirectly some famous scientists) to pieces in that other thread when I said that would be the case.

I trashed your use of potentially inaccurate source data...at the time that thread occurred, you guys were using images created under inconsistent lighting with inconsistent exposure from source cameras that were either potentially or explicitly declared pre-release...then claiming accurate results. I wasn't disputing the numbers...just the methodology and proclamations of total accuracy. You'll notice that once people started getting their hands on actual release versions of the 5D III and tests were done on dark frames from legit copies, I took the results as far more accurate.

you need to read up more on this....

Its just the normalization thing again, and its the same deal...you can't create what you don't have. If you didn't start out with dynamic range in-camera, you aren't going to correct blown highlights with some math and scaling.

The real problem though, is if DxO is wrong about that, even if their method of deriving that number is consistent...what else could they be wrong about? If DxO can interpret "normalized DR" numbers such that they indicate a camera is supposedly capable of more than the hardware would allow, what else might they be wrong about?

maybe it is not they who are confused

Not saying they are confused...I am not sure their model is as relevant as I used to think it was though. I don't have a problem with their screen DR numbers (at the moment...if they claim the next Nikon 14-bit DSLR is capable of 14.4 stops of screen DR, I'll completely reverse my opinion of DxO in total!) Same issue...you don't create what you don't have with a bit of averaging, and you don't recover what you lost at a hardware level with software. It was gone long, long before it even reached the software. I don't think anyone is confused...I think there is probably some bias in interpretation, and potentially a flaw in the model. That wouldn't mean their numbers are incorrect within the definition of their model...just that they don't demonstrate the real-world capabilities of the actual camera hardware as well as I used to think.

They certainly seemed to miss the ball on the 5D III low ISO performance...between DxO's own SNR and high ISO DR scores for the 5D III, I'd have figured it would win that category.

They penalized the 5D3 for making the color filter array more color blind which helps improve SNR while making different colors harder to tell apart and worsening chroma noise, they may have over done it a bit, but that is where that came from and they have been consistent in their formula for that.

I think that may demonstrate my issue with DxO superbly. When comparing 5D III photos to D800 photos, from a direct-from-camera standpoint, the 5D III seems to have FAR better color. The D800 in many of the video reviews I've seen seemed to exhibit rather flat color, often with a distinct green cast. It wasn't just an LCD screen problem regarding the color cast either...the same green cast showed up during post-processing as well. So even if Canon tweaked the CFA to improve SNR, it certainly doesn't seem to have affected color rendition in the real world enough to penalize them as much as DxO did.

I think it came out a little bit much in this case since the D800 slipped just above the bonus points zone and it perhaps overly penalizes the 5D3, but there is nothing technically wrong or confused about what they did measurement by measurement there although I think you can argue that their weightings are maybe not the most sensible for the overall score there. I do think it leads to some somewhat odd results at times. The overall sensor is even odder and even they say there is no way to make one number tell the story.

I think the idea of "bonus points" in what is supposed to be an objective comparison is an odd thing at best, and an element of subjectivity in general. There shouldn't be "bonus points" for anything in an objective test...ISO is ISO, you don't give an ISO score bonus points because some dot on a plot fell just barely inside of a particular "zone" and call it a win, while another dot on a plot fell just barely outside of a particular "zone", and call it a loss. Its not objective.

The fact that other people put so much weight on them, though, and seem to acknowledge them at face value WITHOUT any critical evaluation, is curious to me, and I simply can't help but harass when I see just as much fanboyism regarding DxO as you claim Canon photographers here exhibit regarding their gear (the latter of which is more understandable, since it IS the physical piece of gear that actually empowers ones photographic creativity...not a set of numbers from DxO.)

A bit rich since you seriously harassed me post after post for a while because DxO was so much better than me and that so much weight had to be given TO DXO.

Same as my point above, I was disputing your use of inaccurate source data (inconsistent photos taken with pre-release cameras) and claiming total accuracy in your own results. The thing that really irked me at the time, though, was how you immediately started the doomsday reports about how Canon had failed all of their customers and failed to live up to untold years of expectation, yadda yadda...based on numbers derived from inconsistent pre-release source data...regardless of whether it was backed up by renown scientists or not. I'd slam you again for the same reasons if you made the same premature, sensationalist claims again.

Regardless of how accurate they may have been in the end, the claim was a bit bogus, and my point about DxO at the time was that they never used a random, inconsistent set of source data to produce their numbers...they used very carefully produced images and a consistent process to produce consistent results (which, despite my current misgivings about DxO's new results for the D800, I still adhere to.) Within their model, their results are consistent...but I have some problems with how they have rated the D800 and given it what I believe are some unrealistic ratings (not everything, but a couple things.) Its hard to trust results when trust has been broken with what seem to be bogus numbers like 14.4 stops DR from a 14-bit camera, or giving some "bonus points" and a win to an ISO score when the competition was only fractionally worse. Sketchy, and worth disputing and discussing, at the very least.

And, in the end...it doesn't matter. Its clear that, despite its technological inferiority, however major or minor any given individual may wish to construe those inferiorities, the 5D III takes awesome photographs. Its clear that both the D800 and 5D III take awesome photographs. So, in the end...real-world results trump technological inferiority.

Canon has better color in video because the D800 only reads ever third line for video. That has nothing to do with stills performance. The 5D3 has better video SNR than the D800, perhaps by 1.5-2 stops or so, yeah, and avoids the color moire the D800 has.

I was using actually black frames from actual 5D3s, as were the others, and you trashed us to pieces. And went on and on about how who do we think we are, DxO? and just wait, DxO, will show our ridiculously carried out results to be totally wrong, etc. etc. at first.

Normalization. I do not think it means what you think it means. ;D
Someone else can give it a go and try to explain.

The bonus points are objective in the sense that they are given in a strictly defined way and have always been doled out the same way. They are subjective in that the whole weighting of their overall ratings is just what they decided to go with and I do think their overall low light ratings system is a bit weird myself and the overall sensor rating is just plain weird since how do you compare ten different factors with one number? But their plot data looks good. I've always largely ignored their over sensor ratings and their overall sub-ratings and just focused on there plot data. But yeah the overal low light score does seem a bit unfair to the 5D3. The one good thing is it might keep Canon from cheating the color array filters any further though.

The color array filters don't determine colors directly, that's really on how well the associated color profile for the camera is in DPP or ACR or C1 or whatnot, they more determine whether the sensor can tell different shades apart of not and how much you need to boost channels to reach a normal profile under different lighting conditions (under tungsten they all have about the same metamerism, under daylight they differ a lot more). A more color-blind CFA means you need more boost some channels more and that leads to more chroma noise (although it helps keep down luma noise, which is why they do it).

 

[briansquibb]

Is it just me, or didn't this happen once before? When DXO published a test, and then changed it? Or was that a different review site? (I know Luminous Landscape got itself into trouble once or twice, but I could swear that DXO also "changed it's mind" too once...)

Just wondering. I'm a Nikon owner, and I don't believe the findings lol. The 5D 3 should perform much better in low light, from the images I've seen. I dunno, maybe it's chroma noise that is bringing the 5D mk3 RAW score down? I never consider color noise because it's so dang easy to remove, but Canon usually does have more color noise in RAW files even if the "properly NR'd" files end up being better... *shrug*

(I do post-production for a living, so I spend all day staring at D700 12 MP files and 5D 2 sRAW1 files. The 5D mk2 cleans up really well in sRAW1, and beats the D700 "in the real world", ...but the un-NR'd 21 MP files are what kill the DXO score.

=Matt=

As there was issues with the software I wonder what DxO used?

 

[LetTheRightLensIn]

Re: DxO results out for 5D3

Patting yourself on the back is not attractive.

Yeah I know. It's stooping to the bashers level in a way. And no matter what happened before, it never goes down well.

But it's also necessary since they had many people believe all sorts of tests methods were invalid that were not invalid and they shouldn't be misled into not believing in the methods.

I knew I should've been more light-hearted about it, that always goes over better and it's a better way to act. Normally I do.

Anyway, the one thing that I just dont understand on these forums is why soooo much emphasis is placed on one aspect of a camera? I totally agree that the 5dmarkiii DR is subpar to the 800 but what about the other million aspects of the camera? Do they not count also?

They do count and I've said so many times. The AF and fps and all that, awesome improvements. I've even said this in this thread earlier. I said it was great that is tied for best SNR ever.

But if you praise something then the usual suspects don't respond and the thread quickly falls off the charts but if you dare mention anything not the best about Canon they attack you and then the threads go on forever as you explain and defend everything so it seems like you only talk about one thing when you actually talk about all sorts of things.

Also, it takes a LOT to get Canons attention and some inside hints are that they haven't really cared to pay attention to low ISO DR much internally.

Either way, kudos to the OP for being right but that is in no way going to make me switch platforms or make me regret buying a mark iii. Its a great camera plain and simple. People jumping ship are going to be sorely disappointed when their bottom line doesnt increase just from switching. Anyway, Im finished with work and its time for me to head home and get my hands on my mark iii. Seriously I have shot every day with it and I already have had some Nikonites drooling at my photos! lol. Grass is always greener I guess.
And to the above poster - if you are going to buy the 800 just based on that I think you are making a mistake. You should test each out, see which feels better to you in hand and also see what images are more pleasing to you. Also, what camera suits your needs the best.

 

[LetTheRightLensIn]

Re: DxO results out for 5D3

I mean, just looking at these curves, there's something really fishy about ISO 100 on the D800:

It's actually all of the other curves that are fishy ;D. Ideally it WOULD be a straight line plot. It's read noise issues at certain stages of the read out that cause them to curve flat towards ISO100 for the non-exmor cameras.

Furthermore, LTRLI: you keep claiming how it's cool that your calculation of DR based off of black/white RAW files from the camera match DXO's lab tests (where they actually shoot patches of varying brightness).

What I find interesting is that no one seems to question that methodology of calculating DR from black/white RAW files! I know jrista had issues with the source files LTRLI was using, but what of the methodology?

Why does it match DXO so well? I find that interesting/weird/fascinating... wish some of you would comment on it, especially LTRLI!

Probably because I think they are actually doing the same thing. I think they only use all the other parts of their fancy wedge for SNR and the integrated tonal response stuff and all that other stuff. For the basic dynamic range I think they just take their top blown out wedge (same as the white frame we talk about) and for then take a bottom wedge that contains the blackpoint (same as out lens cap on black frame) and then do the same thing we did, so I suspect it's the same, although they are not 100% clear. (also it's not really MY method since plenty of others use it and used it well, well, well before I ever did)

Therefore these tests are relevant to me, as are discussions of the validity of these tests. Let's try & not bash anyone.

 

[Eric_S]

The DXOMark scores are normalized to allow an appropriate comparison of sensors. In essence, their concept is to treat the camera as a black box and compare performance when printing to a fixed image size. That seems reasonable. The 5DMkIII has better pixel level noise performance than the D800, but when printed to the same size, the overall noise of the Nikon sensor is lower. The 5DMkIII produces cleaner jpegs than the D800 and it has a higher frame rate. The 5DMkIII is a phenomenal camera. Unfortunately, the pre-release hype seems to have deviated a bit farther from reality than usual. Anybody who is thinking of changing brands, either way, because of the differences between the D800 and the 5DMkIII must not have much of an investment in glass. When it comes to capturing a great image, does the difference matter? I say Hooray for Nikon. I applaud their technical achievement. Assume the that D800 sensor really is better...So what. I prefer the system I have and I am certainly not going to replace all of my glass. Given that the 5DMkIII is an excellent camera, what does the DXOMark score really mean to me? ... It means that Canon is going to pull out all of the stops and give us something really jaw dropping for the next iteration. When you are number one its easy to become a bit complacent. As such, the D800 may well be the best news long time Canon shooters have had for a long time. If specs matter all that much in your final image, by all means go out and whine, gnash your teeth and cry foul. If on the other hand, you want to take great pictures, then the MkIII is a significant improvement over the MkII and its a great camera. I don't actually put camera specs on my images...do you?