DxOMark vs. Reality

[paul13walnut5]

@bdunbar79

Well it's like me and basketball.  In HS I couldn't miss in practice from the 3 point arc.  So you could say I was a great 3-point shooter.  However, I never hit a 3 in a game because I never got open and wasn't even quick enough to get open.  So it was actually meaningless.

We here on the forum know that this is just a sensor score.  Howver, consumers don't know that and if they just read the scores they automagically assume it is a better camera.  That's the problem.  It isn't.  Sensor is just ONE measured aspect of a camera and as a scientist, when I read their crap, it comes off to me as rather unscientific.  Whether it is or isn't, it just comes off that way to me.

You can defend it all you want, but it looks like crap to me and I don't use their scores at all.  Each to their own.

Do consumers care?  When I bought my first camera I bought a Canon.  Why?  Because thats what my Dad had used for years, and what my sister used.  I think the majority of consumers are the same. 

Now I'm a bit more serious about it I read various reviews, get hands on, take in a memory card, try a couple of lenses.  I'm not very scientific, and don't photograph charts for my test, or in fact for my work or pleasure.
Maybe I could be more demanding, but I just want a camera that fits my lenses, does decent video and takes decent pics.

My cameras do that for me, I might help out a little with grading or RAW processing now and then, but I'm not losing any sleep what so ever over what Nikon or DxO are doing.   I can understand folk at the very top of their game chasing that extra 1% to the nth degree, or whatever.  I'm not at the top of my game, and I doubt that most of us here are. 

One of my favourite photographers, Maritn Parr, did a lot of his best work on a Nikon F90.  He was asked once what kind of lens he used and he had to look at the front of it to see. I mean that as no sleight.  I think we just lose track of what is important sometimes.  I think this obsession with DxO is psuedo-autistic, and as we are all churning out great images on Canon gear, can deduce it doesn't really mean that much.

I've always thought of DxO as being like reviewing a car that doesn't have a gearbox.  I've never seen a difference on a print or on screen that convinces me Canon is as far behind Nikon as the fanboys interpretation of the stats would have you believe.

If I was in the market for a 36MP camera, then I might buy a D800, I'd certainly consider it.  But I'm not.  To be honest if I could get a DSLR that did video with a stills resolution that permitted straight digital sampling to 1080 (so say a 3840 wide sensor) I would be absolutely delighted.  On that basis I will be delighted to see what DxO make of the c100 or c300, as one can only image they'll hate it!

[Aglet]

How come the 5D2 IQ was impeccable before the D800? I find it still impeccable today. :|

it was not when it was released
it is not much better after all the firmware updates
it made a big splash being the first CHEAP full-frame and those who wanted or needed that were so enthused with this new toy they paid little regard to its IQ shortcomings because it offered IQ benefits and features previously unavailable.

Endlessly trumpeting on that basis suckered me in to buying one; my most regrettable Canon purchase.
not for lack of due diligence, but for lack of honest and clear information about the 5D2's weaknesses, which became clear enough after I used it for a while. .. and after others began posted about its less than ideal low ISO FPN issues.

If you still find it meets your needs you either lucked out with a good one or you don't mind crushing your blacks a few more levels than some of us.

edit adding DxOmark comment below:

and THIS is what annoys me about DxOmarks results. not just that they assign a vague overall score to a camera, based solely on measured sensor merits, but that they do not adequately disclose the testing criteria and data in a way that would allow the technically astute reader the opportunity to evaluate the data on their own.  And, for the most part that I've found, neither do other sensor tech sites.

When FPN affects a sensor to the degree some of the Canon's (and some other mfr's products) were affected, it would have been very valuable to have a good idea whether the noise was random and acceptable or whether it was patterned and possibly objectionable to prospective purchasers.

DxOmark's data is useful but incomplete and that makes it much less useful than it would have been in my particular instance.

[RLPhoto]

How come the 5D2 IQ was impeccable before the D800? I find it still impeccable today. :|

it was not when it was released
it is not much better after all the firmware updates
it made a big splash being the first CHEAP full-frame and those who wanted or needed that were so enthused with this new toy they paid little regard to its IQ shortcomings because it offered IQ benefits and features previously unavailable.

Endlessly trumpeting on that basis suckered me in to buying one; my most regrettable Canon purchase.
not for lack of due diligence, but for lack of honest and clear information about its weaknesses, which became clear enough after I used it for a while. .. and after others began posted about its less than ideal low ISO FPN issues.

If you still find it meets your needs you either lucked out with a good one or you don't mind crushing your blacks a few more levels than some of us.

*Facepalm

If your ultra-mega analytical about bringing up black's 4-stops, which by the way is why they are called black's not midtones, then you have some other serious issues to contend with. (IE: Timing for Ideal Light)

I'd have 7D landscapes that are heavily processed and pushed to +3 in the corners than were accepted for Istock. Thats some very strict standards.

[neuroanatomist]

Last time you mentioned this "low ISO FPN issue" I did what you suggested and did a Google search for images, the only examples that search threw up were poorly exposed skies. Now I print big with a 1Ds MkIII, essentially the same sensor as the 5D MkII, and I have never had your "issue" I really want to see examples of what you are shooting, and how you are shooting it, that best illustrates your experiences.

I, too, would be interested in seeing examples of the low ISO FPN issue in real-world images, ones that don't require 100% crops and have descriptions like, "Canon 5D Mark II Fixed Pattern Noise, ISO 1600, +4 exposure," and, "...with Fill Light set to +100 to exaggerate the vertical banding in shadows."

[neuroanatomist]

I think the WHAT is perfectly clear.  What's not clear is the real world benefit in common shooting situations for one photographer vs. another.   The sharper lens is an apt example - while having a sharper lens certainly offers a theoretical benefit, for many real world situations, that benefit is irrelevant because factors ranging from chosen subject (intentional softening for portraiture, a need to shoot at a very narrow aperture) to workflow/output (downsizing for web/small prints) render the 'benefit' of the increased lens sharpness moot.  For example, looking at the MTF chart of the MP-E 65mm, it's not a very sharp lens...but for the vast majority of the shots I take with that lens, a sharper version of it would not matter.

Thus, the request for real-world examples of shots that were rendered 'unusable' by the low ISO FPN of the 5DII.

[Orangutan]

If you do not understand WHAT you can do in the REAL world with larger DR , why still discussing it?

Let me illustrate it this way: people have used slide film for decades, and it is supposed to have limited DR.  Apparently, slide film had other qualities that, for many purposes, were more important than its limited DR, and  many people chose that compromise.  Sure, photographers would have loved slide film that had 14 stops of DR, while retaining the other qualities of slide film, but that was not available. 

The same is true for modern DSLR's: we would all love DSLR's that have 16 stops of clean DR, while retaining all the other qualities of the overall device (AF, speed, high ISO, color qualities, etc).  No one (so far as I can tell) is saying that they don't want more DR.  They're saying that, as with slide film, they have chosen their compromise until a better compromise is available.  At least, so it appears to me.

[neuroanatomist]

Neuro.  You did not answer my question, do you understand - and read again what I asked
Underexposing, head room , bring in high lights more than 4 stops above middle grey
 If you understand this we take  lesson 2

Certainly, I understand.  But no one is asking for theoretical 'lessons'.  Rather, both me and privatebydesign are asking for actual examples of images 'ruined' by the FPN of the 5DII, and some idea of the frequency at which the low ISO FPN renders real-world images unusable. 

[RLPhoto]

Neuro.  You did not answer my question, do you understand - and read again what I asked
Underexposing, head room , bring in high lights more than 4 stops above middle grey
 If you understand this we take  lesson 2

Certainly, I understand.  But no one is asking for theoretical 'lessons'.  Rather, both me and privatebydesign are asking for actual examples of images 'ruined' by the FPN of the 5DII, and some idea of the frequency at which the low ISO FPN renders real-world images unusable.

Yes, FPN ruins the perfect test charts of obsessive camera disorder sufferers.

[neuroanatomist]

I have shown pictures after pictures taken with my d800 and 5dmk2, I have shown how it is possible to expose and recover high lights (sun sets etc etc) and the adjust the picture by moving middle grey back and lift shadows and darker tones. I think it is time that you and others stop denying the benefit of a wide DR and where unfortunately not the Canon is the leader.

shall we take lesson 2   ?

Let's skip right to Lesson 5:
Q: What is the sum of 2 + 2?
A: You must stop denying the power of arithmetic, and realize the benefit of the D800 for simple math.

Point being, you are answering questions that aren't being asked, attributing statements/beliefs to people which are not their own, and for some reason, the D800 is always part of your answer, even if it's a complete non sequitur.

To be very clear, I am neither denying the benefit of wide DR nor suggesting that Canon is a leader in that specific aspect of sensor performance.  As I've repeatedly stated, if a wide DR were my primary need, I'd be shooting with a D800.

To reiterate, what's being asked is neither 'does the D800 have better DR than the 5DII?' nor 'can an image with wider DR be pushed harder in post than an image with less DR?'.  I'm not talking about contrived test scenarios.  I could just as easily contrive a scenario where the D800 would fail miserably (take a shot with the D800 with a Nikon lens at 5:1 magnification, for example). 

I've taken several thousand shots with a 5DII in the ISO 100-400 range, and guess how many I've rejected due to 'horrible low ISO FPN'?  Zero.

Yes, FPN ruins the perfect test charts of obsessive camera disorder sufferers.

Indeed.  It also ruins shots of QPcards on barbecues, but for some strange reason, that subject constitutes 0% of my shots.   

[neuroanatomist]

Stop then  denying the benefits of great DR,  and this BS about  "real life pictures" etc etc, it is up to peoples skills to use great DR or not and as resolution it is not bad to have.

Thanks for yet another non-answer to a not-asked question.  If your lessons are intended to be instructive as to how to obfuscate and avoid answering pointed questions, and attribute motives to people that they do not espouse, perhaps those would be better delivered on a forum devoted to politics. 

[Aglet]

I've requested permission of a family whose casual photos I shot last summer where FPN is noticeable on dark clothing.
I expect they'll approve my request.
It's about the last time I used the camera.  Customer was totally pleased with the results, I was not.

I've basically sidelined the 5d2 as an inadequate FF Rebel with serious FPN problems, not worthy of MY use so it's seen very little action with me.
I got the thing to do landscape and other outdoor and high DR work.  It's not been able to adequately fulfill that role, the way I want to use it, so to me it was an expensive PoS with limited ability.  It DID deliver some really great images in many situations where lighting was not challenging, but was not able to do so when and where I needed it to. At the time I was also using a 40D which showed less FPN when using similar PP.  I still like and use the 40D.
So a new 5D2 having worse raw file IQ than my older, cheaper camera was a big disappointment, to put it mildly.

Maybe 5d2 was one of the better tools available at the time, doesn't mean it was good enough.  A better tool was announced less than 3 weeks prior to the 5d2, the d90.  However I was an ignorant Canon fanboy at that time and paid little attention to the competition with all the full-frame excitement and high expectations.
Now a cheap consumer camera from a competitor nearly 2 years ago, at 1/5th the cost, grossly outperforms 5d2 in this part of the operating regime it only reinforces my opinion that the 5d2 was, and still is, a grossly overhyped bit of gear.  Despite that, the vast majority of users are happy to continue using it.  That's fine for them.
Those who are content with the shortcomings of a tool they're using are not going to convince those of us who need a better tool that it's OK!  Thanks for trying.  People on here are truly helpful and a great resource to plenty of newbies.

Hopefully I'll get permission to post a few samples where maybe you'll actually SEE the problem.  Possibly helping spare some avid users some unpleasant surprises that they haven't yet discovered.

You guys may need to be reminded. I'm not a Canon hater, just a disappointed long time Canon user who opened his eyes and found better tools for certain shots from the competition.
I still like using my bucket of Rebel bodies, 40D, and 60D and am looking fwd to an improved 7D Mk 2 or 70D.

@privatebydesign - i often print big too. expose for the highlights, bring up the rest with various curve tools until the image DR is compressed to a useful range for printing.  This is where the 5d2 (&7D) can fail if there was a lot of DR in the original scene.  The sensor may be similar or even the same between this and your 1 series body but there's a lot more to the rest of the electronic guts that can make the difference in FPN.  Quality is rarely skimped on the 1 series.

I got better results with my 40D, 60D, and various Rebels than what the 5d2 delivered.  And now my modern Nikon gear blows them all away whenever I need to do this this kind of processing.  Right tool for the job, but not necessarily the same tool for all jobs.

@Mikael Risedal - Thanks for knowing and understanding. I don't know if I've seen your real world comparison shots other than the one of a shed with interior shadows.  Apparently that does not qualify as a real world image for some people.  What if you want to spy on the contents of your neighbor's shed? 

I suspect Neuro will tell me that I wasted 1 or 2 possible stops worth of DR on some of the shots by some of the method I used.  I.E.  I could have ETTR more if I were using Neutral or Faithful instead of Standard settings so my RGB playback histogram would more closely reflect raw headroom.  Yup... Might have.  I don't have to worry about this on my other bodies tho.

[neuroanatomist]

I suspect Neuro will tell me that I wasted 1 or 2 possible stops worth of DR on some of the shots by some of the method I used.

Highly unlikely.  Thanks for working toward sharing real-world examples, though (I don't consider a scenario intentionally chosen and designed to make a point to be 'real-world'). 

I've taken several thousand shots with a 5DII in the ISO 100-400 range, and guess how many I've rejected due to 'horrible low ISO FPN'?  Zero.

I should add, by way of maintaining objectivity and putting things in perspective: I've taken several thousand shots with a 5DII in the ISO 100-400 range, and guess how many I've rejected due to misfocusing by the less than stellar AF system of the 5DII?  Hundreds at least, likely thousands. 

[Zlatko]

I've taken several thousand shots with a 5DII in the ISO 100-400 range, and guess how many I've rejected due to 'horrible low ISO FPN'?  Zero.

Same here.  Three years of heavy shooting with this camera.  Zero photos rejected due to this "horrible" problem.  The 5DII has been good.

I should add, by way of maintaining objectivity and putting things in perspective: I've taken several thousand shots with a 5DII in the ISO 100-400 range, and guess how many I've rejected due to misfocusing by the less than stellar AF system of the 5DII?  Hundreds at least, likely thousands.

Again, same here.  AF could have been better.