August 31, 2014, 12:55:12 AM

Author Topic: DxOMark vs. Reality  (Read 78114 times)

elflord

  • 5D Mark III
  • ******
  • Posts: 705
    • View Profile
Re: DxOMark vs. Reality
« Reply #210 on: December 21, 2012, 06:06:29 AM »
Sorry, but my reply wasn't directed generally, nor to you specifically. If you read my previous posts, hopefully you'd realize your questions are tangential and I'm saying none of the statements you seem to be expressing on my behalf.

After rereading I think I see what you were getting at -- it was just very puzzling on a first reading (and yes I understand that taken literally  it was inconsistent with what you've posted before which is why I found it puzzling)

canon rumors FORUM

Re: DxOMark vs. Reality
« Reply #210 on: December 21, 2012, 06:06:29 AM »

bdunbar79

  • Canon EF 300mm f/2.8L IS II
  • *******
  • Posts: 2563
    • View Profile
Re: DxOMark vs. Reality
« Reply #211 on: December 21, 2012, 10:36:30 AM »
Well it's like me and basketball.  In HS I couldn't miss in practice from the 3 point arc.  So you could say I was a great 3-point shooter.  However, I never hit a 3 in a game because I never got open and wasn't even quick enough to get open.  So it was actually meaningless.

We here on the forum know that this is just a sensor score.  Howver, consumers don't know that and if they just read the scores they automagically assume it is a better camera.  That's the problem.  It isn't.  Sensor is just ONE measured aspect of a camera and as a scientist, when I read their crap, it comes off to me as rather unscientific.  Whether it is or isn't, it just comes off that way to me.

You can defend it all you want, but it looks like crap to me and I don't use their scores at all.  Each to their own.
2 x 1DX
Big Ten, GLIAC, NCAC

RLPhoto

  • Canon EF 300mm f/2.8L IS II
  • *******
  • Posts: 3367
  • Gear doesn't matter, Just a Matter of Convenience.
    • View Profile
    • My Portfolio
Re: DxOMark vs. Reality
« Reply #212 on: December 21, 2012, 11:41:43 AM »
How come the 5D2 IQ was impeccable before the D800? I find it still impeccable today. :|

thepancakeman

  • 6D
  • *****
  • Posts: 451
  • If at first you don't succeed, don't try skydiving
    • View Profile
Re: DxOMark vs. Reality
« Reply #213 on: December 21, 2012, 11:52:28 AM »
Well it's like me and basketball.  In HS I couldn't miss in practice from the 3 point arc.  So you could say I was a great 3-point shooter.  However, I never hit a 3 in a game because I never got open and wasn't even quick enough to get open.  So it was actually meaningless.

You, too, huh?  Excellent analogy!

bdunbar79

  • Canon EF 300mm f/2.8L IS II
  • *******
  • Posts: 2563
    • View Profile
Re: DxOMark vs. Reality
« Reply #214 on: December 21, 2012, 12:32:02 PM »
Yeah then I had to "find another sport" a year or two later :).
2 x 1DX
Big Ten, GLIAC, NCAC

tnargs

  • Rebel T5i
  • ****
  • Posts: 138
    • View Profile
Re: DxOMark vs. Reality
« Reply #215 on: January 14, 2013, 10:22:55 PM »
260 posts, how about a summary?  :)

Well, it seems that when people legitimately criticize DxO Labs' application of their measurements, apologists rush in to defend their measurements.

That's called an air ball, guys.

And when people point out that these DxO scores (applications) are being misused by review sites etc in a way that gives the wrong impression, apologists rush in to say that's not really DxO Labs' fault.

You must be joking.

Let us say Mercedes Inc engaged you to independently measure the quality of their product vis a vis say, Ford Inc, which they had also measured and knew would favour them. They also told you to summarize your findings and publish them. You go ahead and do it but goodness gracious, what's this? Your website is technically correct and shows Mercedes' quality advantage at the deepest level, but your way of summarizing and organizing and communicating gives the general impression that Ford is the better overall vehicle!

Suddenly, due to your excellent technical reputation, the world takes notice of your findings. Reviewers start to refer to 'The Ford Advantage' on their websites. Fleet buyers start recommending to boardrooms that current Mercedes contracts for trucks and buses should not be renewed, partly due to a new quality value matrix by an industry technical expert that favours other marques. The boardroom executive gossip soon reaches the ears of Mercedes executives.

Suddenly your client is on the phone to you in an absolute Teutonic fury. Yes, they say, your base measurements are correct but what on earth have you done with it? Why the blazes that way? Don't you realize how everyone is interpreting it?

Here is your answer: (hold on, wait for it, it's a beauty): "It's not my fault if they are not well enough informed to  grasp the underlying deep measurements correctly."

pfft

Good luck getting your invoice processed.

tron

  • 1D X
  • *******
  • Posts: 1816
    • View Profile
Re: DxOMark vs. Reality
« Reply #216 on: January 15, 2013, 09:04:47 AM »
How come the 5D2 IQ was impeccable before the D800? I find it still impeccable today. :|
+1000000 Very true. Rumor has it that a camera does not become worse simply because newer models appear  ;D  ;D  ;D

canon rumors FORUM

Re: DxOMark vs. Reality
« Reply #216 on: January 15, 2013, 09:04:47 AM »

neuroanatomist

  • CR GEEK
  • ********
  • Posts: 13988
    • View Profile
Re: DxOMark vs. Reality
« Reply #217 on: January 15, 2013, 09:14:43 AM »
Rumor has it that a camera does not become worse simply because newer models appear

Is that a CR1 or CR2 rumor?   ;)
EOS 1D X, EOS M, and lots of lenses
______________________________
Flickr | TDP Profile/Gear List

tron

  • 1D X
  • *******
  • Posts: 1816
    • View Profile
Re: DxOMark vs. Reality
« Reply #218 on: January 15, 2013, 10:30:33 AM »
Rumor has it that a camera does not become worse simply because newer models appear

Is that a CR1 or CR2 rumor?   ;)
Definitely a CR3 one  ;D

RLPhoto

  • Canon EF 300mm f/2.8L IS II
  • *******
  • Posts: 3367
  • Gear doesn't matter, Just a Matter of Convenience.
    • View Profile
    • My Portfolio
Re: DxOMark vs. Reality
« Reply #219 on: January 15, 2013, 10:52:22 AM »
How come the 5D2 IQ was impeccable before the D800? I find it still impeccable today. :|
+1000000 Very true. Rumor has it that a camera does not become worse simply because newer models appear  ;D  ;D  ;D

ITS OVER NINE THOUSAND!  :o

paul13walnut5

  • Guest
Re: DxOMark vs. Reality
« Reply #220 on: January 15, 2013, 11:22:26 AM »
@bdunbar79
Quote
Well it's like me and basketball.  In HS I couldn't miss in practice from the 3 point arc.  So you could say I was a great 3-point shooter.  However, I never hit a 3 in a game because I never got open and wasn't even quick enough to get open.  So it was actually meaningless.

We here on the forum know that this is just a sensor score.  Howver, consumers don't know that and if they just read the scores they automagically assume it is a better camera.  That's the problem.  It isn't.  Sensor is just ONE measured aspect of a camera and as a scientist, when I read their crap, it comes off to me as rather unscientific.  Whether it is or isn't, it just comes off that way to me.

You can defend it all you want, but it looks like crap to me and I don't use their scores at all.  Each to their own.

Do consumers care?  When I bought my first camera I bought a Canon.  Why?  Because thats what my Dad had used for years, and what my sister used.  I think the majority of consumers are the same. 

Now I'm a bit more serious about it I read various reviews, get hands on, take in a memory card, try a couple of lenses.  I'm not very scientific, and don't photograph charts for my test, or in fact for my work or pleasure.
Maybe I could be more demanding, but I just want a camera that fits my lenses, does decent video and takes decent pics.

My cameras do that for me, I might help out a little with grading or RAW processing now and then, but I'm not losing any sleep what so ever over what Nikon or DxO are doing.   I can understand folk at the very top of their game chasing that extra 1% to the nth degree, or whatever.  I'm not at the top of my game, and I doubt that most of us here are. 

One of my favourite photographers, Maritn Parr, did a lot of his best work on a Nikon F90.  He was asked once what kind of lens he used and he had to look at the front of it to see. I mean that as no sleight.  I think we just lose track of what is important sometimes.  I think this obsession with DxO is psuedo-autistic, and as we are all churning out great images on Canon gear, can deduce it doesn't really mean that much.

I've always thought of DxO as being like reviewing a car that doesn't have a gearbox.  I've never seen a difference on a print or on screen that convinces me Canon is as far behind Nikon as the fanboys interpretation of the stats would have you believe.

If I was in the market for a 36MP camera, then I might buy a D800, I'd certainly consider it.  But I'm not.  To be honest if I could get a DSLR that did video with a stills resolution that permitted straight digital sampling to 1080 (so say a 3840 wide sensor) I would be absolutely delighted.  On that basis I will be delighted to see what DxO make of the c100 or c300, as one can only image they'll hate it!

Aglet

  • 1D Mark IV
  • ******
  • Posts: 954
    • View Profile
Re: DxOMark vs. Reality
« Reply #221 on: January 15, 2013, 11:58:08 AM »
How come the 5D2 IQ was impeccable before the D800? I find it still impeccable today. :|
it was not when it was released
it is not much better after all the firmware updates
it made a big splash being the first CHEAP full-frame and those who wanted or needed that were so enthused with this new toy they paid little regard to its IQ shortcomings because it offered IQ benefits and features previously unavailable.

Endlessly trumpeting on that basis suckered me in to buying one; my most regrettable Canon purchase.
not for lack of due diligence, but for lack of honest and clear information about the 5D2's weaknesses, which became clear enough after I used it for a while. .. and after others began posted about its less than ideal low ISO FPN issues.

If you still find it meets your needs you either lucked out with a good one or you don't mind crushing your blacks a few more levels than some of us.

edit adding DxOmark comment below:

and THIS is what annoys me about DxOmarks results. not just that they assign a vague overall score to a camera, based solely on measured sensor merits, but that they do not adequately disclose the testing criteria and data in a way that would allow the technically astute reader the opportunity to evaluate the data on their own.  And, for the most part that I've found, neither do other sensor tech sites.

When FPN affects a sensor to the degree some of the Canon's (and some other mfr's products) were affected, it would have been very valuable to have a good idea whether the noise was random and acceptable or whether it was patterned and possibly objectionable to prospective purchasers.

DxOmark's data is useful but incomplete and that makes it much less useful than it would have been in my particular instance.
« Last Edit: January 15, 2013, 12:12:24 PM by Aglet »

RLPhoto

  • Canon EF 300mm f/2.8L IS II
  • *******
  • Posts: 3367
  • Gear doesn't matter, Just a Matter of Convenience.
    • View Profile
    • My Portfolio
Re: DxOMark vs. Reality
« Reply #222 on: January 15, 2013, 12:06:08 PM »
How come the 5D2 IQ was impeccable before the D800? I find it still impeccable today. :|
it was not when it was released
it is not much better after all the firmware updates
it made a big splash being the first CHEAP full-frame and those who wanted or needed that were so enthused with this new toy they paid little regard to its IQ shortcomings because it offered IQ benefits and features previously unavailable.

Endlessly trumpeting on that basis suckered me in to buying one; my most regrettable Canon purchase.
not for lack of due diligence, but for lack of honest and clear information about its weaknesses, which became clear enough after I used it for a while. .. and after others began posted about its less than ideal low ISO FPN issues.

If you still find it meets your needs you either lucked out with a good one or you don't mind crushing your blacks a few more levels than some of us.

*Facepalm

If your ultra-mega analytical about bringing up black's 4-stops, which by the way is why they are called black's not midtones, then you have some other serious issues to contend with. (IE: Timing for Ideal Light)

I'd have 7D landscapes that are heavily processed and pushed to +3 in the corners than were accepted for Istock. Thats some very strict standards.

canon rumors FORUM

Re: DxOMark vs. Reality
« Reply #222 on: January 15, 2013, 12:06:08 PM »

neuroanatomist

  • CR GEEK
  • ********
  • Posts: 13988
    • View Profile
Re: DxOMark vs. Reality
« Reply #223 on: January 15, 2013, 12:35:47 PM »
Last time you mentioned this "low ISO FPN issue" I did what you suggested and did a Google search for images, the only examples that search threw up were poorly exposed skies. Now I print big with a 1Ds MkIII, essentially the same sensor as the 5D MkII, and I have never had your "issue" I really want to see examples of what you are shooting, and how you are shooting it, that best illustrates your experiences.

I, too, would be interested in seeing examples of the low ISO FPN issue in real-world images, ones that don't require 100% crops and have descriptions like, "Canon 5D Mark II Fixed Pattern Noise, ISO 1600, +4 exposure," and, "...with Fill Light set to +100 to exaggerate the vertical banding in shadows."
EOS 1D X, EOS M, and lots of lenses
______________________________
Flickr | TDP Profile/Gear List

neuroanatomist

  • CR GEEK
  • ********
  • Posts: 13988
    • View Profile
Re: DxOMark vs. Reality
« Reply #224 on: January 15, 2013, 02:19:48 PM »
I think the WHAT is perfectly clear.  What's not clear is the real world benefit in common shooting situations for one photographer vs. another.   The sharper lens is an apt example - while having a sharper lens certainly offers a theoretical benefit, for many real world situations, that benefit is irrelevant because factors ranging from chosen subject (intentional softening for portraiture, a need to shoot at a very narrow aperture) to workflow/output (downsizing for web/small prints) render the 'benefit' of the increased lens sharpness moot.  For example, looking at the MTF chart of the MP-E 65mm, it's not a very sharp lens...but for the vast majority of the shots I take with that lens, a sharper version of it would not matter.

Thus, the request for real-world examples of shots that were rendered 'unusable' by the low ISO FPN of the 5DII.
EOS 1D X, EOS M, and lots of lenses
______________________________
Flickr | TDP Profile/Gear List

canon rumors FORUM

Re: DxOMark vs. Reality
« Reply #224 on: January 15, 2013, 02:19:48 PM »