October 20, 2014, 02:21:22 PM

Author Topic: DxOMark vs. Reality  (Read 80823 times)

Kernuak

  • 1D X
  • *******
  • Posts: 1108
    • View Profile
    • Avalon Light Photoart
Re: DxOMark vs. Reality
« Reply #135 on: December 15, 2012, 05:08:26 PM »

Quote
75% of what I shoot is ISO 800 and above, and I'm sure I'm not alone....therefore their score for DR is almost worthless to me.....AND misleading, since most of us are just going to look at the score, not the method/process for achieving the score.

It's called "landscape" and is well documented, so it's hardly misleading. If you're shooting at ISO 800 or higher, you should be looking either at the high ISO use case score or the measurement plots
Actually, I disagree with their high ISO scoring system and that is based on my experience of real world low light, high ISO shooting. They still set an arbitrary figure of 9 stops of DR on the high ISO scores and give it equal weighting to the amount of noise. However, when shooting in low light, the dynamic range is usually very low, which makes the weighting irrelevant. If they applied appropriate weighting to low noise, there would potentially be very different results. I'm sure the gap between the D800/800E and the D3s, D4, 1D X and 5D MkIII would be much higher in real world shooting in those conditions. Of course, I haven't measured it, so can't make a categorical assertion, but it is certainly my feeling, based on experience and also by looking at images online. The latter four cameras are designed to cope with low light conditions, so it is reasonable to expect that, that is the case. That is in fact my biggest gripe with DxO, that their separate scores are too arbitrary and don't appear to be based on any real world conditions. Perhaps they do say somewhere, but it would help if they published how they came to deciding on what factors to include in their scoring systems.

I try to follow you but do not understand what you mean.
please explain
In low light conditions, the dynamic range in a scene is much less than any modern sensor can capture, therefore whether or not one sensor can capture more DR than another is irrelevant. As I've never measured it, it is rather subjective, but I would estimate that DR is in the 6 stop range for many of the low light conditions that pushed me into buying the 5D MkIII (that's purely an example and wasn't the only reason for buying it). When you start getting into that sort of lighting, then a test should be looking more at the amount of noise in an image and less at how much DR the sensor can capture. DR is desirable for some photography, but unimportant in others and any test of a sensor should reflect that in any objective testing. What I would like to know, is how DxO come up with their cut-off points. For example, why 9 stops of DR for low light testing? Why not 10 or 8? Like Neuro and one or two others, I have been involved in scientific research (albeit nothing to do with optics) and I know that statistics can be used to emphasise pretty much anything if you don't look too closely. That's why it is important to do the right tests and the right analysis. You get it wrong and don't apply proper rigor to the testing in medical research and you could put patients' lives at risk further down the line.
Ultimately though, photography is an art form and therefore subjective. What one person might find acceptable, someone else might find it objectionable. Nowhere is this more obvious than noise and some even add noise to add atmosphere. Even blown highlights or blocked out shadows can add to an image if done correctly, although again, some people will hate it while others love it.
Canon 5D MkIII, 7D, 300mm L IS f/2.8 and a few other L's

canon rumors FORUM

Re: DxOMark vs. Reality
« Reply #135 on: December 15, 2012, 05:08:26 PM »

Kernuak

  • 1D X
  • *******
  • Posts: 1108
    • View Profile
    • Avalon Light Photoart
Re: DxOMark vs. Reality
« Reply #136 on: December 15, 2012, 05:11:43 PM »
@Neuro
You are always basing your conclusions for "who makes better cameras" on sales. But thats not how it works, you cant judge a camera on its sales.

Lets take the german/russian trabi: Cheap Car, but there where better ones. Nevertheless it sold. Was it the best? No.
Actually, I don't think he was saying any camera or sensor was better than another, rather that the perception of "better" (however it is measured) is actually irrelevant. The only thing that matters for a business is sales figures. If sales or profits are increasing, then they are achieving their goals (at least in most cases).
Canon 5D MkIII, 7D, 300mm L IS f/2.8 and a few other L's

neuroanatomist

  • CR GEEK
  • ********
  • Posts: 14708
    • View Profile
Re: DxOMark vs. Reality
« Reply #137 on: December 15, 2012, 05:41:55 PM »
@Neuro
You are always basing your conclusions for "who makes better cameras" on sales. But thats not how it works, you cant judge a camera on its sales.

No, I base my conclusions on which camera is better (for me) using criteria and performance aspects that matter to me.  In my opinion, that's what everyone buying a dSLR should use to make their own decision, and in most cases, that's likely the case.

It is a fact that Canon sells more dSLRs than Nikon.

Takes together, that suggests that in aggregate, more people have judged that Canon dSLRs satisfy more of their decision-making criteria.

Consider - there's an election, at the end of which (hanging chads notwithstanding), there's a winning candidate and a losing candidate. 'Winner' is based on a count of votes.  You voted for the candidate you thought was 'the best'...but that candidate may not have won.  You're entitled to your opinion, and if you picked the losing candidate, that doesn't invalidate your opinion. But at the end of the counting, you need to understand that the candidate you liked best lost, and your opinion is in the minority.

There are lots of ways to judge something.  The problem is that the vast majority of them are biased by personal opinion.  What's best for you isn't necessarily best for everyone. I use sales figures as a metric because that's one of very few unbiased metrics available.  We could argue all day about which candidate would be 'best for the country' or which camera is 'the best for taking pictures'.  But there was an election, with a winner and a loser...and Canon has sold more cameras than Nikon.

Those are the facts, plain and simple.  If it helps, while you don't have the option to be governed/represented by the losing candidate, you can go buy a Nikon dSLR.
EOS 1D X, EOS M, and lots of lenses
______________________________
Flickr | TDP Profile/Gear List

tortilla

  • PowerShot G1 X II
  • ***
  • Posts: 65
    • View Profile
Re: DxOMark vs. Reality
« Reply #138 on: December 16, 2012, 04:11:14 AM »
A skilled photographer is not the same as  he or she  is a good  copyist and know how a digital work flou works.  Have you  anytime worked in a darkroom with a negative film  then you will understand what I mean and likewise it is with a raw files.

A skilled photographer with also skills in the digital "dark room" kan do so much more than a photographer with no  photoshops skills.  And even more with a camera who has 14 stops DR.
best regards , pro since 30 years back.

Well, when you need 14 stops DR to "rescue" your pictures via Photoshop, then you should ask yourself, if you`re a real pro.
When it comes to the the point where you really need 14 stops of DR, you've taken your picture in a wrong way.

And: none will see the difference. Neither a customer nor other pro photographer.

So, does that mean HDR imaging is only for amateurs like me who are not able to exposure right, because they don't find the full-automatic setting? ;)  Real world lighting situations can sometimes have way above 14 EV DR.

dilbert

  • Canon EF 300mm f/2.8L IS II
  • *******
  • Posts: 3086
    • View Profile
Re: DxOMark vs. Reality
« Reply #139 on: December 16, 2012, 07:06:00 PM »
...
Consider - there's an election, at the end of which (hanging chads notwithstanding), there's a winning candidate and a losing candidate. 'Winner' is based on a count of votes.  You voted for the candidate you thought was 'the best'...but that candidate may not have won.  You're entitled to your opinion, and if you picked the losing candidate, that doesn't invalidate your opinion. But at the end of the counting, you need to understand that the candidate you liked best lost, and your opinion is in the minority.

Note that in elections, the outcome of who wins can be more complex where a person can win without having a overall majority of people vote in their favour.  Elections are just another analogy that fails as you could just as easily argue that it is not always the best person or the person with the best policies that wins but rather the person that sells themselves and their policies the best, meaning that a good percentage of people had the wool pulled over their eyes (i.e the winner should not have won.)

But really, lets not drag politics into discussion about cameras.

Quote
But there was an election, with a winner and a loser...and Canon has sold more cameras than Nikon.

I prefer to look at trends rather than specific points in time, i.e I'm more interested in knowing if Canon/Nikon's DSLR share is growing/shrinking than what specific percentage it is.

neuroanatomist

  • CR GEEK
  • ********
  • Posts: 14708
    • View Profile
Re: DxOMark vs. Reality
« Reply #140 on: December 16, 2012, 07:29:41 PM »
I prefer to look at trends rather than specific points in time, i.e I'm more interested in knowing if Canon/Nikon's DSLR share is growing/shrinking than what specific percentage it is.

That's fine, as long as you realize that one quarter does not equal a trend...  FWIW, Canon has gained market share at Nikon's expense for the past 5 years or so. 
EOS 1D X, EOS M, and lots of lenses
______________________________
Flickr | TDP Profile/Gear List

dilbert

  • Canon EF 300mm f/2.8L IS II
  • *******
  • Posts: 3086
    • View Profile
Re: DxOMark vs. Reality
« Reply #141 on: December 17, 2012, 01:44:01 AM »
I prefer to look at trends rather than specific points in time, i.e I'm more interested in knowing if Canon/Nikon's DSLR share is growing/shrinking than what specific percentage it is.

That's fine, as long as you realize that one quarter does not equal a trend...  FWIW, Canon has gained market share at Nikon's expense for the past 5 years or so.

Well it is hard to see Canon maintaining its sales numbers without continued fire sales of (for example) the 5D3 at a discount of over $500 of the MSRP. Or one might say that such fire sales are evidence that vendors are struggling to sell 5D3 stock at MSRP and that Canon sales numbers will suffer if they don't happen.

canon rumors FORUM

Re: DxOMark vs. Reality
« Reply #141 on: December 17, 2012, 01:44:01 AM »

neuroanatomist

  • CR GEEK
  • ********
  • Posts: 14708
    • View Profile
Re: DxOMark vs. Reality
« Reply #142 on: December 17, 2012, 06:25:13 AM »
Well it is hard to see Canon maintaining its sales numbers without continued fire sales of (for example) the 5D3 at a discount of over $500 of the MSRP. Or one might say that such fire sales are evidence that vendors are struggling to sell 5D3 stock at MSRP and that Canon sales numbers will suffer if they don't happen.

Fire sale...hyperbolize much?  As for sales numbers 'suffering' I just had a look at Amazon's dSLR sales rankings this busy holiday shopping season, and the top three best sellers are from....Canon (top four on the list, but the Canon T4i is #1 and #4, with different lenses).  And while we all know the D800 is a much better camera (so telleth us the almighty DxOMark), and it's cheaper, too, the 5DIII is in the Top 20 (#14), and the D800 is not (#21).  Canon does seem to be suffering badly, oh my.  ::)
EOS 1D X, EOS M, and lots of lenses
______________________________
Flickr | TDP Profile/Gear List

neuroanatomist

  • CR GEEK
  • ********
  • Posts: 14708
    • View Profile
Re: DxOMark vs. Reality
« Reply #143 on: December 17, 2012, 09:59:57 AM »
Is Amazon a reference?

It's a reference in the sense that one can see the data, so claims can be substantiated.  Do you have data that you can show to substantiate the claim than the D800 is outselling the 5DIII 20:1 in Sweden? 

I wonder how many cameras are sold in Sweden, compared to the USA?  There are ~33 people in the US for every 1 person in Sweden.  Raw numbers matter far more than ratios.

Nikon V1 best selling camera in Europe

I have read (in a Nikon press release) that the Nikon 1 was identified as the best selling compact camera system in Europe in 2012.  Where was it published that the Nikon V1 was the best selling camera in Europe?

Nikon has increased  theres SLR selling with 26% 2012.

Not exactly.  I think you're looking at the results for the 1st half of their FY2013, which are the 2nd and 3rd quarters of the 2012 calendar year (Canon and Nikon's fiscal years are offset).  Performance in the first calendar quarter of the year were less impressive, barely making up for the loss in the last half of 2011. 

Over those same two quarters (2Q12 and 3Q12), Canon reported a gain of 47% and a loss of 7%, respectively, in the dSLR segment.  Which is better, when looked at over a 6 month period - a 26% gain, or a 47% gain for the first 3 months and a 7% loss for the next 3 months?
EOS 1D X, EOS M, and lots of lenses
______________________________
Flickr | TDP Profile/Gear List

verysimplejason

  • 1D X
  • *******
  • Posts: 1353
    • View Profile
    • My Flickr Account
Re: DxOMark vs. Reality
« Reply #144 on: December 17, 2012, 10:40:15 AM »
Here's some statistics from flickr.   I don't know how relevant this is but this shows the most used cameras by flickr users.

http://www.flickr.com/cameras/

verysimplejason

  • 1D X
  • *******
  • Posts: 1353
    • View Profile
    • My Flickr Account
Re: DxOMark vs. Reality
« Reply #145 on: December 17, 2012, 10:41:45 AM »
Hi Mike, but sorry we can't see anything... Can you re-post the links of your sources?  Thanks.

neuroanatomist

  • CR GEEK
  • ********
  • Posts: 14708
    • View Profile
Re: DxOMark vs. Reality
« Reply #146 on: December 17, 2012, 11:27:42 AM »
What I show you  is that there a world out side yours
I'm well aware of that, thanks.  I'm quoting sales figures from Canon and Nikon reflecting global sales, and aggregated market reports, and those show that Canon has a much larger market share than Nikon, that Canon has outsold Nikon year over year for the past 5 years, with the exception of 3Q12, and that means that Canon is the world leader in dSLR sales.  "Best and most beautiful" are subjective, billions of ¥ in revenue and number of units sold are not. 

Nikon SLR NR 1 selling in Japan...in  2011
Ok, so let's look at that.  There are two ways to interpret that statement, either Nikon was #1 in Japan in 2011, or Nikon had the #1 selling model in Japan in 2011.  The D3100 was the best selling dSLR in Japan in 2011.  The fact that the D3100 was the best selling dSLR in Japan in 2011 is misleading - the D3100 was the current model in it's segment for the entire year; in that segment, Canon's Kiss X4 was the current model at the beginning of the year, and was replaced by the Kiss X5 during 2011.  If you add the Kiss X4 and Kiss X5 sales together, that far exceeds D3100 sales.

Moreover, 6 of the top 20 cameras on that list are Canon models compared to 4 Nikon models on the list, and if you add up the sales by vendor, Canon dSLRs had 27% of the Japan market in 2011, compared to 22.5% for Nikon.  So...was Nikon really #1 in Japan in 2011?  No.
EOS 1D X, EOS M, and lots of lenses
______________________________
Flickr | TDP Profile/Gear List

neuroanatomist

  • CR GEEK
  • ********
  • Posts: 14708
    • View Profile
Re: DxOMark vs. Reality
« Reply #147 on: December 17, 2012, 02:38:32 PM »
rather uninteresting who sells the most SLRs - right?

To shareholders, such things are very interesting...and because publicly-traded companies have an obligation to their shareholders (it's their primary obligation, in fact), such things are of paramount importance to those companies.  If it's important to them, it should be important to consumers of their products.
EOS 1D X, EOS M, and lots of lenses
______________________________
Flickr | TDP Profile/Gear List

canon rumors FORUM

Re: DxOMark vs. Reality
« Reply #147 on: December 17, 2012, 02:38:32 PM »

PackLight

  • Guest
Re: DxOMark vs. Reality
« Reply #148 on: December 17, 2012, 03:00:55 PM »
So I guess DxO scores have a negative impact on CAJ share prices, and the dividend CAJ pays.
We should watch DxO's new scoring system for lenses closely, we may want to short the stock.

sdsr

  • 5D Mark III
  • ******
  • Posts: 685
    • View Profile
Re: DxOMark vs. Reality
« Reply #149 on: December 17, 2012, 03:29:50 PM »

Well it is hard to see Canon maintaining its sales numbers without continued fire sales of (for example) the 5D3 at a discount of over $500 of the MSRP. Or one might say that such fire sales are evidence that vendors are struggling to sell 5D3 stock at MSRP and that Canon sales numbers will suffer if they don't happen.

If "fire sales" matter, what would you have us infer from the current $700 discount on the D600 + kit lens via certain (all?) US (only US?) vendors?

canon rumors FORUM

Re: DxOMark vs. Reality
« Reply #149 on: December 17, 2012, 03:29:50 PM »