August 22, 2014, 10:09:46 AM

Author Topic: DxOMark vs. Reality  (Read 77819 times)

Alrik89

  • PowerShot G1 X II
  • ***
  • Posts: 60
    • View Profile
Re: DxOMark vs. Reality
« Reply #90 on: October 11, 2012, 10:02:50 AM »
Guys, you've got a real problem, when you assess camera sensors basing on some stupid numbers like DxO.

If you're a professional photographer (a skilled one) then none of your customers will ever see the difference in the IQ/dynamic range/whatever between a Nikon and a Canon DSLR.

So, this discussion, which one has mounted the biggest d1ck... sorry, sensor, in a camera case, is total bu11shit. I think, none of you has ever come into a situation, when he thought stuff like "crap, the dynamic range in this picture is too low. The customer will be very unpleased. I wish, i had a Nikon."

Get a life.

A skilled photographer is not the same as  he or she  is a good  copyist and know how a digital work flou works.  Have you  anytime worked in a darkroom with a negative film  then you will understand what I mean and likewise it is with a raw files.

A skilled photographer with also skills in the digital "dark room" kan do so much more than a photographer with no  photoshops skills.  And even more with a camera who has 14 stops DR.
best regards , pro since 30 years back.

Well, when you need 14 stops DR to "rescue" your pictures via Photoshop, then you should ask yourself, if you`re a real pro.
When it comes to the the point where you really need 14 stops of DR, you've taken your picture in a wrong way.

And: none will see the difference. Neither a customer nor other pro photographer.

canon rumors FORUM

Re: DxOMark vs. Reality
« Reply #90 on: October 11, 2012, 10:02:50 AM »

marinien

  • Canon AE-1
  • ***
  • Posts: 78
    • View Profile
Re: DxOMark vs. Reality
« Reply #91 on: October 11, 2012, 10:22:17 AM »
Can someone please calculate and compare the surface areas between the orange and red lines because I'm still not convinced when and how to interprete those graphs and which sensor is better and when?
DxO measurements is a bullshit if they can't provide the simple math formulas to calculate the integrals. I'm so dissapointed...

Euh, I don't see any reason to calculate the surface areas between the lines in order to know which sensor is better and when.
Just have a look at theses graphs again and forget everything about integral.

<sarcasm off> :D

Anyway - the bigger the area in either low or high iso part of the graph, the bigger the difference between them there and the simplest way to calculate this area are integrals when you know the math formulas. But anyway it was just to cheer you all up in this serious discussion...

Haha, thanks  ;)
7D | EF-S 17-55 | EF 100mm f/2.8 L IS Macro | 580EX II | Benro C3780T + Markins M20

PackLight

  • Guest
Re: DxOMark vs. Reality
« Reply #92 on: October 11, 2012, 12:22:26 PM »
The 1DX got good grades from DxO: does that mean it's a crappy camera since those grades have nothing to do with reality?  ;D

This will be very intresting to know, Canon have increased the DR   and shows very good high iso scores
if the figures are OK  ( are  DXO still crap?)

Yes, the assumptions and overall score they draw from the tests are still crap. There scores are drawn from to narrow of a parameter to be of real use.

shinyknights

  • Guest
Re: DxOMark vs. Reality
« Reply #93 on: October 11, 2012, 03:01:53 PM »
I'm curious if DxOMark is a credible source of info.
Here, they say that the Canon 5D Mark III is good until 2293 ISO
With Nikon's D800, it says that it is good until 2853 ISO.
They dont even provide picture samples!

http://www.dxomark.com/index.php/Cameras/Camera-Sensor-Database/Canon/EOS-5D-Mark-III
http://www.dxomark.com/index.php/Cameras/Camera-Sensor-Database/Nikon/D800

And I'm reading more and I see examples on cameralabs.com.
And here, they show that Canon's 5D Mark III destroys Nikon's D800, particularly at 12800 ISO.

http://www.cameralabs.com/reviews/Canon_EOS_5D_Mark_III/Canon_5D3_vs_Nikon_D800_noise.shtml

Any thoughts on this guys?

PackLight

  • Guest
Re: DxOMark vs. Reality
« Reply #94 on: October 11, 2012, 03:17:58 PM »
I'm curious if DxOMark is a credible source of info.
Here, they say that the Canon 5D Mark III is good until 2293 ISO
With Nikon's D800, it says that it is good until 2853 ISO.
They dont even provide picture samples!

http://www.dxomark.com/index.php/Cameras/Camera-Sensor-Database/Canon/EOS-5D-Mark-III
http://www.dxomark.com/index.php/Cameras/Camera-Sensor-Database/Nikon/D800

And I'm reading more and I see examples on cameralabs.com.
And here, they show that Canon's 5D Mark III destroys Nikon's D800, particularly at 12800 ISO.

http://www.cameralabs.com/reviews/Canon_EOS_5D_Mark_III/Canon_5D3_vs_Nikon_D800_noise.shtml

Any thoughts on this guys?


I think if you use DxO information it is important to know in what context they test, the test reults and ignore the values they place as scores.

As for the test at ISO 12800 you mention, as far as I am concerned the samples would be irrelevant as well because I wouldn't go that far with the 5D. I would with the 1D X.  I think a comparison at ISO 6400, an ISO I might go to on occasion tells more as does the ISO 3200 comparison.
« Last Edit: October 11, 2012, 03:23:45 PM by PackLight »

K-amps

  • 1D X
  • *******
  • Posts: 1512
  • Whatever looks great !
    • View Profile
Re: DxOMark vs. Reality
« Reply #95 on: October 11, 2012, 04:24:19 PM »
I'm curious if DxOMark is a credible source of info.
Here, they say that the Canon 5D Mark III is good until 2293 ISO
With Nikon's D800, it says that it is good until 2853 ISO.
They dont even provide picture samples!

http://www.dxomark.com/index.php/Cameras/Camera-Sensor-Database/Canon/EOS-5D-Mark-III
http://www.dxomark.com/index.php/Cameras/Camera-Sensor-Database/Nikon/D800

And I'm reading more and I see examples on cameralabs.com.
And here, they show that Canon's 5D Mark III destroys Nikon's D800, particularly at 12800 ISO.

http://www.cameralabs.com/reviews/Canon_EOS_5D_Mark_III/Canon_5D3_vs_Nikon_D800_noise.shtml

Any thoughts on this guys?


They have certain criteria like 30db signal to noise ratio ... any shot that gets below this is tossed out. If they change their criteria to (taking an arbitrary number) 25db, then perhaps 5d3 will have 11000 ISO (another arbitrary number)  and D800 will have 4000 ISO (Yet another arbitrary number)... shifting the criteria around will swing the score back to the Canon Camp. Thats what pissed people about them is the criteria they use to weigh qualities of a sensor for it's "Overall Score".

The criteria they use effectively nullifies Canon's real world high ISO advantage.

The other criteria they have in place is that at the 30db SNR level, at least 18 bit color depth be mantained for the ISO score, now who plucked this number from someones...  expletive is another debate altogether, but it again weights the balance in favor of the Sonikon sensors... (The color bit depth rears it's ugly head again for overall scores.)

I think even with current technology levels fo Canon sensors, with their high ISO advantage, DxO could tweak their criteria and Canon scores could be above Nikons... but there is one area where playing fuzzy math will not help canon and that is the DR number... this is what many Canon Loyaists want Canon to improve.
« Last Edit: October 11, 2012, 04:30:34 PM by K-amps »
EOS-5D Mk.iii 
Sigma 24-105mm F4 ART; EF 70-200 F/2.8L Mk.II; EF 85mm L F/1.2 Mk. II; EF 100mm L F/2.8 IS Macro, 50mm F/1.8ii;  TC's 2x Mk.iii; 1.4x Mk.iii

elflord

  • 5D Mark III
  • ******
  • Posts: 705
    • View Profile
Re: DxOMark vs. Reality
« Reply #96 on: October 11, 2012, 08:00:28 PM »
I'm curious if DxOMark is a credible source of info.
Here, they say that the Canon 5D Mark III is good until 2293 ISO
With Nikon's D800, it says that it is good until 2853 ISO.
They dont even provide picture samples!

Their scores are based on measurements, not picture samples.

If you want the full story behind the scores, click on the measurements tab. The scores compress a lot of information using subjective (and therefore somewhat arbitrary) criteria.

canon rumors FORUM

Re: DxOMark vs. Reality
« Reply #96 on: October 11, 2012, 08:00:28 PM »

straub

  • Guest
Re: DxOMark vs. Reality
« Reply #97 on: October 12, 2012, 03:29:33 AM »
Yes - but possibly not in the way I think you mean. The original RAW image is spatially oversampled (20MP or more) with respect to the final output resolution (8MP). The process of converting the original high-resolution image to the smaller image is downsampling.

The Wikipedia pages are helpful here, and explain this probably better than I can:

  http://en.wikipedia.org/wiki/Oversampling
  http://en.wikipedia.org/wiki/Downsampling

These also give an example of how this can be exploited to give more resolution (DR) than the actual converter (sensor pixel).

And that must be why an 8KHz downsampling of a CD has more than double the dynamic range of the original, right? You can google theorems, fine, but apparently have no idea what they mean and where they are applicable. So you must work at DXO, then?

Try a test--take a D800 photo of Stouffer wedges, push exposure in post how much you like, and make a note of the darkest one. Now downsample to 1/64, i.e. ~0.5mpix and repeat. How many extra wedges appeared? Exactly as many as how much the *oversampling theorem* is applicable here--i.e. zero.

MarkII

  • PowerShot G1 X II
  • ***
  • Posts: 65
    • View Profile
Re: DxOMark vs. Reality
« Reply #98 on: October 12, 2012, 05:46:57 AM »
And that must be why an 8KHz downsampling of a CD has more than double the dynamic range of the original, right? You can google theorems, fine, but apparently have no idea what they mean and where they are applicable. So you must work at DXO, then?
The downsampling will reduce the bandwidth but increase the resolution of the samples. You trade off precision in the amplitude (quantisation noise) for bandwidth (loosing high frequencies). This is exactly the same as happens when you down sample a digital image - and just as you might not want to listen to all your CDs at 8KHz you probably don't want to print out all your images at 600x400 no matter how much this improves the DR.

I posted the Google links as they are usefully informative on the subject. You, rather than posting an informed and referenced response, just posted an insult (FYI, I am a Physicist  who has worked with DSP and signal processing for more than 20 years, and I have no connection to DXO).

Try a test--take a D800 photo of Stouffer wedges, push exposure in post how much you like, and make a note of the darkest one. Now downsample to 1/64, i.e. ~0.5mpix and repeat. How many extra wedges appeared? Exactly as many as how much the *oversampling theorem* is applicable here--i.e. zero.
Well, I do not have a D800 to test (and have not commented on this specifically).

I find that the DXO measurement data matches pretty well my experience when shooting with those cameras that I have (GF1, 40D, 5DII, 5DII). You can take issue with the fairly arbitrary criteria that they use to produce their overall scores and I am sure that DXO sometimes make mistakes, but the individual measurements - if you understand what they are - have always looked pretty good to me and correlate well with my experience of the 5DIII (particularly when compared to the 5DII).

Simply wanting something to be different does not make it so.

tron

  • 1D X
  • *******
  • Posts: 1806
    • View Profile
Re: DxOMark vs. Reality
« Reply #99 on: October 12, 2012, 06:46:51 AM »
I can understand the improvement in noise when downsampling but can someone prove that downsampling an image (to 8Mpix for example) improves DR?

I ask this because I believe in practice what is burned in the highlights has been lost forever and when  there are shadow areas I cannot image how a "dark" pixel will benefit from its neighbour equally dark ones.

Thanks.

elflord

  • 5D Mark III
  • ******
  • Posts: 705
    • View Profile
Re: DxOMark vs. Reality
« Reply #100 on: October 12, 2012, 07:25:17 AM »
I can understand the improvement in noise when downsampling but can someone prove that downsampling an image (to 8Mpix for example) improves DR?

I ask this because I believe in practice what is burned in the highlights has been lost forever and when  there are shadow areas I cannot image how a "dark" pixel will benefit from its neighbour equally dark ones.

Dynamic range is the number of stops between "saturation point" and blackpoint. Saturation point is where the highlights get "burned out". This doesn't change. Black point is the point at which SNR is 0db (that is, signal to noise ratio is 1).  Downsampling reduces noise, so the SNR at what used to be the blackpoint goes up (to 5db for example) and the new blackpoint after downsampling  is some way down from that.


tron

  • 1D X
  • *******
  • Posts: 1806
    • View Profile
Re: DxOMark vs. Reality
« Reply #101 on: October 12, 2012, 09:48:57 AM »
I can understand the improvement in noise when downsampling but can someone prove that downsampling an image (to 8Mpix for example) improves DR?

I ask this because I believe in practice what is burned in the highlights has been lost forever and when  there are shadow areas I cannot image how a "dark" pixel will benefit from its neighbour equally dark ones.

Dynamic range is the number of stops between "saturation point" and blackpoint. Saturation point is where the highlights get "burned out". This doesn't change. Black point is the point at which SNR is 0db (that is, signal to noise ratio is 1).  Downsampling reduces noise, so the SNR at what used to be the blackpoint goes up (to 5db for example) and the new blackpoint after downsampling  is some way down from that.
Thanks!

nightbreath

  • 6D
  • *****
  • Posts: 456
    • View Profile
    • Свадебный фотограф в Днепропетровске
Re: DxOMark vs. Reality
« Reply #102 on: October 13, 2012, 02:44:08 AM »
Dynamic range is the number of stops between "saturation point" and blackpoint. Saturation point is where the highlights get "burned out". This doesn't change. Black point is the point at which SNR is 0db (that is, signal to noise ratio is 1).  Downsampling reduces noise, so the SNR at what used to be the blackpoint goes up (to 5db for example) and the new blackpoint after downsampling  is some way down from that.
I didn't quite catch this. Could you expand your answer on my example? Here's how I understand downsampling:

Assume that we have 4 neighboring pixels that represent in reality 4 black squares, but in our RAW file (due to noise impact) they have next brightness levels:
1 0
0 3

So, when I downsample I get 1 pixel that has brightness level 1 (for example, if I downsample by next formula: (1+0+0+3)/4 ).

It means that before downsampling we were looking at image with average noise level equal to 1 and we do the same after downsampling.

So what am I doing wrong compared to all of you that have noise level reduced after downsammpling?

P.S. Sorry for asking silly questions, i'd just like to understand  :)
Wedding photography. My personal website: http://luxuryphoto.com.ua

canon rumors FORUM

Re: DxOMark vs. Reality
« Reply #102 on: October 13, 2012, 02:44:08 AM »

BXL

  • PowerShot G1 X II
  • ***
  • Posts: 54
    • View Profile
Re: DxOMark vs. Reality
« Reply #103 on: October 13, 2012, 06:38:39 AM »
Those of you who defend DxO scores, why do you have Canon gear?
Because Canon offers the better package. It's not all about the sensor. It's the combination of the sensor, AF, ergonomics, processor and lenses. Similar to cars, where one company offers the better engine, but another company offers the better car. The best engine won't do the job if the car "sucks" ;)
450D | EF 50 f/1.8 | EF 100 f/2 | 17-50mm f/2.8 VC

heptagon

  • EOS M2
  • ****
  • Posts: 151
    • View Profile
Re: DxOMark vs. Reality
« Reply #104 on: October 13, 2012, 07:43:39 AM »
Those of you who defend DxO scores, why do you have Canon gear?
Because Canon offers the better package. It's not all about the sensor. It's the combination of the sensor, AF, ergonomics, processor and lenses. Similar to cars, where one company offers the better engine, but another company offers the better car. The best engine won't do the job if the car "sucks" ;)
You wouldn't go offroad with a racecar or wear slick tires in the mud.

Every job needs the right tool.

canon rumors FORUM

Re: DxOMark vs. Reality
« Reply #104 on: October 13, 2012, 07:43:39 AM »