October 21, 2014, 08:01:00 PM

Author Topic: IQ comparison; or how meaningful is DXO  (Read 9889 times)

Apop

  • EOS M2
  • ****
  • Posts: 153
    • View Profile
    • Apophoto
Re: IQ comparison; or how meaningful is DXO
« Reply #45 on: September 06, 2013, 07:06:21 AM »
What I wanted to show is, that the DXOmark score is nonsense. Their measurements are nice, but even they lack information to conclude which camera makes the best pictures. Sure, there are some things that can be derived from the DXO numbers, but others not. One is for example noise; you can have the same amount of noise for two cameras, but to the human eye they look different, because of the patterns and the colors they appear in.

I also found the test interessting, because it even worked for myself, since I forgot most positions of the cameras, and also did a blind test (and I judged the pictures differently then when I knew from what camera they were).

My conclusion.
-The DXOmark score difference between the 70D and the D7100 is definitly not justified.
-The Fujifilm x-pro1 makes some nice pictures.
-The Sony a99 is a bit dissapionting, the D7100 and the 70D produce pictures that are about on the same level.
-FF is better, but not but the difference is not as big as I thought (the 70D was often rated higher than the D600 at JPEG).

I really tried to make a fair test; I took samples from colorcards to show noise performance at low ISO, I took parts with high contrast and some with details. So I think the comparison is quite fair. If it is meaningful to you, I don't know, this is up to you.

You do not really think that posting selected OOF crops of JPEGs with different noise reduction proves anything, do you?

Tell me which brand you want to see a winner, and I will post similar crops from IR proving that that brand is the best.

Nokia :p?

If people want to proof that DXO is meanigless they need to do a bit better than posting samples like this.
See if you can replicate their results , and hope that they were wrong , then people try replicate yours and hope you were right ! :p

canon rumors FORUM

Re: IQ comparison; or how meaningful is DXO
« Reply #45 on: September 06, 2013, 07:06:21 AM »

aj1575

  • EOS M2
  • ****
  • Posts: 173
    • View Profile
Re: IQ comparison; or how meaningful is DXO
« Reply #46 on: September 06, 2013, 07:29:31 AM »
I think to say that DXO is complete nonsense isn't fair.

I did not say that DXO is complete nonsense, I said that the DXOmark score is nonesense.

aj1575

  • EOS M2
  • ****
  • Posts: 173
    • View Profile
Re: IQ comparison; or how meaningful is DXO
« Reply #47 on: September 06, 2013, 07:47:48 AM »

You do not really think that posting selected OOF crops of JPEGs with different noise reduction proves anything, do you?

Tell me which brand you want to see a winner, and I will post similar crops from IR proving that that brand is the best.

Two things here, dp uses different lenses. For Canon APS-C they use a EF 50mm f1.4 for Nikon the 50mm f1.4 AF-S and for the FF they use the 85mm f1.8 from each brand. They shot at f8. So the differencr in quality should be rather small (check the tests of these lenses at various sites, also DXOmark)

The OOF claim is understandable. I was thinking the same when I looked at the samples. The D600 looks worse in the poker card than the 70D. But I do not think that this is an OOF problem, for several reasons.
1. The 7100D and other Nikons show this problem, so either they focus many Nikon cameras wrong, but not the Canons; or the Nikon AF is not accurate enough...
2. This softnes does not appear on the whole picture, you just need to move down on dp-site tool a little bit to the black and white circle. This seems to be on the same plane, but it looks sharp enough to me.

So to me, this is not an OOF problem, the Nikon sensors just have a problem to resolve that properly.

Pi

  • 1D Mark IV
  • ******
  • Posts: 937
    • View Profile
    • Math and Photography
Re: IQ comparison; or how meaningful is DXO
« Reply #48 on: September 06, 2013, 08:13:25 AM »

You do not really think that posting selected OOF crops of JPEGs with different noise reduction proves anything, do you?

Tell me which brand you want to see a winner, and I will post similar crops from IR proving that that brand is the best.

Two things here, dp uses different lenses. For Canon APS-C they use a EF 50mm f1.4 for Nikon the 50mm f1.4 AF-S and for the FF they use the 85mm f1.8 from each brand. They shot at f8. So the differencr in quality should be rather small (check the tests of these lenses at various sites, also DXOmark)

How about DOF and focus plane differences?

Quote
So to me, this is not an OOF problem, the Nikon sensors just have a problem to resolve that properly.

Tell me that you are kidding.

http://www.the-digital-picture.com/Reviews/ISO-12233-Sample-Crops.aspx?Lens=640&Camera=614&Sample=0&FLI=0&API=2&LensComp=287&CameraComp=453&SampleComp=0&FLIComp=0&APIComp=2

D7100:



Mt Spokane Photography

  • EF 50mm F 0.7 IS
  • *********
  • Posts: 8864
    • View Profile
Re: IQ comparison; or how meaningful is DXO
« Reply #49 on: September 06, 2013, 10:56:26 AM »
DXO measures the sensor characteristics.  Those are hard facts. 

No, they are the results of DxO's testing process and interpretations. There's quite a bit of disagreement as to whether or not DxO's tests are accurate and/or meaningful.

As someone who ran a lab for NASA, I think the test data  represents hard facts.  I've been thru hell repeating and defending my tests by experts who are skeptics, so I can recognize when someone is incompetent. I also think they are accurate results for the samples they had.   someone can always find a unit that performs differently.   You should be able to create the same setup and compare results.
 
Now, interpretation of the data is where people tend to disagree, and assigning a value number to a sensor based on a secret formula is silly, particularly when you link it to a camera model and give the camera a score.  There is a lot more to a camera than the sensor, and DXO does not test that.

qwerty

  • PowerShot G1 X II
  • ***
  • Posts: 49
    • View Profile
Re: IQ comparison; or how meaningful is DXO
« Reply #50 on: September 06, 2013, 03:47:59 PM »
Just to try to drag the thread back on-topic, here are my favorites in each row:
1d
2a
3d
4b
5c (d had less noise, but seemed oversmoothed)
6d (d had less noise and seemed "sharp enough", but I get the feeling that if there were more detail it would have looked oversmoothed)

I want to thank the OP for making this comparison, and would be curious to see more of the same.  I know a few issues have been pointed out in the thread, and a second pass might be even better.

If I had the time, what I would do is download the raw files of a standard scene from somewhere (I thought dpreview posted them somewhere, but either I am misremembering, bad at searching, or they took them down when they added their scene comparison widget) and convert them yourself using lightroom with the same settings, except scaled so that the image sizes are the same for the different cameras.  You can even keep them as tiffs until final conversion to keep everyone (less un-)happy.

Oh, and how about a less inflammatory title for the next thread? : o )
« Last Edit: September 06, 2013, 04:16:46 PM by qwerty »

canon rumors FORUM

Re: IQ comparison; or how meaningful is DXO
« Reply #50 on: September 06, 2013, 03:47:59 PM »