December 21, 2014, 12:02:08 AM

Author Topic: DxO & MTF Charts ... a little help please!  (Read 5204 times)

mackguyver

  • Canon EF 300mm f/2.8L IS II
  • ********
  • Posts: 3048
  • Who Dares Wins
    • View Profile
    • My Personal Work
Re: DxO ... a little help please!
« Reply #15 on: February 13, 2014, 01:34:49 PM »
The best way to use DxO's measurements are for comparison in scenarios you would actually use.    I particularly like to use the the "Field Map" view.

Just watch out for their mistakes.  For example,  compare the 17-40L wide open to the 16-35L II stopped down two stops to f/5.6, and check out the field maps.  The 17-40 is infamous for mushy corners wide open, but you don't see that on DxO.  Is it believable that the corners of the 17-40 wide open are sharper than the corners of the 16-35 at f/5.6?  Not to me…
I'm still scratching my head on those measurements, too, at least after getting over the 70-200 f/2.8 IS I/II review :)

I always like to use multiple resources - Lenstip, SLRgear, & Photozone are usually the three that seem to have the best reviews/measurements.  As they're all free, it makes it pretty easy to be an informed consumer now that we've reached the death of almost all photo shops.  I like to borrow lenses from CPS, too, before making big purchases.

1. How does the new Tamron 150-600 compare to my 300 f/.28 II IS at 300mm & f/5.6 in terms of sharpness? 
DxOMark shows the 300mm is sharper across the entire frame.
In the first set of images (150-600 vs 300), the 300mm is evenly colored, while the 150-600 isn't ... is that representing sharpness or CA?
That shot represents sharpness.

canon rumors FORUM

Re: DxO ... a little help please!
« Reply #15 on: February 13, 2014, 01:34:49 PM »

meli

  • EOS M2
  • ****
  • Posts: 167
    • View Profile
Re: DxO ... a little help please!
« Reply #16 on: February 13, 2014, 01:52:00 PM »
... all this got me thinking that for a company like DxO that is capable/equipped to conduct sophisticated lens/sensor tests, what stops them from comparing lenses/sensors at 10 different ISO measurements and in 10 different conditions with 10 different subject movements and then come up with scores in a chart, so people can come to a more informed conclusion :-\
...and with 10 different samples of the same camera/lens picked with appropriate criteria.
They probably could, but they wont. What you wish takes time, personnel, and money. And all this to run a free service. That doesnt fly really well.
Quote
For example they could, in controlled environment, throw a ball and see how well the camera/lens can auto focus at various apertures and then come up with several scores for sharpness ...
Dxo doesnt do AF. Its a whole other beast and frankly its difficult to come up with an objective/exhaustive suite of lab/field AF tests.

lescrane

  • PowerShot G1 X II
  • ***
  • Posts: 66
    • View Profile
Re: DxO ... a little help please!
« Reply #17 on: February 13, 2014, 02:18:00 PM »
So since some of you are willing to explain wtf these charts mean to those of us who have never seen them, can you give a very basic explain. of this chart?
  Obviously the darker green the better, the more uniform the better, but what does the square area mean? Is it the frame of the image area? eg, darker in the middle is sharper, lighter around the edges, less sharp?

mackguyver

  • Canon EF 300mm f/2.8L IS II
  • ********
  • Posts: 3048
  • Who Dares Wins
    • View Profile
    • My Personal Work
Re: DxO ... a little help please!
« Reply #18 on: February 13, 2014, 02:19:08 PM »
So since some of you are willing to explain wtf these charts mean to those of us who have never seen them, can you give a very basic explain. of this chart?
  Obviously the darker green the better, the more uniform the better, but what does the square area mean? Is it the frame of the image area? eg, darker in the middle is sharper, lighter around the edges, less sharp?
Yes, that it exactly right.  The rectangle is the sensor area and darker (green, vs. yellow or red) and more uniform is better :)

In this case, the Tamron puts in a pretty good showing considering the price difference.
« Last Edit: February 13, 2014, 02:21:18 PM by mackguyver »

AlanF

  • 1D X
  • *******
  • Posts: 1176
    • View Profile
Re: DxO ... a little help please!
« Reply #19 on: February 13, 2014, 02:25:56 PM »
Mac
Thanks for bringing up this method of comparison. I have just used it to compare the 150-600mm with the 100-400 in a thread about the Tammy to disprove an assertion that it is a confirmed soft lens.

DxO measurements are very useful for comparing different lenses on the same body or even better the same lens on different bodies.
5D III, EOS-M, Powershot SX50, 300/2.8 II, 1.4xTC III, 2xTC III, 70-200/4 IS, 24-105, 15-85, 100-400 II, Sigma 10-20, EOS-M, 18-55, f/2 22.

mackguyver

  • Canon EF 300mm f/2.8L IS II
  • ********
  • Posts: 3048
  • Who Dares Wins
    • View Profile
    • My Personal Work
Re: DxO ... a little help please!
« Reply #20 on: February 13, 2014, 02:34:41 PM »
Mac
Thanks for bringing up this method of comparison. I have just used it to compare the 150-600mm with the 100-400 in a thread about the Tammy to disprove an assertion that it is a confirmed soft lens.

DxO measurements are very useful for comparing different lenses on the same body or even better the same lens on different bodies.
No problem and I saw you feeding the troll on that post, LOL.  It is a nice tool and you can compare up to three lenses at a time.  As Neuro says, it's not perfect, but it's free and relatively easy to use.  Also, I forgot to mention the-digital-picture.com in my last post.  Their tools are nice to use for comparisons as well.

Rienzphotoz

  • Canon EF 300mm f/2.8L IS II
  • ********
  • Posts: 3326
  • Peace unto all ye Canon, Nikon & Sony shooters
    • View Profile
Re: DxO ... a little help please!
« Reply #21 on: February 13, 2014, 02:42:17 PM »
I forgot to mention the-digital-picture.com in my last post.  Their tools are nice to use for comparisons as well.
When I first started out with DSLRs, the only source I used to trust for reviews was the-digital-picture.com ... also, they make it very simple to understand.
Canon 5DMK3 70D | Nikon D610 | Sony a7 a6000 | RX100M3 | 16-35/2.8LII | 70-200/2.8LISII | 100/2.8LIS | 100-400LIS | 40/2.8 | 50/1.4 | 85/1.8 | 600EX-RTx2 | ST-E3-RT | 24/3.5 T-S | 10-18/4 OSS 16-50 | 24-70/4OSS | 55/1.8 | 55-210 OSS | 70-200/4 OSS | 28-300VR | HVL-F43M | GoPro Black 3+ & DJI Phantom

canon rumors FORUM

Re: DxO ... a little help please!
« Reply #21 on: February 13, 2014, 02:42:17 PM »

jrista

  • Canon EF 400mm f/2.8L IS II
  • *********
  • Posts: 4814
  • EOL
    • View Profile
    • Nature Photography
Re: DxO ... a little help please!
« Reply #22 on: February 13, 2014, 02:47:16 PM »
I forgot to mention the-digital-picture.com in my last post.  Their tools are nice to use for comparisons as well.
When I first started out with DSLRs, the only source I used to trust for reviews was the-digital-picture.com ... also, they make it very simple to understand.

Bryan's reviews at TDP are indeed excellent. I probably have more respect for his reviews than anyones. They are just strait forward, cover the key technical aspects but also demonstrate real-world usage. Really doesn't get better than that. Just wish he would test more brands. ;)

BTW, Rienz...I'd be happy to explain how to use an MTF. It seems complicated, all those lines, but once you get the general idea, they are actually EXTREMELY informative, and it isn't all that difficult to understand.

As for using DxO, @mackguyver pretty much nailed it. Use their measures, which are comparable, and ignore the scores. The measures are pretty decent (except transmission, that one is pretty useless because they don't account for differing apertures, so it really is an aperture measure, not a transmission measure.) DxO's lens measures are handy because they can be directly compared. If you want similar detailed information about lenses that can be manually compared, DPR lens reviews have similar information. Two windows side-by-side would let you compare lenses with DPR information.

Rienzphotoz

  • Canon EF 300mm f/2.8L IS II
  • ********
  • Posts: 3326
  • Peace unto all ye Canon, Nikon & Sony shooters
    • View Profile
Re: DxO ... a little help please!
« Reply #23 on: February 13, 2014, 02:52:09 PM »
BTW, Rienz...I'd be happy to explain how to use an MTF. It seems complicated, all those lines, but once you get the general idea, they are actually EXTREMELY informative, and it isn't all that difficult to understand.

Hi Jon,

Please do ... I'd appreciate that very much ... and am sure others like me will also appreciate it very much.

Thanks in advance
Canon 5DMK3 70D | Nikon D610 | Sony a7 a6000 | RX100M3 | 16-35/2.8LII | 70-200/2.8LISII | 100/2.8LIS | 100-400LIS | 40/2.8 | 50/1.4 | 85/1.8 | 600EX-RTx2 | ST-E3-RT | 24/3.5 T-S | 10-18/4 OSS 16-50 | 24-70/4OSS | 55/1.8 | 55-210 OSS | 70-200/4 OSS | 28-300VR | HVL-F43M | GoPro Black 3+ & DJI Phantom

lescrane

  • PowerShot G1 X II
  • ***
  • Posts: 66
    • View Profile
Re: DxO ... a little help please!
« Reply #24 on: February 13, 2014, 03:19:55 PM »
So since some of you are willing to explain wtf these charts mean to those of us who have never seen them, can you give a very basic explain. of this chart?
  Obviously the darker green the better, the more uniform the better, but what does the square area mean? Is it the frame of the image area? eg, darker in the middle is sharper, lighter around the edges, less sharp?
Yes, that it exactly right.  The rectangle is the sensor area and darker (green, vs. yellow or red) and more uniform is better :)

In this case, the Tamron puts in a pretty good showing considering the price difference.

thanks.  I compared it to the 100-400L in 70D crop body and the tammy looks pretty good at 400.

jrista

  • Canon EF 400mm f/2.8L IS II
  • *********
  • Posts: 4814
  • EOL
    • View Profile
    • Nature Photography
Re: DxO ... a little help please!
« Reply #25 on: February 13, 2014, 03:57:08 PM »
BTW, Rienz...I'd be happy to explain how to use an MTF. It seems complicated, all those lines, but once you get the general idea, they are actually EXTREMELY informative, and it isn't all that difficult to understand.

Hi Jon,

Please do ... I'd appreciate that very much ... and am sure others like me will also appreciate it very much.

Thanks in advance

Alright. Here goes. First, I'm just going to cover MTF charts. You don't actually need to know all the fundamental science that goes into resolving power to actually understand an MTF chart. All you really need to know is how to read the MTF chart, and that will tell you pretty much everything you need to know about the theoretical characteristics of a lens. Since most manufacturers publish MTF charts, and most use the same general standard (30lp/mm resolving power\sharpness & 10lp/mm contrast), they are pretty easy to compare brand to brand as well.

So, first off, an example MTF chart. This is the EF 24-70mm f/2.8 L II:




There are two charts here, because they represent the wide angle and narrow angle extremes of the zoom ratio. This one single chart tells you everything you need to know about a lens, and you can compare two MTF charts to each other to determine differences in each lenses performance (or, determine the differences between one extreme and the other of a zoom.)



So, first a breakdown of the chart itself. The MTF chart (absent any plot) represents the resolving power (sharpness and contrast) of a lens, from the center of the frame to the corner of the frame. The center of the frame is represented by the leftmost edge. The corner of the frame is represented by the rightmost edge. The vertical (y-axis) scale is an indication of how close to "ideal" resolving power gets. The vertical scale ranges from 0.0 to 1.0. In "historical" terms (and this really stems from the film days, so take it with the understanding that it definitely does not apply quite the same way today), MTF above 0.6 is "good/satisfactory" and MTF above 0.8 is "excellent/superior". Personally, I make the assumption, given how crisply modern digital sensors resolve detail, and the fact that people are increasingly publishing their photos online in full 1080p resolution, that MTF above 0.7 is "good/satisfactory" and above 0.9 is "excellent/superior".



Now for the plot. There are usually eight lines plotted on an MTF chart. These lines are blue and black (for Canon, other manufacturers may use other colors), thick and thin, solid and dashed. These lines represent how the lens reproduces sets of diagonal lines, the first set angled 45° and the second set 90° perpendicular to the first. These lines are called Sagittal and Meridional lines. They are angled at 45° and 135° because that is the ideal orientation to measure the behavior of optics from center to corner (vs. 0° and 90°, which would be more ideally suited to testing a lens center to edge...which is really insufficient.) The reason perpendicularly angled lines are used is because lenses do not behave the same when resolving detail at all angles...astigmatism in the lens design will often affect how lenses perform with fine detail angled differently across the lens. From the center to top right/bottom left corners, sagittal lines are angled parallel to the vector from the center to the corners of the lens. Meridional lines are angled perpendicular to the vector from center to top right/bottom left corners.

There are four sets of lines in total used in a standard MTF test. The first set are the 45° & perpendicular 10lp/mm lines. These lines are alternating white and black (technically speaking, the chart base is white, and there are thicker lines drawn at even spacing with thick white gaps between them, and thinner lines drawn at even spacing with thin white gaps between them.) The second set are 45° & perpendicular 30lp/mm lines. The 10lp/mm lines, which represent lower resolution detail, are used to measure lens contrast. The softer the transition between thick dark and thick white, the lower the contrast of the lens. The 30lp/mm lines, which represent higher resolution detail, are used to measure resolving power.



Can the black lines be resolved as fully separated with a white line in-between (resolved, high contrast)? If parallel black lines are separated, how quickly does the white line in-between become fully white (sharpness)?



So, we have an MTF chart that represents resolving power (y-axis) from center to corner of a lens (x-axis). This chart has eight curves plotted on it, that represent four sets of sagittal and meridional lines on a test chart. Why eight lines, rather than four? The MTF chart contains plots for both wide-open (max aperture) performance as well as f/8 performance. So, four sets of lines for max aperture:

  • f/2.8 sagittal thick solid (10lp/mm)
  • f/2.8 meridional thick dashed (10lp/mm)
  • f/2.8 sagittal thin solid (30lp/mm)
  • f/2.8 meridional thin dashed (30lp/mm)
  • f/8 sagittal thick solid (10lp/mm)
  • f/8 meridional thick dashed (10lp/mm)
  • f/8 sagittal thin solid (30lp/mm)
  • f/8 meridional thin dashed (30lp/mm)



Now, onto interpreting an MTF chart.

Using these eight curves plotted on an MTF chart, you can derive everything you need to know about the lens. The higher up the chart they are plotted, the better the contrast and resolution. You will normally see that as the curves move from the left edge to the right edge of the chart, they tend to "fall off", they drop lower. This is an indication of how the lens' performance chances from center to corner. You will also notice that one set of curves, either the solid curves or the dashed curves , tend to perform better than the other. This is an indication of astigmatism in the lens...sagittal curves (solid) may maintain higher performance than meridional curves (dashed).

This so happens to be exactly the case with the 24-70mm lens. At both wide (24mm) and narrow (70mm), the 24-70 resolves sagittal lines (45° angle) better than meridional lines (90° angle), and the meridional curves tend to fall off quicker in the midframe to the corners than sagittal curves do. Sometimes you may notice that some of these curves don't have a consistent falloff, they may turn "bump", resulting in slightly better resolving power just past midframe, then fall off again all the way into the corner. The nature of each curves falloff is an indication of how the various optical aberrations affect a lens' performance from center to corner. Depending on exactly what aberrations a lens may suffer from and to what degree, the nature of the curves' falloff will differ. Astigmatism and other imperfections in lens manufacture will result in falloff differing even from sagittal to meridional lines.

You will also find, once you start reading MTF charts, that wide angle lenses tend to have more chaotic late midframe and corner performance than normal and longer lenses. Telephoto/supertelephoto lenses will often have nearly flat MTF curves from center to corner (especially if they are higher end). It is not unusual to see crazy meridional performance in lenses as you approach the corners in wider angle lenses, especially ultrawide to wide angle zooms. There are simply certain compromises that must be made in order to produce a wide angle zoom lens that performs acceptably at as many focal lengths as possible.



That's pretty much it. A crash course in reading and understanding MTF charts. There is a lot more theory that builds up to WHY these kinds of tests are used to accurately measure lens resolving power, but you don't necessarily need to understand all that underlying grit in order to effectively use MTF charts themselves. All they really are is a plot of four types of detail from the center to the corner of the lens. Those plots measure contrast and resolution (sharpness & acutance) across the surface of the lens, and offer a fairly precise indication of how optical aberrations will affect your IQ in different regions of your frame.

candc

  • 5D Mark III
  • ******
  • Posts: 681
    • View Profile
Re: DxO ... a little help please!
« Reply #26 on: February 13, 2014, 04:00:57 PM »
are you interested in the sigma 120-300? dxo analysis is useful for comparing similar lenses on the same body. the dxo accutance map measures sharpness across the field as does an mtf chart but dxo measures actual performance of the lens, i think mtf charts are based on a model?
« Last Edit: February 13, 2014, 04:18:23 PM by candc »

mackguyver

  • Canon EF 300mm f/2.8L IS II
  • ********
  • Posts: 3048
  • Who Dares Wins
    • View Profile
    • My Personal Work
Re: DxO ... a little help please!
« Reply #27 on: February 13, 2014, 04:29:59 PM »
are you interested in the sigma 120-300? dxo analysis is useful for comparing similar lenses on the same body. the dxo accutance map measures sharpness across the field as does an mtf chart but dxo measures actual performance of the lens, i think mtf charts are based on a model?
Most manufacturers (other than Zeiss & Leica, I think, and maybe a few others) use theoretical MTFs generated by computer instead of actual measurements.  That doesn't make them invalid, but they aren't going to be as precise.

canon rumors FORUM

Re: DxO ... a little help please!
« Reply #27 on: February 13, 2014, 04:29:59 PM »

zim

  • 1D Mark IV
  • ******
  • Posts: 794
    • View Profile
Re: DxO ... a little help please!
« Reply #28 on: February 13, 2014, 04:32:44 PM »
Jrista, thank you for that excellent concise explanation.

candc

  • 5D Mark III
  • ******
  • Posts: 681
    • View Profile
Re: DxO ... a little help please!
« Reply #29 on: February 13, 2014, 04:49:16 PM »
that is a great explanation but i will never remember it all. is there a simple way to look at it to get a quick summary? such as the black line for a general indicator and how tight the lines are together? do these mtf charts generally translate into what is found in testing of real lenses?

thanks,

canon rumors FORUM

Re: DxO ... a little help please!
« Reply #29 on: February 13, 2014, 04:49:16 PM »