DxO & MTF Charts ... a little help please!

Re: DxO ... a little help please!

AlanF said:
Mac
Thanks for bringing up this method of comparison. I have just used it to compare the 150-600mm with the 100-400 in a thread about the Tammy to disprove an assertion that it is a confirmed soft lens.

DxO measurements are very useful for comparing different lenses on the same body or even better the same lens on different bodies.
No problem and I saw you feeding the troll on that post, LOL. It is a nice tool and you can compare up to three lenses at a time. As Neuro says, it's not perfect, but it's free and relatively easy to use. Also, I forgot to mention the-digital-picture.com in my last post. Their tools are nice to use for comparisons as well.
 
Upvote 0
Re: DxO ... a little help please!

mackguyver said:
I forgot to mention the-digital-picture.com in my last post. Their tools are nice to use for comparisons as well.
When I first started out with DSLRs, the only source I used to trust for reviews was the-digital-picture.com ... also, they make it very simple to understand.
 
Upvote 0
Re: DxO ... a little help please!

Rienzphotoz said:
mackguyver said:
I forgot to mention the-digital-picture.com in my last post. Their tools are nice to use for comparisons as well.
When I first started out with DSLRs, the only source I used to trust for reviews was the-digital-picture.com ... also, they make it very simple to understand.

Bryan's reviews at TDP are indeed excellent. I probably have more respect for his reviews than anyones. They are just strait forward, cover the key technical aspects but also demonstrate real-world usage. Really doesn't get better than that. Just wish he would test more brands. ;)

BTW, Rienz...I'd be happy to explain how to use an MTF. It seems complicated, all those lines, but once you get the general idea, they are actually EXTREMELY informative, and it isn't all that difficult to understand.

As for using DxO, @mackguyver pretty much nailed it. Use their measures, which are comparable, and ignore the scores. The measures are pretty decent (except transmission, that one is pretty useless because they don't account for differing apertures, so it really is an aperture measure, not a transmission measure.) DxO's lens measures are handy because they can be directly compared. If you want similar detailed information about lenses that can be manually compared, DPR lens reviews have similar information. Two windows side-by-side would let you compare lenses with DPR information.
 
Upvote 0
Re: DxO ... a little help please!

jrista said:
BTW, Rienz...I'd be happy to explain how to use an MTF. It seems complicated, all those lines, but once you get the general idea, they are actually EXTREMELY informative, and it isn't all that difficult to understand.
Hi Jon,

Please do ... I'd appreciate that very much ... and am sure others like me will also appreciate it very much.

Thanks in advance
 
Upvote 0
Re: DxO ... a little help please!

mackguyver said:
lescrane said:
So since some of you are willing to explain wtf these charts mean to those of us who have never seen them, can you give a very basic explain. of this chart?
Obviously the darker green the better, the more uniform the better, but what does the square area mean? Is it the frame of the image area? eg, darker in the middle is sharper, lighter around the edges, less sharp?
Yes, that it exactly right. The rectangle is the sensor area and darker (green, vs. yellow or red) and more uniform is better :)

In this case, the Tamron puts in a pretty good showing considering the price difference.

thanks. I compared it to the 100-400L in 70D crop body and the tammy looks pretty good at 400.
 
Upvote 0
Re: DxO ... a little help please!

Rienzphotoz said:
jrista said:
BTW, Rienz...I'd be happy to explain how to use an MTF. It seems complicated, all those lines, but once you get the general idea, they are actually EXTREMELY informative, and it isn't all that difficult to understand.
Hi Jon,

Please do ... I'd appreciate that very much ... and am sure others like me will also appreciate it very much.

Thanks in advance

Alright. Here goes. First, I'm just going to cover MTF charts. You don't actually need to know all the fundamental science that goes into resolving power to actually understand an MTF chart. All you really need to know is how to read the MTF chart, and that will tell you pretty much everything you need to know about the theoretical characteristics of a lens. Since most manufacturers publish MTF charts, and most use the same general standard (30lp/mm resolving power\sharpness & 10lp/mm contrast), they are pretty easy to compare brand to brand as well.

So, first off, an example MTF chart. This is the EF 24-70mm f/2.8 L II:

ef_24-70_wide_mtf.gif

ef_24-70_tele_mtf.gif


There are two charts here, because they represent the wide angle and narrow angle extremes of the zoom ratio. This one single chart tells you everything you need to know about a lens, and you can compare two MTF charts to each other to determine differences in each lenses performance (or, determine the differences between one extreme and the other of a zoom.)



So, first a breakdown of the chart itself. The MTF chart (absent any plot) represents the resolving power (sharpness and contrast) of a lens, from the center of the frame to the corner of the frame. The center of the frame is represented by the leftmost edge. The corner of the frame is represented by the rightmost edge. The vertical (y-axis) scale is an indication of how close to "ideal" resolving power gets. The vertical scale ranges from 0.0 to 1.0. In "historical" terms (and this really stems from the film days, so take it with the understanding that it definitely does not apply quite the same way today), MTF above 0.6 is "good/satisfactory" and MTF above 0.8 is "excellent/superior". Personally, I make the assumption, given how crisply modern digital sensors resolve detail, and the fact that people are increasingly publishing their photos online in full 1080p resolution, that MTF above 0.7 is "good/satisfactory" and above 0.9 is "excellent/superior".



Now for the plot. There are usually eight lines plotted on an MTF chart. These lines are blue and black (for Canon, other manufacturers may use other colors), thick and thin, solid and dashed. These lines represent how the lens reproduces sets of diagonal lines, the first set angled 45° and the second set 90° perpendicular to the first. These lines are called Sagittal and Meridional lines. They are angled at 45° and 135° because that is the ideal orientation to measure the behavior of optics from center to corner (vs. 0° and 90°, which would be more ideally suited to testing a lens center to edge...which is really insufficient.) The reason perpendicularly angled lines are used is because lenses do not behave the same when resolving detail at all angles...astigmatism in the lens design will often affect how lenses perform with fine detail angled differently across the lens. From the center to top right/bottom left corners, sagittal lines are angled parallel to the vector from the center to the corners of the lens. Meridional lines are angled perpendicular to the vector from center to top right/bottom left corners.

There are four sets of lines in total used in a standard MTF test. The first set are the 45° & perpendicular 10lp/mm lines. These lines are alternating white and black (technically speaking, the chart base is white, and there are thicker lines drawn at even spacing with thick white gaps between them, and thinner lines drawn at even spacing with thin white gaps between them.) The second set are 45° & perpendicular 30lp/mm lines. The 10lp/mm lines, which represent lower resolution detail, are used to measure lens contrast. The softer the transition between thick dark and thick white, the lower the contrast of the lens. The 30lp/mm lines, which represent higher resolution detail, are used to measure resolving power.

153D62414D8D7BE822745E


Can the black lines be resolved as fully separated with a white line in-between (resolved, high contrast)? If parallel black lines are separated, how quickly does the white line in-between become fully white (sharpness)?



So, we have an MTF chart that represents resolving power (y-axis) from center to corner of a lens (x-axis). This chart has eight curves plotted on it, that represent four sets of sagittal and meridional lines on a test chart. Why eight lines, rather than four? The MTF chart contains plots for both wide-open (max aperture) performance as well as f/8 performance. So, four sets of lines for max aperture:

[list type=decimal]
[*]f/2.8 sagittal thick solid (10lp/mm)
[*]f/2.8 meridional thick dashed (10lp/mm)
[*]f/2.8 sagittal thin solid (30lp/mm)
[*]f/2.8 meridional thin dashed (30lp/mm)
[*]f/8 sagittal thick solid (10lp/mm)
[*]f/8 meridional thick dashed (10lp/mm)
[*]f/8 sagittal thin solid (30lp/mm)
[*]f/8 meridional thin dashed (30lp/mm)
[/list]



Now, onto interpreting an MTF chart.

Using these eight curves plotted on an MTF chart, you can derive everything you need to know about the lens. The higher up the chart they are plotted, the better the contrast and resolution. You will normally see that as the curves move from the left edge to the right edge of the chart, they tend to "fall off", they drop lower. This is an indication of how the lens' performance chances from center to corner. You will also notice that one set of curves, either the solid curves or the dashed curves , tend to perform better than the other. This is an indication of astigmatism in the lens...sagittal curves (solid) may maintain higher performance than meridional curves (dashed).

This so happens to be exactly the case with the 24-70mm lens. At both wide (24mm) and narrow (70mm), the 24-70 resolves sagittal lines (45° angle) better than meridional lines (90° angle), and the meridional curves tend to fall off quicker in the midframe to the corners than sagittal curves do. Sometimes you may notice that some of these curves don't have a consistent falloff, they may turn "bump", resulting in slightly better resolving power just past midframe, then fall off again all the way into the corner. The nature of each curves falloff is an indication of how the various optical aberrations affect a lens' performance from center to corner. Depending on exactly what aberrations a lens may suffer from and to what degree, the nature of the curves' falloff will differ. Astigmatism and other imperfections in lens manufacture will result in falloff differing even from sagittal to meridional lines.

You will also find, once you start reading MTF charts, that wide angle lenses tend to have more chaotic late midframe and corner performance than normal and longer lenses. Telephoto/supertelephoto lenses will often have nearly flat MTF curves from center to corner (especially if they are higher end). It is not unusual to see crazy meridional performance in lenses as you approach the corners in wider angle lenses, especially ultrawide to wide angle zooms. There are simply certain compromises that must be made in order to produce a wide angle zoom lens that performs acceptably at as many focal lengths as possible.



That's pretty much it. A crash course in reading and understanding MTF charts. There is a lot more theory that builds up to WHY these kinds of tests are used to accurately measure lens resolving power, but you don't necessarily need to understand all that underlying grit in order to effectively use MTF charts themselves. All they really are is a plot of four types of detail from the center to the corner of the lens. Those plots measure contrast and resolution (sharpness & acutance) across the surface of the lens, and offer a fairly precise indication of how optical aberrations will affect your IQ in different regions of your frame.
 
Upvote 0
Re: DxO ... a little help please!

are you interested in the sigma 120-300? dxo analysis is useful for comparing similar lenses on the same body. the dxo accutance map measures sharpness across the field as does an mtf chart but dxo measures actual performance of the lens, i think mtf charts are based on a model?
 
Upvote 0
Re: DxO ... a little help please!

candc said:
are you interested in the sigma 120-300? dxo analysis is useful for comparing similar lenses on the same body. the dxo accutance map measures sharpness across the field as does an mtf chart but dxo measures actual performance of the lens, i think mtf charts are based on a model?
Most manufacturers (other than Zeiss & Leica, I think, and maybe a few others) use theoretical MTFs generated by computer instead of actual measurements. That doesn't make them invalid, but they aren't going to be as precise.
 
Upvote 0
Re: DxO ... a little help please!

that is a great explanation but i will never remember it all. is there a simple way to look at it to get a quick summary? such as the black line for a general indicator and how tight the lines are together? do these mtf charts generally translate into what is found in testing of real lenses?

thanks,
 
Upvote 0
Re: DxO ... a little help please!

candc said:
that is a great explanation but i will never remember it all. is there a simple way to look at it to get a quick summary? such as the black line for a general indicator and how tight the lines are together? do these mtf charts generally translate into what is found in testing of real lenses?

thanks,

You can't really narrow it down. That's the fundamental fallacy everyone falls into, and why a lot of the stuff DXO, DPR, and the whole host of other "lens testers" produce is largely useless.

If you want to eliminate anything, eliminate the set of curves that don't fit with the apertures you will most use. If you will mostly shoot wide open, then you can ignore the blue curves. If you will mostly shoot stopped down, then you can ignore the black curves. You could ignore dashed curves, and just use the solid curves, but then your not really getting the whole picture.

There is no "one number tells all". That's just a fallacy. The attempt to utterly simplify everything is really what gets you into trouble.

When it comes to the difference between an MTF and a lens test, lens tests all ultimately run into the "sensor bound" problem. Sensors have a fixed resolving power...it's the same across the entire area of the sensor. Lenses, on the other hand, have a non-linear resolving power that falls off as you stop down. At apertures wider than f/8, the potential for a lens to resolve much finer detail than the sensor becomes very real. The problem is, final "output" resolution has an asymptotic relationship with the lowest common denominator. Since the sensor usually IS that lowest common denominator, at faster apertures, where lenses have the potential to resolve a LOT of detail, may all end up looking the same in the end. Why? Well, let's say your sensor can resolve 50lp/mm, and you have four lenses capable of resolving, at f/4, 100lp/mm, 130lp/mm, 150lp/mm, and 173lp/mm (the latter is the maximum diffraction-limited resolving power of an f/4 lens.) The problem is that all of these lenses will all appear to resolve somewhere between 45-49lp/mm with a "real world" lens test, like the kind that DXO does. They are all SENSOR BOUND! The SENSOR cannot resolve more than 50lp/mm, so that is your absolute limit on final output resolution (the resolution measured in the RAW images by computer algorithms.)

So, first off, your standard lens test that tests lenses attached to cameras are largely useless for any apertures above f/8, however from f/8 and narrower, the vast majority of lenses are diffraction limited, so they will all perform the same anyway.

Second, MTFs really don't have any relationship with artificial lens test results, because they are either performed algorithmically based on fairly accurate computer models that account for overall lens construction and design, as well as material traits; or they are performed with optical lens bench testing, which uses a special apparatus to test JUST the lens. Synthetic MTFs will usually indicate just a little bit better performance than Real MTFs generated with a optical test bench, however both will be largely similar, and neither will bear any resemblance to your "standard lens+camera" tests.

If you want to keep it simple: Pick one set of solid lines, for max aperture or f/8 (depending on whichever you use most), and go with that.
 
Upvote 0
Re: DxO ... a little help please!

ok, i am getting closer i think, bold lines on chart = bold lines in testing for contrast, fine lines = resolution. black is wide open, blue is f/8 solid lines are radians, dashed are perpendicular, i am a bit fuzzy on what the radian and perpendicular lines tell you but i am getting the idea,

thanks
 
Upvote 0
Re: DxO ... a little help please!

candc said:
ok, i am getting closer i think, bold lines on chart = bold lines in testing for contrast, fine lines = resolution. black is wide open, blue is f/8 solid lines are radians, dashed are perpendicular, i am a bit fuzzy on what the radian and perpendicular lines tell you but i am getting the idea,

thanks

Sagittal (radian) and meridional (perpendicular) tell you about lens astigmatism (they will usually diverge when plotted across the chart).
 
Upvote 0
Re: DxO ... a little help please!

i am going to have to read up on that, i hear about astigmatism in a lens all the time but don't fully understand what it is and how it translates to a lens performance. i know it is an unwanted characteristic in a lens and now i think i can see from the mtf chart that the closer together the dashed and solid lines of a particular pair are then the better the astigmatism is controlled?
 
Upvote 0
Re: DxO ... a little help please!

privatebydesign said:

those are good articles thanks, i am going to have to read through them again a few times but what grabbed me is that you should look at the uppermost set of lines, the bold ones because they translate best to what you visually correlate with lens performance? that is to say that the contrast seems to be more important than absolute resolving power. i noticed that on the mtf chart for the 24-70ii, the bold lines were flat at the top and the fine lines dropped off significantly more.
 
Upvote 0
Re: DxO ... a little help please!

candc said:
privatebydesign said:

those are good articles thanks, i am going to have to read through them again a few times but what grabbed me is that you should look at the uppermost set of lines, the bold ones because they translate best to what you visually correlate with lens performance? that is to say that the contrast seems to be more important than absolute resolving power. i noticed that on the mtf chart for the 24-70ii, the bold lines were flat at the top and the fine lines dropped off significantly more.

It is easy to get absolute in an online article. I wouldn't say things are as simple as "contrast is more important than absolute resolving power". It depends. As a bird photographer, absolute resolving power is very important to me, so I pay close attention to the thin lines myself. Some lenses have great contrast with lower resolution detail, but they simply can't separate finer details. That would ultimately mean that I might be able to resolve the major details of a birds feathers, but not actually be able to cleanly separate the structure of the feathers themselves. I'd be ok with a slight loss of contrast for larger details, if it means I have the ability to separate finer detail.

You can reduce the information in an MTF according to your needs. Don't let someone else tell you what your needs are, though. ;) If contrast is more important to you, well then sure, pay attention to the thicker solid lines.
 
Upvote 0
Re: DxO ... a little help please!

jrista said:
candc said:
privatebydesign said:

those are good articles thanks, i am going to have to read through them again a few times but what grabbed me is that you should look at the uppermost set of lines, the bold ones because they translate best to what you visually correlate with lens performance? that is to say that the contrast seems to be more important than absolute resolving power. i noticed that on the mtf chart for the 24-70ii, the bold lines were flat at the top and the fine lines dropped off significantly more.

It is easy to get absolute in an online article. I wouldn't say things are as simple as "contrast is more important than absolute resolving power". It depends. As a bird photographer, absolute resolving power is very important to me, so I pay close attention to the thin lines myself. Some lenses have great contrast with lower resolution detail, but they simply can't separate finer details. That would ultimately mean that I might be able to resolve the major details of a birds feathers, but not actually be able to cleanly separate the structure of the feathers themselves. I'd be ok with a slight loss of contrast for larger details, if it means I have the ability to separate finer detail.

You can reduce the information in an MTF according to your needs. Don't let someone else tell you what your needs are, though. ;) If contrast is more important to you, well then sure, pay attention to the thicker solid lines.

That's the thing I am trying to get straight because its a bit confusing. What the mtf measures is micro contrast which we see as resolution at some point, not overall image contrast. The mtf chart for the 24-70ii shows really good contrast center to edge but the resolution lines don't look that impressive. Its known to be a super sharp lens by all accounts so that seems to reinforce the point that what we see as resolution is really closely related to contrast?
 
Upvote 0
Re: DxO ... a little help please!

candc said:
jrista said:
candc said:
privatebydesign said:

those are good articles thanks, i am going to have to read through them again a few times but what grabbed me is that you should look at the uppermost set of lines, the bold ones because they translate best to what you visually correlate with lens performance? that is to say that the contrast seems to be more important than absolute resolving power. i noticed that on the mtf chart for the 24-70ii, the bold lines were flat at the top and the fine lines dropped off significantly more.

It is easy to get absolute in an online article. I wouldn't say things are as simple as "contrast is more important than absolute resolving power". It depends. As a bird photographer, absolute resolving power is very important to me, so I pay close attention to the thin lines myself. Some lenses have great contrast with lower resolution detail, but they simply can't separate finer details. That would ultimately mean that I might be able to resolve the major details of a birds feathers, but not actually be able to cleanly separate the structure of the feathers themselves. I'd be ok with a slight loss of contrast for larger details, if it means I have the ability to separate finer detail.

You can reduce the information in an MTF according to your needs. Don't let someone else tell you what your needs are, though. ;) If contrast is more important to you, well then sure, pay attention to the thicker solid lines.

That's the thing I am trying to get straight because its a bit confusing. What the mtf measures is micro contrast which we see as resolution at some point, not overall image contrast. The mtf chart for the 24-70ii shows really good contrast center to edge but the resolution lines don't look that impressive. Its known to be a super sharp lens by all accounts so that seems to reinforce the point that what we see as resolution is really closely related to contrast?

An MTF measures both microcontrast as well as larger-scale contrast. The 30lp/mm is more of your "microcontrast" (sharpness and acutance), where as 10lp/mm is general contrast. That's why both sets of thin and thick line pairs are used.

The 24-70 is an excellent lens...but it IS both wider angle and it is a zoom. Both of those facts require that certain compromises be made. Compare the 24-70 II with the previous, or with third-party lenses in the same general range, and you'll see why it is considered a very high quality lens with high resolving power. It's got very good resolution for the focal length bracket it falls into.

You want to see some REALLY high resolution lenses? Here:

EF 300mm f/2.8 L II:
mtf.jpg


EF 600mm f/4 L II:
ef600lisiiu_mtf.gif


As you get to longer focal lengths, the falloff towards the corner drops. More light is collimated, you don't have to worry about bending highly oblique light. The more you bend light, the tougher it is to maintain resolution from center to corner.

Even the older EF 300 f/2.8 L Mark I was an unbelievably stellar lens:

ef_300_28mtf.gif


But again, these are all primes, and they are telephotos. Mostly collimated light, no highly oblique incident light angles to deal with, and they use the highest quality optical glass around.

Also, keep in mind that you have blue and black lines, meaning f/8 and max aperture. Depending on what the maximum aperture is, the black lines may drop considerably due to optical aberrations. If you want to look at diffraction limited performance, use the blue lines. That is f/8, and it is easier to compare lenses at f/8, as more likely than not most lenses you compare will be diffraction limited, or close to it, at that aperture. You'll notice that f/8 resolution lines tend to edge higher up the chart.

Another example would be the 70-200 f/2.8 L II. At 70mm and 200mm:

ef70-200lisiiu_wide_mtf.gif


ef70-200lisiiu_tele_mtf.gif


Notice the differences here. At 70mm, a shorter focal length that has to deal with more oblique off-axis light, has greater falloff in the corners, and struggles more with meridional test lines (especially the high resolution ones). However at 200mm, you can see the whole set of lines has moved up the chart, and that faloff to corner is slower and less dramatic.

This is the power of an MTF chart. They tell you a LOT about optics, and generally at a glance (once you know how to read them.) Learning how to read them just takes some time and practice. Eventually, comparing lenses or focal lengths really boils down to some momentary glances at the charts, and you absorb a whole lot of information all at once. (At least...that's how it works with me...)
 
Upvote 0