Question about TDP's sample crops

[insanitybeard]

Hopefully one of you jolly fine fellows can answer a question that has been bugging me for a while!?

When using The Digital Pictures' sample crops to compare lens performance against other lenses or the same lens used on a crop and full frame body, for example, it astounds me how soft the crop body rendition is in comparison. The below link illustrates my question:

http://www.the-digital-picture.com/Reviews/ISO-12233-Sample-Crops.aspx?Lens=687&Camera=453&Sample=0&FLI=3&API=3&LensComp=687&CameraComp=736&SampleComp=0&FLIComp=3&APIComp=3

The 70-200 2.8L IS II is obviously one of Canon's legendary lenses, but why does the crop crop (crop body crop!)- which is a 60D in this case- look so much softer than the full frame 1Ds III? Is it just that the 60D has a very soft output? I chose f5.6 as the aperture for both bodies so there should be no diffraction issues but the comparison is much the same at other apertures anyway. Even more surprising in the above comparison is how soft the corners are on the 60D crops compared to the 1Ds III, which surprises me because the crop body is not using the lens's full image circle. The first time I used TDP's comparison was with ultrawide zooms so I just figured that it was the fact that the full frame ultra wides were performing better than a crop equivalent (think EF-S 10-22 on crop compared against 17-40L on full frame, but when I started looking at other lenses I saw it was pretty much always the same- the FF crops were visibly sharper with the same lens. Any thoughts?

 

[Loswr]

You're not the first one to ask this...

http://www.canonrumors.com/forum/index.php?topic=10289.0

(Side note - if/when you get to AlanF's feather shots in that topic, he describes the theoretical reasons why the 7D will outresolve a 5DIII, based on his shots with just a 7D. A few months later, he got himself a 5DIII and came to the conclusion that practical reality trumps theoretical comparison, and that in the real world, the 7D is not sharper. )

 

[insanitybeard]

Thanks neuro, so the primary factor is the camera to subject distance, I did wonder if that played a part. I have the switch to full frame planned in the long term but it's a way off yet!

 

[qwerty]

Thanks neuro, so the primary factor is the camera to subject distance, I did wonder if that played a part. I have the switch to full frame planned in the long term but it's a way off yet!

Its not the distance from the image to the camera, its the fact that the crop image (as projected onto the sensor) is enlarged by a greater factor to get the same sized image on your monitor. Neuro's comment in the other thread was that if you adjusted the distance such that the projected images on the sensors was the same size, then FF would not have the same advantage (because, to get the same on-screen image, you would have to crop the FF image, then enlarge it by the same amount as the crop image -- in that case the higher pixel density on the APS-C camera could give a higher resolution final image). As with most questions comparing different crop factors, it comes down to a question of what you hold constant and what you vary in your hypotheticals.



As a quick example, compare the 200 f/2 (or 300 2.8 ii or whatever other lens of a similar generation) on a crop body (you can stop down a notch or two) to the 500 f/4 on full frame. The 500 f/4 is be further away from the subject, but will still yield a much sharper image when enlarged to a fixed size.

I posted a similar response at http://www.canonrumors.com/forum/index.php?topic=14089.msg255114#msg255114



When it comes to noise or resolution for a given number of pixels, there is no replacement for displacement sensor size.

 

[AlanF]

You're not the first one to ask this...

http://www.canonrumors.com/forum/index.php?topic=10289.0

(Side note - if/when you get to AlanF's feather shots in that topic, he describes the theoretical reasons why the 7D will outresolve a 5DIII, based on his shots with just a 7D. A few months later, he got himself a 5DIII and came to the conclusion that practical reality trumps theoretical comparison, and that in the real world, the 7D is not sharper. )

Neuro
That is not quite accurate. For much of the time the 5DIII does outperform the 7D because of its better IQ. But, there are occasions when the 7D beats the 5DIII with the same lens - for example, taking photos of the moon; also between critical distances when the 5DIII cannot resolve closely spaced lines but the 7D can.

The TDP comparisons are unfair when comparing crop sensors with FF. TDP displays an image that exactly fills the sensor. So, the camera is at variable distances away, depending on the focal length of the lens and the crop factor. For example, the crop sensor camera has to be 1.6x further away than the FF with the same lens to fill its sensor with the same image. So, the crop is at a disadvantage. A fairer comparison in many cases is to have the crop and FF exactly the same distance away from the target, which is the way the cameras are used for bird photography etc. Here, the image will be the same size on both the crop and the FF sensors, and the advantages of the crop camera's smaller pixels in terms of resolution will be seen as well as negating some of its poorer noise disadvantage since both images occupy the same area.

The same is true when comparing lenses of different focal lengths.

 

[insanitybeard]

Its not the distance from the image to the camera, its the fact that the crop image (as projected onto the sensor) is enlarged by a greater factor to get the same sized image on your monitor. Neuro's comment in the other thread was that if you adjusted the distance such that the projected images on the sensors was the same size, then FF would not have the same advantage (because, to get the same on-screen image, you would have to crop the FF image, then enlarge it by the same amount as the crop image -- in that case the higher pixel density on the APS-C camera could give a higher resolution final image). As with most questions comparing different crop factors, it comes down to a question of what you hold constant and what you vary in your hypotheticals.



As a quick example, compare the 200 f/2 (or 300 2.8 ii or whatever other lens of a similar generation) on a crop body (you can stop down a notch or two) to the 500 f/4 on full frame. The 500 f/4 is be further away from the subject, but will still yield a much sharper image when enlarged to a fixed size.

I posted a similar response at http://www.canonrumors.com/forum/index.php?topic=14089.msg255114#msg255114



When it comes to noise or resolution for a given number of pixels, there is no replacement for displacement sensor size.

Qwerty, thanks for your reply, what I am struggling to grasp based upon what you have said, is does that extra enlargement really add that much softness? And why, specifically in the example I used, is the 60D corner crop so much softer (relatively) than the 1Ds III corner crop, (The 60D centre crop is soft compared to the full frame, but the 60D corner crop is even worse) when the 60D is using more of the centre part of the lens' image circle? Surely less corner degradation should be apparent?

 

[qwerty]

Its not the distance from the image to the camera, its the fact that the crop image (as projected onto the sensor) is enlarged by a greater factor to get the same sized image on your monitor. Neuro's comment in the other thread was that if you adjusted the distance such that the projected images on the sensors was the same size, then FF would not have the same advantage (because, to get the same on-screen image, you would have to crop the FF image, then enlarge it by the same amount as the crop image -- in that case the higher pixel density on the APS-C camera could give a higher resolution final image). As with most questions comparing different crop factors, it comes down to a question of what you hold constant and what you vary in your hypotheticals.



As a quick example, compare the 200 f/2 (or 300 2.8 ii or whatever other lens of a similar generation) on a crop body (you can stop down a notch or two) to the 500 f/4 on full frame. The 500 f/4 is be further away from the subject, but will still yield a much sharper image when enlarged to a fixed size.

I posted a similar response at http://www.canonrumors.com/forum/index.php?topic=14089.msg255114#msg255114



When it comes to noise or resolution for a given number of pixels, there is no replacement for displacement sensor size.

Qwerty, thanks for your reply, what I am struggling to grasp based upon what you have said, is does that extra enlargement really add that much softness? And why, specifically in the example I used, is the 60D corner crop so much softer (relatively) than the 1Ds III corner crop, (The 60D centre crop is soft compared to the full frame, but the 60D corner crop is even worse) when the 60D is using more of the centre part of the lens' image circle? Surely less corner degradation should be apparent?

I don't have a good answer for that... you usually expect the corners to be better relative to the center on a crop camera (with the same lens), as you say. I have some ideas, but they are basically speculation at this point.

You can also see Canon's MTF charts here http://www.usa.canon.com/cusa/consumer/products/cameras/ef_lens_lineup/ef_70_200mm_f_2_8l_is_ii_usm
APS-C corners are around 15mm, FF corners are around 25mm. Note that Canon's MTF chart shows you the contrast in the image projected on the sensor for a fixed number of lines per mm. It does not tell you much about how this changes with the line pitch.

 

[LetTheRightLensIn]

1. 60D has 18MP on a surface area 2.56x smaller than the 1Ds3 fits 21MP. So it stresses lenses more.

2. It is possible the 60D has split greens and thus comes out a touch softer 'per pixel' after de-bayer than the 1Ds3

3. It is possible that the AA filter on the 60D is tuned stronger

4. It could demonstrate how tricky lens tests are to carry out well. I suspect that TDP only focuses at the center and doesn't refocus for corners or edges so just a trace mis-alignment, like a paper thin one, and.... because it is a little curious that the far frame samples on APS-C look so soft in some cases from such sharp lenses even taking into account #1,2 and 3. (I have to say that even center frame my own ultra-careful tests have often disagreed with TDP, especially in the past (originally they actually relied on AF for their test samples and even well into the liveview era). More results, while certainly not always the same, tend to be closer to photozone.de on average. Copy to copy variation also certainly plays some role here.)

EDIT: as AlanF says the cop camera is also farther away which might slightly hurt it.

 

[LetTheRightLensIn]

A few months later, he got himself a 5DIII and came to the conclusion that practical reality trumps theoretical comparison, and that in the real world, the 7D is not sharper. )

In what scenario?? The 7D easily pulls in more detail than the 5D3 when reach limited. I have both extremely carefully carried out test shots and real world bird shots demonstrating that. Romy has such demonstrating the same vs the 5D2. (With a high-contrast, near black near white subject that is well lit even at ISO6400 the 7D pulls way more detail than 5D2 or 5D3, of course that is a bit of an ideal ISO6400 scenario as not all subjects will be mostly white with fine black lines and in the most brightly lit part of the frame.)

 

[AlanF]

A few months later, he got himself a 5DIII and came to the conclusion that practical reality trumps theoretical comparison, and that in the real world, the 7D is not sharper. )

In what scenario?? The 7D easily pulls in more detail than the 5D3 when reach limited. I have both extremely carefully carried out test shots and real world bird shots demonstrating that. Romy has such demonstrating the same vs the 5D2. (With a high-contrast, near black near white subject that is well lit even at ISO6400 the 7D pulls way more detail than 5D2 or 5D3, of course that is a bit of an ideal ISO6400 scenario as not all subjects will be mostly white with fine black lines and in the most brightly lit part of the frame.)

See my reply to Neuro (reply #4 - it helps to read the whole thread). I agree absolutely with what you say. The 7D does indeed pull in more detail when the subject is reach limited. I also showed earlier that the 7D sensor does separate two lines which are two pixels apart on its sensor, and it has smaller pixels than the 5D III. The 5D III performs better outside of those situations. I have had to repeat this dual scenario several times in different threads. The deciding difference between the 5D III and the 7D for me is consistency of AF. The 5D is spot on just about every time whereas for the 7D I have to take several shots to make sure one is tack sharp. The better AF is documented both by LensRental testing and the charts on the FoCal site, so the difference is not restricted to my copies of the 5D III and 7D.

 

[heptagon]

I suspect that TDP only focuses at the center and doesn't refocus for corners or edges so just a trace mis-alignment, like a paper thin one, and....

As far as I know they do separately focus for every spot, repeat the test more than once and take the best result for each.

Besides thin DoF there is field curvature on a few lenses which makes it impossible to get a flat screen in focus.

 

[insanitybeard]

Thanks for your thoughts LTRLI and AlanF, I didn't think about focus issues (but then I was assuming something similar to Photozone's approach where they refocus for the corners if required for the sharpest result). I agree with focus consistency on the 7D not being great, my 7D has struggled to find accurate focus at all some times with the 70-200 f4 L, and that was aimed at targets which (IMO) had good contrast, such as a swan against a dark background, it took several attempts to get decent focus- and by that I don't mean it was a little way out.

 

[insanitybeard]

I suspect that TDP only focuses at the center and doesn't refocus for corners or edges so just a trace mis-alignment, like a paper thin one, and....

As far as I know they do separately focus for every spot, repeat the test more than once and take the best result for each.

Besides thin DoF there is field curvature on a few lenses which makes it impossible to get a flat screen in focus.

Good point, but I don't think the 70-200 IS II is a lens that suffers with field curvature. As you say, TDP must refocus if required for the corners and use the best result from several tests.

 

[Loswr]

A few months later, he got himself a 5DIII and came to the conclusion that practical reality trumps theoretical comparison, and that in the real world, the 7D is not sharper. )

In what scenario?? The 7D easily pulls in more detail than the 5D3 when reach limited. I have both extremely carefully carried out test shots and real world bird shots demonstrating that.

Yes, with well-lit subjects, preferably not moving, and if the 7D's AF doesn't miss, and as long as you don't have to trade away sharpness for noise reduction.

While the 7D can deliver slightly more detail (especially in theory), in practice my 1D X usually does deliver more detail.

 

[privatebydesign]

I did a series of tests with a 7D and a 1Ds MkIII (with slightly less resolution than a 5D MkIII), I did "real world" tests and "bench" tests, that I have shown before, so sorry to the regulars.

Even when the test situation was set up to favour the crop sensor in a focal length limited situation with base iso, heavy tripod, cable release, 10X Live View manual focus, flash illumination, optimum aperture etc etc I still found there to be very little difference between the two same size crops. Bare in mind these ideal test situations are so divorced from the reality of actual shooting situations (unless you only do still lifes) it is comical, in the "real world tests there is zero resolution difference in focal length limited situations. Well those were my findings after actually testing it all myself to my own satisfaction, which was a shame because I wanted to buy a 7D but realised there was no advantage to me doing so.

Here are the bench tests, first image to the left, complete ff image with crop camera image overlaid in red box, images on the right are both cameras cropped, 7D to 100% and 1Ds MkIII upscaled to match the pixel numbers. You can argue the resize but that is how I would use the cameras so that is what I did.

Second image, same setup but a human hair that i happened to notice, the 7D is around 200% enlargement and the 1Ds MkIII is well over 300%.

Now some have said there is a big difference in these two, I can't see it, the 7D does have a slight resolution advantage, but nothing much, and these are ideal bench test situations set up to show the biggest difference. Introduce AF into the equation, shoot higher than base iso, less than ideal light, handhold etc etc, and what small differences there are disappear.

 

[LetTheRightLensIn]

A few months later, he got himself a 5DIII and came to the conclusion that practical reality trumps theoretical comparison, and that in the real world, the 7D is not sharper. )

In what scenario?? The 7D easily pulls in more detail than the 5D3 when reach limited. I have both extremely carefully carried out test shots and real world bird shots demonstrating that. Romy has such demonstrating the same vs the 5D2. (With a high-contrast, near black near white subject that is well lit even at ISO6400 the 7D pulls way more detail than 5D2 or 5D3, of course that is a bit of an ideal ISO6400 scenario as not all subjects will be mostly white with fine black lines and in the most brightly lit part of the frame.)

See my reply to Neuro (reply #4 - it helps to read the whole thread). I agree absolutely with what you say. The 7D does indeed pull in more detail when the subject is reach limited. I also showed earlier that the 7D sensor does separate two lines which are two pixels apart on its sensor, and it has smaller pixels than the 5D III. The 5D III performs better outside of those situations. I have had to repeat this dual scenario several times in different threads. The deciding difference between the 5D III and the 7D for me is consistency of AF. The 5D is spot on just about every time whereas for the 7D I have to take several shots to make sure one is tack sharp. The better AF is documented both by LensRental testing and the charts on the FoCal site, so the difference is not restricted to my copies of the 5D III and 7D.

yeah the 5D3 AF is more precise, especially if you also pair it with a new lens with the higher precision AF braking slop measurement sensor but even if not (for near stationary objects the 7D AI Servo actually seems to be better though and for picking out birds in branches and it seems about the same for surfing otherwise it's definitely not as good in terms of precision for one shots stills or for tracking things like soccer/football).

 

[LetTheRightLensIn]

I suspect that TDP only focuses at the center and doesn't refocus for corners or edges so just a trace mis-alignment, like a paper thin one, and....

As far as I know they do separately focus for every spot, repeat the test more than once and take the best result for each.

Besides thin DoF there is field curvature on a few lenses which makes it impossible to get a flat screen in focus.

FC wouldn't matter if they refocus for edges

 

[LetTheRightLensIn]

A few months later, he got himself a 5DIII and came to the conclusion that practical reality trumps theoretical comparison, and that in the real world, the 7D is not sharper. )

In what scenario?? The 7D easily pulls in more detail than the 5D3 when reach limited. I have both extremely carefully carried out test shots and real world bird shots demonstrating that.

Yes, with well-lit subjects, preferably not moving, and if the 7D's AF doesn't miss, and as long as you don't have to trade away sharpness for noise reduction.

While the 7D can deliver slightly more detail (especially in theory), in practice my 1D X usually does deliver more detail.

In the field, in real practice, my 7D usually delivered a lot more detail than my 5D2 and a decent bit more than my 5D3 (and I say this as someone who sold the 7D to get the 5D3 not as some 7D owner who couldn't get the 7D and is trying to justify it away). The one thing I really, in real world practice, miss my 7D for is the reach. I simply did get more detail on most bird shots. (and compared to the 5D2 it did better at any ISO since it had better SNR per sensor area and the over-sampling meant you could the same details, with a trace better micro-contrast, on a bird with less debayer artifacts if you were to filter and downsample to 5D2 scale or put the 7D print farther away or print them both to the same size paper, etc.).

 

[LetTheRightLensIn]

I did a series of tests with a 7D and a 1Ds MkIII (with slightly less resolution than a 5D MkIII), I did "real world" tests and "bench" tests, that I have shown before, so sorry to the regulars.

I got much different results (as did Romy), the reach advantage looks real enough to me (the ISO6400 sample is admittedly a very ideal scenario though and the subject doesn't present noise well):
same lens, same target, same lighting, same distance to target, same settings
7D on top, then 5D2 then 5D3 then the 7D pasted onto the bottom again

ACR RAW ISO100:


SOOC JPGS ISO100:


RAW ISO6400 (top is 7D downscaled to 5D size, then 5D2, then 5D3, then 7D full size):

 

[Dianoda]

Something else to consider - the 60D/7D sensors have stronger AA filters than the newer 18MP sensors (beginning with the T4i), so the 60D output looks softer at the pixel level compared to the T4i/T5i/SL1/EOS M. The output from the newer APS-C bodies does not look all that different from the FF cameras as far as pixel level detail in ISO 100 test shots is concerned, have a look:

5DIII vs. EOS M:
http://www.the-digital-picture.com/Reviews/ISO-12233-Sample-Crops.aspx?Lens=458&Camera=792&Sample=0&FLI=0&API=3&LensComp=458&CameraComp=812&SampleComp=0&FLIComp=0&APIComp=3

EOS M vs 60D:
http://www.the-digital-picture.com/Reviews/ISO-12233-Sample-Crops.aspx?Lens=458&Camera=812&Sample=0&FLI=0&API=3&LensComp=458&CameraComp=736&SampleComp=0&FLIComp=0&APIComp=3