FoCal Database for Lens Quality of Focus

[takesome1]

I kind of get the feeling that the point of the thread was to say your lenses are above average.
We can not let you do that by tearing your own abilities down.

But I would concede that by the care you have taken to check that your gear is probably above average. More than likely you wouldn't have tolerated a sub standard lens.

 

[AlanF]

I kind of get the feeling that the point of the thread was to say your lenses are above average.
We can not let you do that by tearing your own abilities down.

Read the first posts - I started this thread to discuss the usefulness of the database. It was after neuro's comments that the values are skewed by poor technique to lower the average values that I quoted my data with poor technique that gave values above average. Don't make it personal and make false assumptions about my motives.

 

[takesome1]

Fortunately, my expensive primes are all above the average ranges.

I re-read it as requested.

 

[AlanF]

The point of starting this thread was to give the heads up that there is a database by which you can test your lenses' performances. I do that now and it has stopped me buying two lenses that were below par and confirmed my suspicions that my old 100-400mm was a bad copy. It also shows how lenses perform on different bodies. If you wish to be a naysayer and use it as yet another debating game, then it is your loss. I wondered how good my lenses were and I am now comforted I didn't buy lemons.

 

[takesome1]

The point of starting this thread was to give the heads up that there is a database by which you can test your lenses' performances. I do that now and it has stopped me buying two lenses that were below par and confirmed my suspicions that my old 100-400mm was a bad copy. It also shows how lenses perform on different bodies. If you wish to be a naysayer and use it as yet another debating game, then it is your loss. I wondered how good my lenses were and I am now comforted I didn't buy lemons.

I do not dispute that it is useful information, I could see it useful if you were continually low with results it could show that either your technique or equipment has a problem. As an indication if you have better than average exceptional equipment I think it would be lacking because of the variables involved.

Also I do not see how it would be a good indication of how lenses perform on different bodies. If this were the case wouldn't the 5Ds R be the worse one in in the sample?

 

[bluenoser1993]

Alan, what does the lens profile look like for the 400 DO @560 on your other bodies?

 

[bluenoser1993]

It's possible they're collecting the data, but kelvin is the only thing I see in the report. I remember older versions use to show EV level during set up, but it doesn't anymore (not that I saw anyway). I agree, if they could group the results with the shutter speed it would compare better.

The 30w was LED value, not the equivalent, but still way to low. Not to mention LED is not the best for AF anyway.

I'd like to correct myself after looking at old reports in my Focal History. The EV is not in the summary at the beginning of the report, it is included in the details for each AFMA value tested, so is collected. The shutter speed is also collected in 2.0 version reports. Reports from tests I did on version 1... do not have a shutter speed.

 

[AlanF]

Alan, what does the lens profile look like for the 400 DO @560 on your other bodies?

See below. At 560, the 400 is very good on the 5DS R and 7DII. A few of us who had the 400 DO II lens agreed that there was very little improvement, if any, in resolution on going from the 1.4xTC to the 2xTC on these bodies with similarly small pixels. Conversely, there was a good gain with the 5DIV body and 2xTC. I had earlier found the same with the 300/2.8 II and 1.4xTC and 2xTC on the 5DIII (and now IV) vs the 5DS R and 7DII. The FoCal QoF data in the tables I presented on page 1 bear this out as well, although they measure acutance. So, with the 7DII I stick to 560mm, and with the 5DIV I use the 2xTC.

 

[AlanF]

The point of starting this thread was to give the heads up that there is a database by which you can test your lenses' performances. I do that now and it has stopped me buying two lenses that were below par and confirmed my suspicions that my old 100-400mm was a bad copy. It also shows how lenses perform on different bodies. If you wish to be a naysayer and use it as yet another debating game, then it is your loss. I wondered how good my lenses were and I am now comforted I didn't buy lemons.

I do not dispute that it is useful information, I could see it useful if you were continually low with results it could show that either your technique or equipment has a problem. As an indication if you have better than average exceptional equipment I think it would be lacking because of the variables involved.

Also I do not see how it would be a good indication of how lenses perform on different bodies. If this were the case wouldn't the 5Ds R be the worse one in in the sample?

You are absolutely correct that the 5DS R (along with the 7DII) should be the worst on the QoF scores (because smaller pixels give less sharp transitions than large ones). And, if you had in fact read the table I presented in the opening post (and repeated with my own data added), you would have seen that the 5DS R is the worst.

 

[takesome1]

You are absolutely correct that the 5DS R (along with the 7DII) should be the worst on the QoF scores (because smaller pixels give less sharp transitions than large ones). And, if you had in fact read the table I presented in the opening post (and repeated with my own data added), you would have seen that the 5DS R is the worst.

Yes there is a reason for it.
The point is that the differences in the bodies are in the design, not necessarily in the quality of picture one should expect of the camera. For that reason comparing how one body performs versus another is really only useful in respect to this test.

Unless you see other information about the bodies that can be derived from it, I do not.

 

[bluenoser1993]

See below. At 560, the 400 is very good on the 5DS R and 7DII. A few of us who had the 400 DO II lens agreed that there was very little improvement, if any, in resolution on going from the 1.4xTC to the 2xTC on these bodies with similarly small pixels. Conversely, there was a good gain with the 5DIV body and 2xTC. I had earlier found the same with the 300/2.8 II and 1.4xTC and 2xTC on the 5DIII (and now IV) vs the 5DS R and 7DII. The FoCal QoF data in the tables I presented on page 1 bear this out as well, although they measure acutance. So, with the 7DII I stick to 560mm, and with the 5DIV I use the 2xTC.

I searched their site a little, but couldn't find how they define average range, knowing their limits would help a bit. I agree with others that the tool will certainly indicate to a user that technique or equipment is poor when well below the average, probably the best use of the tool. However, in your case the the result for the 400 is so far beyond the average I think it merits claim of good equipment and technique. For the 5DsR and 7DII I think it also shows your ability to get the best of the equipment, more so than the masses.

I went to a print shop last night to print a couple targets with far better quality than my printer, one of a larger size for the longer tests as well. Once I resolve the lighting, I intend to get my best result possible for the 100-400 to compare to the results I posted earlier in the thread. I hope it will compare the spread of the average range vs the spread of poor to better technique. I'll share the findings here once I've had a chance to complete the test.

I will add, I like the way you used it to compare teleconverter results.

 

[Mt Spokane Photography]

You cannot logically derive that my technique is performing better than average. What you can draw from the data is that the spread of my values is above the average. The reason for the better than average results could be a better technique or a better range of samples. Given the description of my technique, it is unlikely it is better than average.

The big factor in the testing is vibration during the long exposures that focal uses in poor light. You get around the issue with brighter lighting and the resulting faster shutter speeds, or by a firmer tripod support, or in very few cases, both.

Generally, even in dull light outdoors, its very bright compared to indoor lighting. But, even if it isn't, the fact that your tripod is sitting on the ground rather than the wooden floor typical of many homes, which vibrate and ring like a bell with any movement. So, I would say that your technique likely surpasses most users from the critical vibration issue, because it addresses both issues, firmer support, and brighter lighting.

 

[AlanF]

So, I am either a pretty average experimentalist with some above average lenses or I am better than average experimentalist who gets the best out of his average lenses? What would you prefer to be?

 

[bluenoser1993]

So, I am either a pretty average experimentalist with some above average lenses or I am better than average experimentalist who gets the best out of his average lenses? What would you prefer to be?

LOL. All I'm saying in the case of the 5DsR and 7DII paired with your 400 DO is that with their premium glass I'd like to think Canon's QC is better than what it would take for your lens quality alone to explain how far above the average range it is.

 

[AlanF]

So, I am either a pretty average experimentalist with some above average lenses or I am better than average experimentalist who gets the best out of his average lenses? What would you prefer to be?

LOL. All I'm saying in the case of the 5DsR and 7DII paired with your 400 DO is that with their premium glass I'd like to think Canon's QC is better than what it would take for your lens quality alone to explain how far above the average range it is.

Lensrentals frequently has articles on the poor quality control, e.g: "The summary is almost all of you greatly overestimate the type and amount of optical testing that lenses get, whether it’s at the factory after assembly or in the repair center when it has a problem." https://www.lensrentals.com/blog/2016/09/is-your-camera-really-the-best-optical-test/

They have also measured copy to copy variation in lenses. The variation can be horrendous. Here is one article with references to their earlier studies https://www.lensrentals.com/blog/2015/07/variance-measurement-for-35mm-slr-lenses/

The copy of the 400mm DO II used by image resources is a pretty poor one, not being tack sharp http://www.imaging-resource.com/lenses/canon/ef-400mm-f4-do-is-ii-usm/review/

The ePhotozine copy tested was spectacular. https://www.ephotozine.com/article/canon-ef-400mm-f-4-do-is-ii-usm-lens-review-26785

There is considerable variation even in these super-expensive lenses.

 

[takesome1]

So, I am either a pretty average experimentalist with some above average lenses or I am better than average experimentalist who gets the best out of his average lenses? What would you prefer to be?

It seems everyone thinks you are better than average, I just can not see how that would be offensive.

I don't think a comparison of Riekans reports eastablish lenses as just average or exceptional. It could indicate a bad one though.

 

[bluenoser1993]

Well, I hope someone can explain these results. I did screenshots of the reports at the peek AFMA value so you can see the images side-by-side that are giving the QoF. You can see the extra light in the report, and shutter speed. What you can't see is the heavy weight hanging on the tripod, tripod on basement floor (cement), central heat shut off (no vibration), tripod legs forced into wider spread at floor to reduce wobble, target on solid wall, target larger and of much better print quality, FoCal delay after mirror lock increased to 3 seconds, more time waiting after each AFMA change on body before continuing.

400mm and 140mm were done at almost same distance of 10.1 - 10.3 meters, I didn't have room to get any further away so the new 560mm was done at 10.1m where as the original was done at 13m.

It made sense to see the 400mm QoF increase (though I had hoped for more), but then I was puzzled to see a reduction of QoF at 560mm, and then essentially no change at 140mm.

The one thing I wonder, does FoCal use the declared target size (and hence calculated distance) as a part of the calculation of the QoF? I assumed having the target size increase from 116mm to 209mm would have improved the QoF because of the improved image captured, but maybe it's accounted for?

EDIT - left image is the new (better technique) attempt in each focal length below.

 

[takesome1]

Well, I hope someone can explain these results. I did screenshots of the reports at the peek AFMA value so you can see the images side-by-side that are giving the QoF. You can see the extra light in the report, and shutter speed. What you can't see is the heavy weight hanging on the tripod, tripod on basement floor (cement), central heat shut off (no vibration), tripod legs forced into wider spread at floor to reduce wobble, target on solid wall, target larger and of much better print quality, FoCal delay after mirror lock increased to 3 seconds, more time waiting after each AFMA change on body before continuing.

400mm and 140mm were done at almost same distance of 10.1 - 10.3 meters, I didn't have room to get any further away so the new 560mm was done at 10.1m where as the original was done at 13m.

It made sense to see the 400mm QoF increase (though I had hoped for more), but then I was puzzled to see a reduction of QoF at 560mm, and then essentially no change at 140mm.

The one thing I wonder, does FoCal use the declared target size (and hence calculated distance) as a part of the calculation of the QoF? I assumed having the target size increase from 116mm to 209mm would have improved the QoF because of the improved image captured, but maybe it's accounted for?

EDIT - left image is the new (better technique) attempt in each focal length below.

I can say they are absolutely meaningless when in comparison to each other.
Different lighting produces different results. The size of the target and the distance modify everything. The examples you show are from either different distances, the same distances or different targets.

I think this is a good example of why the averages that focal provides can not be relied on as a guide of lens quality.

If you want to use this number to compare the quality of a lens you need to have all things equal. Here are a few factors I know of that you didn't mention. Matt paper vs glossy for a target. Printing with a printer capable of printing photo's vs a normal laser jet. Target position and placement being square to the set up. Positioning of lighting, is the lighting direct or indirect at target. Type of lighting, I have several halogens that cast shadows.

 

[Loswr]

As a scientist, I always have more unpublished data to present to the referees to counter their arguments.

It also seems that as a scientist, you initially 'published' your best examples, not representative examples.

I may have done that a few times...

 

[AlanF]

Bluenoser, I have done hundreds of calibrations at many different distances using the same target to see if the AFMA changes with distance, from 10-25m, and found regularly the reported QoF changes no more than it does for repeat measurements at the same distance. So target size doesn't make much distance. What your experiments show very clearly, and answer the objections thrown at me, the FoCal procedure is very robust and doesn't depend that much on technique - you have used very poor technique to a much improved one.

I corresponded with FoCal about this a couple of days ago:

On Wed, 1 Feb at 12:00 PM, AlanF wrote:
My experience is that the QoF for my telephoto lenses doesn’t change that much with distance or lighting conditions - I have done loads of repeat runs over the years.

Reply from Reikan:
Yep, the current QoF calculation is designed to provide comparison as much as possible. It's hard to guarantee as test environments can be very different. The more similar the test set up the more likely the results will be directly comparable

I think that the FoCal database is very useful and far more robust than the critics here claim. I'll ask FoCal how the spreads of the average values are calculated.