The old hands know very well how much individual copies of lenses vary and that lens review sites can be unreliable. The unreliability comes from not doing measurements well or because they usually look at only one copy of a lens or both. The gold standard is Lensrentals who do very careful analyses of a large number of lenses using the best equipment manned by competent users. Nevertheless, people choose lenses based on those unreliable measurements. It's worth bringing this up for discussion, and I'll give two examples, based on lenses that many of us use and know well.
First collage, the Canon 100-400mm II. The profiles from two sites of good reputation are quite different. Photozone has the lens sharpest wide open from 200 to 400mm. On the other hand, ePhotozine at 400mm has a great dip in sharpness at 400mm at f/5.6 that leaps up at f/8.
Second collage, Canon 100-400mm II vs Sigma 150-600mm C at 400mm. The profiles from Lenstip has the Canon slightly sharper at 400mm at f/8 than at f/5.6, between the Photozone and ePhotozine results. It has the Sigma performing very poorly on going from 300mm to 450mm. Roger of Lensrentals has compared the average of many copies of each lens at 400mm, and there is very little to choose between them, on average, and points out that individual copies of the lens vary to the extent that either one may be better than the other.
My two copies of the 100-400mm II fit the Photozone pattern, sharpest wide open. And the my Sigma is very close to if not better than the Canons at 400mm.
The conclusion is that the most important comparison is that of your own individual lens and that the sites that just look at one copy of a lens can be taken only as a very rough guide.
First collage, the Canon 100-400mm II. The profiles from two sites of good reputation are quite different. Photozone has the lens sharpest wide open from 200 to 400mm. On the other hand, ePhotozine at 400mm has a great dip in sharpness at 400mm at f/5.6 that leaps up at f/8.
Second collage, Canon 100-400mm II vs Sigma 150-600mm C at 400mm. The profiles from Lenstip has the Canon slightly sharper at 400mm at f/8 than at f/5.6, between the Photozone and ePhotozine results. It has the Sigma performing very poorly on going from 300mm to 450mm. Roger of Lensrentals has compared the average of many copies of each lens at 400mm, and there is very little to choose between them, on average, and points out that individual copies of the lens vary to the extent that either one may be better than the other.
My two copies of the 100-400mm II fit the Photozone pattern, sharpest wide open. And the my Sigma is very close to if not better than the Canons at 400mm.
The conclusion is that the most important comparison is that of your own individual lens and that the sites that just look at one copy of a lens can be taken only as a very rough guide.
) A site presents a finding and we need to review it, question it, consider it, and file it alongside others. With one copy comes great responsibility. NEVER render an opinion on one lens tested on one body. TDP gets 1-3 copies (usually 1 or 2) and tests it on a few bodies with some clearly defined methods. LR famously gets 10 lenses because they can -- but they only test IQ -- handling and AF are not reported. PZ is now testing one lens on both 21 MP and 50 MP bodies, which is nice way to consider the future-proofing of the lens for the high MP world. Everyone else (to my knowledge) is writing War and Peace about one lens on one body, which is dangerous to bet the farm on. So always always always read multiple reviews from trusted sources and mentally aggregate their findings:
