Recommend websites for lens reviews

[xvnm]

Hi,

I'm looking for a good lens review website.

One site that I really like is DPReview (http://www.dpreview.com/lensreviews?sort=brand), in particular their sharpness charts (example: http://www.dpreview.com/lensreviews/canon_100_2p8_is_usm_c16/4). However, they have a very limited database, only 11 Canon lenses as of now.

I also like The Digital Picture (http://www.the-digital-picture.com/Reviews/Canon-Lens-Reviews.aspx) and Ken Rockwell (http://www.kenrockwell.com/canon/lenses/), but they tend to be more on the subjective side of reviews: not a lot of numbers and charts to compare across. Nothing wrong with that, but not exactly what I'm looking for.

DxOMark (http://www.dxomark.com/index.php/Lenses/Camera-Lens-Ratings) has a pretty extensive database, and although some of their reviews can be very controversial here, I have no reason to believe they are not doing a proper job. However, I find most of their charts using colors instead of lines and numbers difficult to compare objectively. Also, sometimes I don't understand their testing criteria.

For instance, the 85/1.8 vs. the 100/2 on the 5DIII: http://www.dxomark.com/index.php/Lenses/Compare-Camera-Lenses/Compare-lenses/%28lens1%29/241/%28brand%29/Canon/%28camera1%29/0/%28lens2%29/798/%28brand2%29/Canon/%28camera2%29/0

The 85 is sharper (15 vs 14), has better transmission (2 vs 2.2) and less aberration (3 vs 5). The 100 has slightly less distortion (0.3% vs 0.4%) and vignetting (1.4 vs 1.6). Pretty similar numbers, I'd say. Yet, the 100 has a score of 30 while the 85 gets only 26. And I don't understand why they always seem to say that all lenses are best wide open ("Best at f=100mm & f/2", "Best at f=85mm & f/1.8")

Another example, Sigma 18-35/1.8 vs. Sigma 35/1.4 on the 7D: http://www.dxomark.com/index.php/Lenses/Compare-Camera-Lenses/Compare-lenses/%28lens1%29/1141/%28lens2%29/1056/%28brand1%29/Sigma/%28camera1%29/0/%28brand2%29/Sigma/%28camera2%29/619

The 35 has better transmission (1.6 vs 1.8 ), distortion (0.2% vs 0.3%), vignetting (0.8 vs 1.1), and aberration (5 vs 7). The 18-35 is only slightly sharper (13 vs 12), yet the 18-35 has a score of 24 and the 35 only 22. I don't understand their numbers.

Also, highly regarded lenses, like the EF-S 10-22 or the 16-35/2.8L, have some relatively poor numbers on DxOMark.

My ideal site would be like DPReview with a database the size of DxOMark What do you recommend?

 

[ajfotofilmagem]

I trust test "www.photozone.de". They test various aspects individually, and tell you what they hoped to find a lens in this category. Besides the graphics, there are pictures that illustrate problems as "longitudinal chromatic aberration," and say that this situation can really be a problem.

 

[Loswr]

I don't understand why they always seem to say that all lenses are best wide open

Because DxOMark's definition of "best" probably isn't the same as yours - I know it's not the same as mine. Their Measurements are good, and usually reliable (except for certain cases, like the Canon 70-200 II which they measures as worse than the MkI version of the lens, defended their measurements when called on them, then quietly ran the tests again on another copy and updated their results).

But their Lens Score is defined as how well a lens performs in 150 lux illumination (like a dimly lit warehouse). So, lenses are "best" when wide open, fast lenses generally score better than slow lenses (is the 50/1.8 really better than the 600/4 II? I suppose it might be, if you want to shoot inside a dimly lit warehouse), and a lens tested on a camera with better high ISO performance better than the same lens tested on a camera with worse high ISO performance.

When you look at the Lens Score, then at the Metrics underneath it, you really want to believe that the Metrics are the basis for the Score....but that's just not the case.

 

[ajfotofilmagem]

Another thing about Photozone is that they relate, for example, who tried three copies of a given lens, to find a copy of which was well centered. See an explanation I found, right on the homepage Photozone, explaining the lack of more tests Sony lenses:

"Test delayed due to defects and poor service ...Yes, we are buying many lenses just for testing.
Defective lens: Sony E 35 mm f/1.8 OSS
Problem: Defect of centralization
Service center: Sony Australia
Current Status: Repair done
Now, waiting for ... took 74 days."

 

[mrsfotografie]

Another thing about Photozone is that they relate, for example, who tried three copies of a given lens, to find a copy of which was well centered. See an explanation I found, right on the homepage Photozone, explaining the lack of more tests Sony lenses:

"Test delayed due to defects and poor service ...Yes, we are buying many lenses just for testing.
Defective lens: Sony E 35 mm f/1.8 OSS
Problem: Defect of centralization
Service center: Sony Australia
Current Status: Repair done
Now, waiting for ... took 74 days."

They may buy many lenses, but I wouldn't say they buy several copies of the same model just to be objective. Many test lenses are borrowed from friendly users, and only a single item is tested. If a lens suffers from decentering, it is mentioned and such as is the case with the Sony lens mentioned, rejected..

 

[ajfotofilmagem]

Another thing about Photozone is that they relate, for example, who tried three copies of a given lens, to find a copy of which was well centered. See an explanation I found, right on the homepage Photozone, explaining the lack of more tests Sony lenses:

"Test delayed due to defects and poor service ...Yes, we are buying many lenses just for testing.
Defective lens: Sony E 35 mm f/1.8 OSS
Problem: Defect of centralization
Service center: Sony Australia
Current Status: Repair done
Now, waiting for ... took 74 days."

They may buy many lenses, but I wouldn't say they buy several copies of the same model just to be objective. Many test lenses are borrowed from friendly users, and only a single item is tested. If a lens suffers from decentering, it is mentioned and such as is the case with the Sony lens mentioned, rejected..

Yes, but I think its important to know that particular model had several samples that suffer from problems. It is also honest enough to say that particular lens was tested and it was found a small error of centralization, especially seen in the lower left corner when the maximum aperture diaphragm, for example.

 

[Solar B]

http://www.slrgear.com/reviews/index.php

http://www.lenstip.com/

 

[jthomson]

http://www.photozone.de/
http://www.lenstip.com/
http://www.lensrentals.com/blog

You need to dig in the LensRentals blog, but Roger provides great information based on multiple copies of the lenses.

 

[bholliman]

The Digital Picture and Photozone are the two best in my opinion.

 

[Mt Spokane Photography]

DP Review uses the same test data as DXO, DXO even opened a test Facility in Seattle near them.

As noted, test methods vary according to the opinion of the tester about the best method.

Also be aware that no one tests just a lens, they test a lens-camera combination, and the result is only valid for that combination. This means you cannot compare a Nikon and a Canon lens tested on different bodies. It also means that you will get drastically different results when comparing a APS-C body and a FF body, the FF body will always test better.

I tend to view several of the test sites, and normally find them in general agreement, except for DXO, who seems to be in disagreement with the rest of the world quite often.

Each site may also test for unique parameters, astronomy photographers want to know about coma, for example, so they might prefer a site like lenstip that tests for coma.

 

[wsmith96]

I don't understand why they always seem to say that all lenses are best wide open

Because DxOMark's definition of "best" probably isn't the same as yours - I know it's not the same as mine. Their Measurements are good, and usually reliable (except for certain cases, like the Canon 70-200 II which they measures as worse than the MkI version of the lens, defended their measurements when called on them, then quietly ran the tests again on another copy and updated their results).

But their Lens Score is defined as how well a lens performs in 150 lux illumination (like a dimly lit warehouse). So, lenses are "best" when wide open, fast lenses generally score better than slow lenses (is the 50/1.8 really better than the 600/4 II? I suppose it might be, if you want to shoot inside a dimly lit warehouse), and a lens tested on a camera with better high ISO performance better than the same lens tested on a camera with worse high ISO performance.

When you look at the Lens Score, then at the Metrics underneath it, you really want to believe that the Metrics are the basis for the Score....but that's just not the case.

I've wondered about their scoring as well and your summary really helped me understand what they are doing. Regarding sharpness, is there any merit to DxO's PMpx score? Are they stopping down the lenses at all to find the sharpest setting for a lens, or are they all pretty much testing wide open regardless?

 

[AlanF]

http://www.slrgear.com/reviews/index.php

http://www.lenstip.com/

+1 along with Photozone.de

They are all quantitative but not comprehensive.
But, TDP is my first port of call because it is so comprehensive. All those sites are unbiased.

 

[Loswr]

I've wondered about their scoring as well and your summary really helped me understand what they are doing. Regarding sharpness, is there any merit to DxO's PMpx score? Are they stopping down the lenses at all to find the sharpest setting for a lens, or are they all pretty much testing wide open regardless?

I'm pretty sure they're using the sharpest value for a given lens at any tested aperture for the P-Mpix value. I think it's useful as an internal comparator, but I don't like the fact that it's a 'black box'. An old physics teacher always used to say, "No units, no answer." I suppose a made-up unit is better than no units, but not by much. Maybe 0.84 sweezleblats better, approximately.

 

[gtog]

... An old physics teacher always used to say, "No units, no answer." ...

Interesting comment in a photography forum -- I expect most photographers would still find f-stops useful. Of course you would think a physicist would still find Pi useful too. Curious. Perhaps a side effect of a teacher grading too many student papers?

 

[Loswr]

I expect most photographers would still find f-stops useful.

Well, I suppose we could say mm/mm, but that's a bit cumbersome. I think he knew about ratios, though. But I suspect there are a fair number of photographers who don't know that the f/number is one.

 

[cid]

... An old physics teacher always used to say, "No units, no answer." ...

Interesting comment in a photography forum -- I expect most photographers would still find f-stops useful. Of course you would think a physicist would still find Pi useful too. Curious. Perhaps a side effect of a teacher grading too many student papers?

Pi is useful, but it's only irrational number representing commonly used constant (let's say ratio). When you want to express something using Pi, you have to add some units. e.g. circle area Pi*R^2 where R is measured in mm or cm or feet or miles, without that you simply don't know how big the circle is

 

[AprilForever]

Hi,

I'm looking for a good lens review website.

One site that I really like is DPReview (http://www.dpreview.com/lensreviews?sort=brand), in particular their sharpness charts (example: http://www.dpreview.com/lensreviews/canon_100_2p8_is_usm_c16/4). However, they have a very limited database, only 11 Canon lenses as of now.

I also like The Digital Picture (http://www.the-digital-picture.com/Reviews/Canon-Lens-Reviews.aspx) and Ken Rockwell (http://www.kenrockwell.com/canon/lenses/), but they tend to be more on the subjective side of reviews: not a lot of numbers and charts to compare across. Nothing wrong with that, but not exactly what I'm looking for.

DxOMark (http://www.dxomark.com/index.php/Lenses/Camera-Lens-Ratings) has a pretty extensive database, and although some of their reviews can be very controversial here, I have no reason to believe they are not doing a proper job. However, I find most of their charts using colors instead of lines and numbers difficult to compare objectively. Also, sometimes I don't understand their testing criteria.

For instance, the 85/1.8 vs. the 100/2 on the 5DIII: http://www.dxomark.com/index.php/Lenses/Compare-Camera-Lenses/Compare-lenses/%28lens1%29/241/%28brand%29/Canon/%28camera1%29/0/%28lens2%29/798/%28brand2%29/Canon/%28camera2%29/0

The 85 is sharper (15 vs 14), has better transmission (2 vs 2.2) and less aberration (3 vs 5). The 100 has slightly less distortion (0.3% vs 0.4%) and vignetting (1.4 vs 1.6). Pretty similar numbers, I'd say. Yet, the 100 has a score of 30 while the 85 gets only 26. And I don't understand why they always seem to say that all lenses are best wide open ("Best at f=100mm & f/2", "Best at f=85mm & f/1.8")

Another example, Sigma 18-35/1.8 vs. Sigma 35/1.4 on the 7D: http://www.dxomark.com/index.php/Lenses/Compare-Camera-Lenses/Compare-lenses/%28lens1%29/1141/%28lens2%29/1056/%28brand1%29/Sigma/%28camera1%29/0/%28brand2%29/Sigma/%28camera2%29/619

The 35 has better transmission (1.6 vs 1.8 ), distortion (0.2% vs 0.3%), vignetting (0.8 vs 1.1), and aberration (5 vs 7). The 18-35 is only slightly sharper (13 vs 12), yet the 18-35 has a score of 24 and the 35 only 22. I don't understand their numbers.

Also, highly regarded lenses, like the EF-S 10-22 or the 16-35/2.8L, have some relatively poor numbers on DxOMark.

My ideal site would be like DPReview with a database the size of DxOMark What do you recommend?

The problem with DXO is that art is a subjective matter. Just because a lens scores oddly there, or even badly, does not mean it is a bad lens. Their numbers are meaningless to me, and I suspect in any real world setting, more or less. Also, the DXO scores do not take into consideration something unbelievably important yet often overlooked: the system. A nice lens which does not work with your system or methods will not help much.

 

[Solar B]

... I think he knew about ratios, though. But I suspect there are a fair number of photographers who don't know that the f/number is one.

What is the f/number?

 

[Northstar]

Slrgear.com and photozone.

 

[Zv]

... I think he knew about ratios, though. But I suspect there are a fair number of photographers who don't know that the f/number is one.

What is the f/number?

f is short for focal length. So f/2 would be - focal length divided by 2. So if you're focal length is 50mm your aperture is .... (Complete the answer and you've got it!)

In other words it a way to express the size of the aperture in relation to the focal length. A ratio. Since not all lenses are the same size it wouldn't help to just say 22mm. You would be left wondering if that is wide or narrow and you would have to figure that part out yourself! It's easier to have a ratio that says "it's half as wide as the focal length" or it's a quarter etc etc.