Most Objective and Less Objective REVIEWER?

[LifeAfter]

...we really need OBJECTIVE reviews to know what to expect from a lens before buying.

Do we? I suspect most such decisions are based on features and price. Plus, everyone has bias. The best you can probably do is read several reviews and synthesize a conclusion.

+1

 

[LifeAfter]

So that pushed me to make this Poll, to see the users opinion and maybe the credibility of the reviewers.

Seems to me that many don't understand what "objective" means. "Objective" is not the same thing as "credible" or "useful". The luminous landscape reviews for example are largely subjective -- they generally do not contain any measurements or test chart shots, they are largely descriptions of the author's experience with and impressions of the lens.

Lens Rentals is a very useful site, and contains a large amount of both objective tests as well as purely subjective "Roger's take" comments.

TDP have the test charts, but I don't see how that website has any business leading this poll. Objective doesn't mean "subjective and agrees with me". Ironically, the two highest scoring sites in the poll are two of the more subjective reviewers (e.g. lenstip and photozone are more objective than LR or TDP)

100% agree with all the above

 

[Pi]

The problem with many sites is - they test lens sharpness only, and nothing else. Many years and many lenses later, I learned to ignore sharpness tests - once you get a modern high quality lens (on FF), sharpness is good enough and the other qualities of the lens like bokeh, AF accuracy, local contrast (higher MTF values) flare, etc., are far more important.

PZ, for example would criticize fast wide primes at f/1.4 for corner performance, comparing them, implicitly, to slower or longer primes. Really, Klauss? Their bokeh test is as joke. DXO is fixated on "acutance" only and they do not even document what that means. They recently reviewed the 135L and roughly speaking, concluded that it was good enough for an old lens, once you get a good deal on it but the Sigma 85 is better. This is a bad joke by people who do not shoot outside their labs. The TDP guy is much more down to earth. Dpreview, IMO, publishes infomercials only, nothing trustworthy there.

Sometimes I think that KR's "reviews" are more useful than PZ, etc.

As we all see, we really need to read all the reviews of important websites like these to have a more objective and definitive information about a product - mostly lenses, before buying it.

And of course taking the most objective and reliable part of all of them:

Dpreview: to get technical information of a product (mostly bodies).
The-Digital-Picture: the comparison tool.
Digitalrev: to laugh a bit.
Lensrentals: Their primarily purpose is commercial, they do more a kind of presentation than a real review.
anyway they are more objective then the above three.
Photozone: They really have they ''bar put too high'', i do think they are the most objective out there.

 

[elflord]

PZ, for example would criticize fast wide primes at f/1.4 for corner performance, comparing them, implicitly, to slower or longer primes. Really, Klauss?

I don't think that's an accurate criticism. When he reviews test results (like soft corners or conspicuous barrel distortion or absence of distortions), he makes a point of mentioning whether those results are expected (or better or worse than expected) for that type of lens.

 

[comsense]

...we really need OBJECTIVE reviews to know what to expect from a lens before buying.

Do we? I suspect most such decisions are based on features and price. Plus, everyone has bias. The best you can probably do is read several reviews and synthesize a conclusion.

Well said. Bias is inherent to human brain function. Only way without bias is letting machine do it. That's why good technical data from well designed setup is valuable (TDP, Photozone, DxO, LensRentals; more the better). You can look at it and make up your mind with your own bias. If you want no technical data and self contradictory but peppy talk, well you have 6 Rockwell fans here.....

 

[AmbientLight]

@comsense:

Incredible, you actually believe that a machine might provide less biased information compared to a human being. How can this be?

Do you believe that machines are created by some sort of greater and completely flawless beings or have you already realized that machines tend to be man-made and as such bring in their own flaws and bias based on how a machine has been designed? ???

 

[comsense]

@comsense:

Incredible, you actually believe that a machine might provide less biased information compared to a human being. How can this be?

Do you believe that machines are created by some sort of greater and completely flawless beings or have you already realized that machines tend to be man-made and as such bring in their own flaws and bias based on how a machine has been designed? ???

I am not trying to get into creation of universe. Its simply that human brain works with lot of associations and information filling and past experience shapes the future perception/action. They are huge advantages (critical to learning and adaptability) which unfortunately comes at a cost (like anything else). So no matter how objective you claim to be, your experience and association shapes your views and analysis.
Machine on the other hand would measure what you make it to measure without feelings or likings. Machine does not change measurements depending on whether Nikon or Canon made it. Of course all machines have errors and range/constraints where measurements are reliable (and they do break down). Ironically most of the blames instruments get are because of poor experimental design and non-optimal use, both of which are human errors.
Before someone throws a pebble into water, attempts are being made to model computing machines that work like human brain, i.e., adaptable neural network with associative powers.
If you are still scratching your head:
For a input = output situation
With machines -
a given 'Input' will give you same 'Output' every time till the machine fails (in workable constrains of machine)
with brain -
'Output' can change hardware such that same 'Input' next time could have different 'Output'....

 

[TWI by Dustin Abbott]

Lens Rentals and Photozone seem to produce some of the most trustable data overall when it comes to lens reviews. TDP has had a lot of weird results IMO (and has the guy EVER tested a single Tamron that wasn't a lemon? does he just go all sloppy with his procedure and not care when he tests them or what? maybe he shoots the chart very close in? I suspect he doesn't refocus for edges which may make results better for some but worse for others). Photozone sometimes says crazy things in the final text review of a lens though IMO even the data plots look good. DxO has had all sorts of utterly absurd lens data on their website (although their sensor data mostly appears to be very reliable and easily the best of any review site in that case).

Dpreview seems to do pretty well with lenses but they have often been behind the times when it comes to looking at sensor performance with poorly thought out DR tests and having resisted normalization for SNR and so on for ages.

Bryan is my favorite overall reviewer (The Digital Picture). I look forward to his reviews the most, but I must confess that this point is a sticking one for me. There are always pretty severe caveats applied to third party lenses. His general purpose zoom recommendations for this year includes the EF 28-135 but does not mention the Tamron 24-70 VC, which is absurd. The only general purpose zoom that could claim to top the Tamron would be the new 24-70L MKII.

 

[Random Orbits]

Bryan is my favorite overall reviewer (The Digital Picture). I look forward to his reviews the most, but I must confess that this point is a sticking one for me. There are always pretty severe caveats applied to third party lenses. His general purpose zoom recommendations for this year includes the EF 28-135 but does not mention the Tamron 24-70 VC, which is absurd. The only general purpose zoom that could claim to top the Tamron would be the new 24-70L MKII.

Well, he has yet to complete the Tamron 24-70 VC review. Would it be fair for him to rank something he hasn't tested?

 

[TWI by Dustin Abbott]

Bryan is my favorite overall reviewer (The Digital Picture). I look forward to his reviews the most, but I must confess that this point is a sticking one for me. There are always pretty severe caveats applied to third party lenses. His general purpose zoom recommendations for this year includes the EF 28-135 but does not mention the Tamron 24-70 VC, which is absurd. The only general purpose zoom that could claim to top the Tamron would be the new 24-70L MKII.

Well, he has yet to complete the Tamron 24-70 VC review. Would it be fair for him to rank something he hasn't tested?

But he has. He just hasn't written a review on it. He's got the ISO 12233 resolution tests done. But once again, like others have mentioned regarding other 3rd party lenses, he's had so many "centering" issues that he can't do a real test. Ironic, considering that the sample variation of the MK1 24-70L was notoriously poor, and yet he found a good enough sample to test and recommend for many years. Many pros (including myself, for that matter) are using the Tamron and loving it. I just have a hard time believing that he could not get a decent sample of the lens and actually write the review on it.

 

[TWI by Dustin Abbott]

P.S. This was Photozone's concluding statement at the end of their review of the new 24-70L II:

"The question of the day is, of course, whether this is "enough" compared to the impressive Tamron AF SP 24-70mm f/2.8 Di USD VC ? Well, we have some doubts here. We'd say that among the primary criteria the Canon lens has an edge in terms of contrast (at max. aperture), build quality and it has a slightly better bokeh. However, the Tamron lens is as sharp in the lower zoom range and provides a much better border quality at 70mm. Additionally it has a unique selling point - an image stabilizer. So unless you're heading into a war zone or are into tough press business (thus requiring max. equipment quality) a premium of one grand (EUR) over the Tamron lens seems a little excessive. "

One might think that in all fairness to the consumer that the Tamron at least get a mention in the lens recommendations on Bryan's site.

P.S. As I have said before, Bryan is my favorite reviewer out there. He just seems to have a blind spot in this particular area.

 

[Loswr]

I just have a hard time believing that he could not get a decent sample of the lens and actually write the review on it.

I don't see that he's mentioned any problems with the Tamron 24-70/2.8 VC. He just hasn't completed the review yet. He's previously stated that he prioritizes reviews of Canon lenses/bodies over 3rd party products...Canon has released quite a few lenses lately.

Did you also notice that he want and re-shot all the ISO 12233 crops for the EF-S lenses with the 60D (many of them only had 50D images)? He also posted that, "There are now 25,344 crops from 8,448 ISO 12233 test images available for review on the site." Speaking as someone who has the same test charts (the largest size costs more than some L-series lenses), and has gone through the process of setting up such tests, ~8,500 test images represents a huge amount of work (and that's just the ISO 12233 crops; he also tests vignetting, flare, and distortion).

 

[sdsr]

I read most of those listed, along with a few others (including comments here, of course), and most of them usually seem to say something useful; reviews that provide real-world examples with comparisons of the same subject taken with different lenses are those I find the most useful (though if I'm interested in a fairly expensive lens, I'll rent a copy and try it for myself). One reason why I like lenstip is that their reviews always include a section on coma, complete with examples, which is an aspect of lens performance that's usually ignored (Rockwell, whatever else on may think of him, usually mentions/shows it too) but can matter to those of us who like taking photos in very low light.

Lensrentals has a huge advantage over the rest in that they get to test large quantities of lenses. Some reviewers will mention efforts made to replace an obviously faulty copy, but often a sub-par lens won't be obviously defective; it may simply seem not very good. Dismay has been expressed over the relative ranking at DxO of the 70-200 2.8 II, but the results may simply accurately reflect the copy/ies they had. This, rather than bias/subjectivity, is probably the main drawback to reviews.

 

[TWI by Dustin Abbott]

I just have a hard time believing that he could not get a decent sample of the lens and actually write the review on it.

I don't see that he's mentioned any problems with the Tamron 24-70/2.8 VC. He just hasn't completed the review yet. He's previously stated that he prioritizes reviews of Canon lenses/bodies over 3rd party products...Canon has released quite a few lenses lately.

Did you also notice that he want and re-shot all the ISO 12233 crops for the EF-S lenses with the 60D (many of them only had 50D images)? He also posted that, "There are now 25,344 crops from 8,448 ISO 12233 test images available for review on the site." Speaking as someone who has the same test charts (the largest size costs more than some L-series lenses), and has gone through the process of setting up such tests, ~8,500 test images represents a huge amount of work (and that's just the ISO 12233 crops; he also tests vignetting, flare, and distortion).

With all due respect, this seems like a bit of a stretch. Bryan did mention sample issues on several occasions. The Tamron has been a pretty big 3rd party release that is a game changer in that particular arena because of adding a stabilizer and has a had a LOT of adopters - including professionals. I would consider it and the Sigma 35mm f/1.4 to be the most significant 3rd party releases in years.

Do I doubt that Bryan has a ton of work on his plate? Absolutely not. Do I think that he simply could not find the time to review the lens? I'm sorry, I don't. He has reviewed quite a few lenses that were released months later, and almost every other big review site has already reviewed the lens.

Of course, like you, this is all just my opinion, so take it for what it is worth. But I'll tell you this, when you are in the field without a tripod, being able to get a shot like this at 1/15th second without a second thought is pretty fantastic!

 

[TWI by Dustin Abbott]

...
Lensrentals has a huge advantage over the rest in that they get to test large quantities of lenses. Some reviewers will mention efforts made to replace an obviously faulty copy, but often a sub-par lens won't be obviously defective; it may simply seem not very good. Dismay has been expressed over the relative ranking at DxO of the 70-200 2.8 II, but the results may simply accurately reflect the copy/ies they had. This, rather than bias/subjectivity, is probably the main drawback to reviews.

This is a very good point, and one that I have found invaluable. I really appreciate Roger's assessments like this because of the unique perspective he has due to the quantity of the lenses he gets to test.

 

[Random Orbits]

With all due respect, this seems like a bit of a stretch. Bryan did mention sample issues on several occasions. The Tamron has been a pretty big 3rd party release that is a game changer in that particular arena because of adding a stabilizer and has a had a LOT of adopters - including professionals. I would consider it and the Sigma 35mm f/1.4 to be the most significant 3rd party releases in years.

Do I doubt that Bryan has a ton of work on his plate? Absolutely not. Do I think that he simply could not find the time to review the lens? I'm sorry, I don't. He has reviewed quite a few lenses that were released months later, and almost every other big review site has already reviewed the lens. Of course, like you, this is all just my opinion, so take it for what it is worth. But I'll tell you this, when you are in the field without a tripod, being able to get a shot like this at 1/15th second without a second thought is pretty fantastic!

And that is the biggest problem I have with this line of thought. TDP is not beholden to anyone for which reviews it does, in what order it does it or how it schedules its reviews. It is a free resource, and it's a good one. Use other resources that have the reviews you want if you'd like. It is the bolded paragraph that smacks of the increasing "entitlement" disease that seems to pervade the net these days....

 

[TWI by Dustin Abbott]

With all due respect, this seems like a bit of a stretch. Bryan did mention sample issues on several occasions. The Tamron has been a pretty big 3rd party release that is a game changer in that particular arena because of adding a stabilizer and has a had a LOT of adopters - including professionals. I would consider it and the Sigma 35mm f/1.4 to be the most significant 3rd party releases in years.

Do I doubt that Bryan has a ton of work on his plate? Absolutely not. Do I think that he simply could not find the time to review the lens? I'm sorry, I don't. He has reviewed quite a few lenses that were released months later, and almost every other big review site has already reviewed the lens. Of course, like you, this is all just my opinion, so take it for what it is worth. But I'll tell you this, when you are in the field without a tripod, being able to get a shot like this at 1/15th second without a second thought is pretty fantastic!

And that is the biggest problem I have with this line of thought. TDP is not beholden to anyone for which reviews it does, in what order it does it or how it schedules its reviews. It is a free resource, and it's a good one. Use other resources that have the reviews you want if you'd like. It is the bolded paragraph that smacks of the increasing "entitlement" disease that seems to pervade the net these days....

I understand what you are saying, and agree to some degree. I am not attacking Bryan; as I have said repeatedly, he is my favorite reviewer. I also do lens reviews that many people have used to guide buying choices, so I am not a parasite.

I have offered this as further evidence to a point that others have raised; there is, perhaps, a bias FOR OEM Canon equipment and AGAINST any 3rd party equipment. This is (IMO) the chink in the armor at the Digital Picture. Since this is an opinion forum, I am entitled to mine. I have tried to defend my point, and will now step away.

 

[TWI by Dustin Abbott]

No more fuel to the fire, but just out of interest:

From the Digital Picture:

"Tamron 24-70mm VC Lens Image Quality - Take 4
Tamron 24-70mm f/2.8 Di VC USD Lens
If you have been following this blog/new page, you know that I have been making a big effort to get good image quality results from the Tamron 24-70mm f/2.8 Di VC USD Lens. This is currently the only full frame-compatible lens available in the 24-70mm focal length range with both an f/2.8 max aperture and Vibration Control - a combination that many of us find very attractive.

Evaluating this lens, as I hinted, has been very problematic. The first lens I bought had a serious image quality issue - Tamron wanted it back for analysis.

The second lens I bought performed well, but was noticeably softer on the right side - the side that shows in the site's ISO 12233 image quality tool. I suspected decentering and sent the lens to Tamron for repair/adjustment.

The lens came back from repair performing worse than before I sent it in. Tamron sent me a shipping label to take a second try at the repair.

The results from the second repair, the fourth test, are now found on the Tamron 24-70mm VC Lens Image Quality page. And, <drumroll>, they look very good!

We spent over a full week testing this lens model for the ISO 12233 chart test alone, but persistence has paid off ... I think we have an as-good-as-it-gets copy in our hands now. This process has not said good things about the quality control for this lens model, but again, the image quality looks very nice to me.

What is not looking good is this lens' AI Servo AF performance. This lens has not been able to accurately focus on subjects moving toward or away from the camera. One Shot AF has been working fine, so this lens has plenty of great uses available for it. More to come."

P.S. I took a look through some of the ISO 12233 charts and compared them with the chief competition (Canon 24-70l II). Wide open, I would give the edge at most focal lengths to the Canon (24mm is debatable and might go Tamron - 35mm strongly favors the Canon) although they are quite close. Interestingly, stopped down to f/4, it would appear that this sample of the Tamron is sharper at all tested focal lengths.

 

[Random Orbits]

No more fuel to the fire, but just out of interest:

From the Digital Picture:

"Tamron 24-70mm VC Lens Image Quality - Take 4
Tamron 24-70mm f/2.8 Di VC USD Lens
If you have been following this blog/new page, you know that I have been making a big effort to get good image quality results from the Tamron 24-70mm f/2.8 Di VC USD Lens. This is currently the only full frame-compatible lens available in the 24-70mm focal length range with both an f/2.8 max aperture and Vibration Control - a combination that many of us find very attractive.

Evaluating this lens, as I hinted, has been very problematic. The first lens I bought had a serious image quality issue - Tamron wanted it back for analysis.

The second lens I bought performed well, but was noticeably softer on the right side - the side that shows in the site's ISO 12233 image quality tool. I suspected decentering and sent the lens to Tamron for repair/adjustment.

The lens came back from repair performing worse than before I sent it in. Tamron sent me a shipping label to take a second try at the repair.

The results from the second repair, the fourth test, are now found on the Tamron 24-70mm VC Lens Image Quality page. And, <drumroll>, they look very good!

We spent over a full week testing this lens model for the ISO 12233 chart test alone, but persistence has paid off ... I think we have an as-good-as-it-gets copy in our hands now. This process has not said good things about the quality control for this lens model, but again, the image quality looks very nice to me.

What is not looking good is this lens' AI Servo AF performance. This lens has not been able to accurately focus on subjects moving toward or away from the camera. One Shot AF has been working fine, so this lens has plenty of great uses available for it. More to come."

P.S. I took a look through some of the ISO 12233 charts and compared them with the chief competition (Canon 24-70l II). Wide open, I would give the edge at most focal lengths to the Canon (24mm is debatable and might go Tamron - 35mm strongly favors the Canon) although they are quite close. Interestingly, stopped down to f/4, it would appear that this sample of the Tamron is sharper at all tested focal lengths.

Yes, I saw it too; interesting. TDP also mentioned that the Tamron's had more issues with Servo AF. Have you found that to be the case? Nice pics with your Tamoron, BTW.

 

[TWI by Dustin Abbott]

No more fuel to the fire, but just out of interest:

From the Digital Picture:

"Tamron 24-70mm VC Lens Image Quality - Take 4
Tamron 24-70mm f/2.8 Di VC USD Lens
If you have been following this blog/new page, you know that I have been making a big effort to get good image quality results from the Tamron 24-70mm f/2.8 Di VC USD Lens. This is currently the only full frame-compatible lens available in the 24-70mm focal length range with both an f/2.8 max aperture and Vibration Control - a combination that many of us find very attractive.

Evaluating this lens, as I hinted, has been very problematic. The first lens I bought had a serious image quality issue - Tamron wanted it back for analysis.

The second lens I bought performed well, but was noticeably softer on the right side - the side that shows in the site's ISO 12233 image quality tool. I suspected decentering and sent the lens to Tamron for repair/adjustment.

The lens came back from repair performing worse than before I sent it in. Tamron sent me a shipping label to take a second try at the repair.

The results from the second repair, the fourth test, are now found on the Tamron 24-70mm VC Lens Image Quality page. And, <drumroll>, they look very good!

We spent over a full week testing this lens model for the ISO 12233 chart test alone, but persistence has paid off ... I think we have an as-good-as-it-gets copy in our hands now. This process has not said good things about the quality control for this lens model, but again, the image quality looks very nice to me.

What is not looking good is this lens' AI Servo AF performance. This lens has not been able to accurately focus on subjects moving toward or away from the camera. One Shot AF has been working fine, so this lens has plenty of great uses available for it. More to come."

P.S. I took a look through some of the ISO 12233 charts and compared them with the chief competition (Canon 24-70l II). Wide open, I would give the edge at most focal lengths to the Canon (24mm is debatable and might go Tamron - 35mm strongly favors the Canon) although they are quite close. Interestingly, stopped down to f/4, it would appear that this sample of the Tamron is sharper at all tested focal lengths.

Yes, I saw it too; interesting. TDP also mentioned that the Tamron's had more issues with Servo AF. Have you found that to be the case? Nice pics with your Tamoron, BTW.

I haven't, but, to be fair, I rarely shoot in a situation where I need critical speed like that. I do shoot events (business, church, weddings, etc...) and haven't had any issue with it in those settings. But that isn't the same as trying to track fast movement like sports. I love the lens, myself.