jeffa4444 said:
JRista and Neuro -- does any other opinion really matter on this topic?
How do both of you measure performance of lenses & sensors?
As for DxO the comment that there lens measurements are not that great they should be straight forwards. MTF readings if the same device is used don't recognise a manufacturer just a reading. On a projector the graticule is the same and its straight forwards seeing lateral aberrations / colour fringing etc, pin cushioning / barrel distortions etc. and whether the lens has even field illumination.
I measure what I own by the quality of my results, especially when I've reach the point where I am pushing the limits of my equipment. It's at the limits when you truly and fully understand the quality of something.
I gauge the presumed quality of what I intend to buy by absorbing
and discerning every potential scrap of information on said intention from every possible source man has devised.

Never base your decisions on a single source, that'll just get you into trouble. Use as many sources of information as you can find, and always keep a critical eye on the details (especially the hidden ones.) Don't take anything for granted, be skeptical until you see actual evidence for any argument or conclusion offered by a reviewer. (The Digital Picture reviews are an excellent resource, he is a great reviewer, and he always offers both subjective as well as objective data to back up his claims.)
It helps to have at least a basic theoretical understanding of cameras and lenses as well. There are limits to how useful your average lens test can be, regardless of who does it, because output resolution (the measurable resolution of images used to gauge the performance of each lens) is limited by the lowest common denominator...these days, that is usually the sensor. Many lenses are capable of resolving FAR more detail at faster diffraction-limited apertures than you might otherwise glean from the average lens review, because the upper bound on resolution is the sensor. There is no such thing as lenses outresolving sensors or sensors outresolving lenses...the two work in concert to produce a photograph, and the resolution of the photograph is the RMS of the resolutions of the lens+sensor. Increase the resolution of either, output resolution increases...however increase the resolution of the least capable, and output resolution will benefit most. IF your sensor can only resolve 50lp/mm, it doesn't matter how good the lens is....120lp/mm, 170lp/mm, 250lp/mm...your standard lens test will always show the "lens" resolution is less than 50lp/mm. Because it's sensor bound! If there was anything to know about lens reviews, that would be it.
In the end, the technical specifications are really, ultimately, not nearly as important as reviewers often make it seem. Photography equipment today is so vastly superior to nearly everything we had in the past (with the possible exception of large format film digitized with high end, high resolution drum scanners), we are utterly spoiled by the highly accessible and easily usable equipment often available at our fingertips on a moment's notice.
Here is another approach. If you cannot entirely describe in both technical and artistic terms exactly why any given piece of your current equipment is holding you back, then your not ready to move to the next level. When you know your equipment backwards and forwards, and fully understand every advantage and disadvantage, every pro and con, and can clearly define your NEED for something more...then and only then should you upgrade, but at that point you should have the skill to filter the useless technical jargon from the useful technical gems. When you can clearly articulate your need for larger pixels or a larger frame/FOV or thinner DOF, then you have a clear reason to upgrade. If you don't know why larger pixels are better, you might find it useful to push your skill with smaller pixels or a smaller sensor until you see how and why they may be holding you back. (Sometimes that realization can't be realized in a box...it's often helpful to compare your art to others art, especially the art of those that inspire you...ARTISTIC comparisons can often help you glean a deeper understanding of WHY one technical tidbit or another has value.)
It's pretty rare that the minute technical details that organizations like DXO putter around with actually have any real
meaning to people's ART. It may have meaning to some people's addiction to collecting the most advanced technological gadgetry known to man...but when it comes to art, there are millions that make do just well, hell far more than "well", with lowly little devices like a Canon 350D. Art is the expression of what you see in your head. Technology
sometimes makes it easier to realize that vision, but when it comes to digital, more often than not, more important than the hardware is the software, and more important than the software is how you use it. Just go digging through 500px sometime. The volume and quality of works with 90+ ratings made with equipment many of us today would label "UTTER CRAP" is astonishing. It's
how you use what you have, not necessarily
what you have.
Technology is not the artist, it's just the brush. Don't get too wrapped up in the technology. Sometimes you just need
a lens and
a camera, and to let you ideas flow.