Sporgon said:
privatebydesign said:
I download the RAW files from DPReview and do my own processing to them for any comparison I am interested in.
I do this too in order to get a feeling of the raw output from different cameras, but the problem I find is that virtually all the images, whether they be from DPR or others, are either exposed for an unedited ooc jpeg, or under exposed. Often the two go together. So it is difficult for me to establish from these on-line downloadable files what the results would be if the exposures were optimal for the kind of output I had in mind. This brings us back to the point you were making about being able to compare optimally exposed and processed files for a specific camera, rather than an across the range standard that often disadvantages Canon.
The tests we do are in a controlled environment using a sphere with an evenly lit aperture that can be consistently checked so we know its the same from one test to another. This has a grate that has an equivalent of a 20 stop DR change and shots taken with this using a base ISO of 100 (for instance) are then analysed automatically i.e. no human interpretation. The process we use is identical to manufacturers.
The same light sphere can also be used to shoot resolution tests with the CIPA High resolution chart which we also do however these are not automatically referenced and do rely on human "knowledge" the same as using a projector to test lenses for a number of aberrations but again nothing different to how the majors lens manufacturers work because we have visited them and had their training as well as our own.
Finally we shoot LOTS of footage / images in varying light conditions and know the ones that will really test equipment.
I stand by the Bill Claff comment my point was his methods are consistent.