Dpreview of the 80D

[[email protected]]

Rishi, I know some may think it's inadvisable to "feed the trolls" and answer questions on a forum like this, but I really appreciate your participation. And please excuse our trolls. There are, though, lots of people on here who will think better of DPR as you answer questions and engage in discussions.

It says a lot - and good things - that you are engaging on the forum. I think DPR will benefit in the comprehensiveness of its coverage and process improvement of its reviewing to keep tabs on Canon Rumors. Even if part of what people here think is B.S., they'll benefit from your responses, and you'll benefit from the non-B.S. bits.

One plug, if I may. I think you and Barney have a great presence of your videos. You do a really good job of coming off as friendly and watchable. But overall, one gets the impression (which might be incorrect) that the people doing the reviews are using Canon equipment temporarily, just for the purpose of the review. There is a different feel that one gets from reviewers that are of the relevant ecosystem. I think that the next time DPR hires an additional reviewer, you might wish to bring on a Canon shooter - not to do all the Canon reviews - but to add this perspective, and in particular, a very strong basis for real world comparison versus previous Canon products. That last review feature - answering the question about whether one should upgrade - is done poorly if it is done by the spec sheet.

Anyway, very happy to have you as a fellow member. -tig

 

[rishi_sanyal]

Another note, many reviewers (esp. DPReview) love to compare the 45 or 61 AF points in optical viewfinder of DSLRs to the 425 AF points in A6300. Then, they wax lyrical about the number of A6300 AF points.

Errr... doesn't the 80D have 80% * 24 MP, that is, 19.2 million AF points in live view?

No, each single pixel is not an AF point - you need strips of pixels to form an AF point.

Why strips? Is that for DPAF or generic contrast-detection?

You need contiguous pixels (not necessarily exactly next to one another - you can skip some pixels) to make a phase measurement. One strip/row might be 'left looking' and another 'right looking', and you correlate features between the two to determine the phase difference between them, which then gives you directional and magnitude information.

For example, see this schematic of the E-M1 scheme of on-sensor PDAF: http://www.imaging-resource.com/PRODS/olympus-e-m1/ZTECH_PDAF_PIXELS.gif.

Cheers,
Rishi

 

[Orangutan]

Another note, many reviewers (esp. DPReview) love to compare the 45 or 61 AF points in optical viewfinder of DSLRs to the 425 AF points in A6300. Then, they wax lyrical about the number of A6300 AF points.

Errr... doesn't the 80D have 80% * 24 MP, that is, 19.2 million AF points in live view?

No, each single pixel is not an AF point - you need strips of pixels to form an AF point.

Why strips? Is that for DPAF or generic contrast-detection?

You need contiguous pixels (not necessarily exactly next to one another - you can skip some pixels) to make a phase measurement. One strip/row might be 'left looking' and another 'right looking', and you correlate features between the two to determine the phase difference between them, which then gives you directional and magnitude information.

For example, see this schematic of the E-M1 scheme of on-sensor PDAF: http://www.imaging-resource.com/PRODS/olympus-e-m1/ZTECH_PDAF_PIXELS.gif.

Cheers,
Rishi

OK, so you were talking about DPAF. So then the "AF point" is the center of that collection of (almost) contiguous pixels?

 

[rishi_sanyal]

Rishi,

First of all, welcome to CR, I hope you enjoy the visit.

Second, I appreciate you taking the time to provide feedback on DPR for us.

Thanks, that's very kind of you.

I appreciate you guys being hard on Canon actually, they are at the top, and they are the one to take pot shots at, and, if they produce a better product because of your reporting, that's a win in my book.

I certainly hope we're not taking 'pot shots' - we've just been modernizing our testing to cover things like Raw image quality and autofocus, things we didn't always look at in a controlled manner or as thoroughly before. If a camera does poorly in any of these tests, we point that out, but we don't find random negative things to say. We have set things we look at, hopefully things that matter to photographers.

I also wouldn't say we're being any harder on Canon than we are Sony, or Nikon for that matter. I myself have written far more negative phrases concerning Sony's ergonomic frustrations than I have pointed out cons on Canons. Have you maybe overlooked our Sony reviews/editorial content? I've used more hyperbolic words to address ergonomic frustrations on Sony cameras than any adjective I've used to describe Canon: 'arthritis-inducing', 'inexplicable', 'inane', 'disorienting', and 'incomprehensible', to cite a few I remember off the top of my head.

Our recent article on the Nikon D5's dynamic range didn't beat around the bush either.

However comma, I would concur that perhaps an AF test environment, similar to a studio scene, or what your see at high end Best Buy's with their center island in the middle of their Camera section which provides subjects in which to shoot would be a wise addition to your reporting. Say, you guys have a train table or similar moving object. High and low contrast objects, and variable lighting to demonstrate how each products AF performs under various lighting with a standard reproducible procedure for testing.

Yes, we've had many, many thoughts, conversations, brainstorms with industry leaders about how we could design meaningful AF tests. What we have now in reviews is just a stop-gap; we are tirelessly working on how to actually build a repeatable set of AF tests that covers a wide variety of shooting situations. Needless to say, it's ridiculously difficult, which may partially be why no one - to my knowledge - has succeeded just yet.

I would however start getting hard on Sony too, why? They seem to take your reviews seriously, perhaps they'll make a better product and give Canon competition? Sony needs some prodding, much harder prodding on their weak lens lineup, UI system and lack of touchscreen; they've had how many chances to fix this and still haven't? You guys need to speak up for us little people and start doling some criticism at your sacred cow (or that is how I perceive DPR's relationship with Sony, if I do, ask yourself why do we think that if there isn't some truth?)

But we do do just that: we continuously address exactly the points you bring up. We go on and on about lack of touchscreen, lack of direct AF point selection, lack of a 'My Menu' or menu organization, mushy buttons/dials, etc. Have a read of the 'Handling' section of my a7R II review, or read my specific frustrations with the RX10 III we recently published: http://bit.ly/1O4rpHP. I literally suggested Sony hire a design firm & redesign their entire UI from the ground up, but, hopefully more helpfully, we try to point out our exact points of contention. Especially because you, the reader, need to know if our particular complaints are relevant to you.

As you allude to, it may very well be our constant prodding on poor Auto ISO implementation, lossy Raw, lack of direct magnification of the focus point, etc., that, at least in part, led to the now class-leading Auto ISO implementation (with direct, one-button, access to minimum shutter speed threshold), uncompressed Raw, and instant 100% magnification of the focus point used in Image Review. I've talked directly to engineers about the pitfalls of stop-down focusing, or how buttons with little haptic feedback can cost you a shot in the field.

You are right that we have a responsibility to point out such flaws to help end users of these products - a responsibility we take very seriously. I can personally say that my end goal is not just to inform our customers to help them choose more wisely, but also to raise awareness of issues - many of which we even learn from our audience members and people like yourself - so that we get better products. After all, I'm a photographer as well and constantly desire better tools for the job.

Thanks again for your feedback and thoughts,
Rishi

 

[rishi_sanyal]

Rishi, I know some may think it's inadvisable to "feed the trolls" and answer questions on a forum like this, but I really appreciate your participation. And please excuse our trolls. There are, though, lots of people on here who will think better of DPR as you answer questions and engage in discussions.

It says a lot - and good things - that you are engaging on the forum. I think DPR will benefit in the comprehensiveness of its coverage and process improvement of its reviewing to keep tabs on Canon Rumors. Even if part of what people here think is B.S., they'll benefit from your responses, and you'll benefit from the non-B.S. bits.

Thanks, very kind of you. Particularly because, yes, it can get tricky here.

One plug, if I may. I think you and Barney have a great presence of your videos. You do a really good job of coming off as friendly and watchable. But overall, one gets the impression (which might be incorrect) that the people doing the reviews are using Canon equipment temporarily, just for the purpose of the review. There is a different feel that one gets from reviewers that are of the relevant ecosystem. I think that the next time DPR hires an additional reviewer, you might wish to bring on a Canon shooter - not to do all the Canon reviews - but to add this perspective, and in particular, a very strong basis for real world comparison versus previous Canon products. That last review feature - answering the question about whether one should upgrade - is done poorly if it is done by the spec sheet.

Anyway, very happy to have you as a fellow member. -tig

We have a couple Canon shooters on staff, and I myself have shot Canon for most of my life, from the film days all the way through to every 5D-series camera. I do my part to make sure that any particular issue called out in a review is not just a familiarity or 'teething' issue.

If anything, our staff actually lacks Sony shooters (to be expected, given they're newer to the game), which might partially explain our constant collective gripes about Sony ergonomics. I say partially because some things, like mushy buttons with little haptic feedback, are absolutely, objectively true. It's important for us to distinguish the objective from the subjective and, yes, I fully concede that's not always possible!

Actually, we encourage all our reviewers to constantly shoot with all systems, so that any time they write a review, they have a rounded perspective. This affords us a unique perspective (I hope): we know which system is best at what (or try to anyway!), which allows us to place any particular facet of a camera in perspective. A shooter of one system only may have a self-selecting bias: if a camera is bad at X, then you may never use X, and may even come to think that X is itself useless (probably now because you've learned to work around it). We want to avoid that as much as possible.

This does, however, mean that sometimes we point out 'issues' that users of a particular system may never have perceived as an issue - simply because of unfamiliarity with a better performing system. This is why comparisons, rather than tests in isolation, are so useful.

That said, you are correct that perspectives on systems from system shooters is definitely valuable. To that end, we are trying more and more to work with pros of every system to understand their needs and how they use their cameras. So I certainly appreciate your point. We're particularly considering input from pros as we're trying to review more pro-level cameras: like the 1D-X II and D5. These are purpose-built machines that perform like no other when it comes to certain types of photography, and we need to make sure we understand the appropriate requirements. So I can tell you right now that for these two particular cameras, we're engaging with pro users of the respective systems.

May I ask where in particular you felt that our reviewer(s) might have had a lack of perspective when it came to Canon?

Thanks,
Rishi

 

[privatebydesign]

May I ask where in particular you felt that our reviewer(s) might have had a lack of perspective when it came to Canon?

Thanks,
Rishi

My biggest issue is the misinformation, half truths, and blatant lies.

Things like:

In the 1DX MkII "examined-in-depth" piece (http://www.dpreview.com/articles/0676551873/canon-eos-1d-x-mark-ii-first-impressions-review?slide=16) at least three times you say things like "not as versatile as Nikon's class-leading 3D tracking" and "given the pinpoint precision Nikon 3D tracking is capable of ", yet in your Nikon D5 examined piece (http://www.dpreview.com/articles/9189851572/hands-on-with-nikon-d5?slide=11) you say nothing about Canon, not even where specs of already available Canon cameras vastly out strip it.

Or things like the 5DSR "impossible-to-control background scene of high contrast, and exposing to retain the sky meant that shadow brightening to make the foreground anything but a sea of black resulted in noise and banding" which you personally couldn't avoid, but can be avoided easily!
http://www.dpreview.com/reviews/canon-eos-5ds-sr/12
http://www.canonrumors.com/forum/index.php?topic=29255.msg582690#msg582690

Example images like that, with accompanying 'explanations', show you to be the biased and corrupted source you have become.

Don't get me wrong, I don't give a darn about fairness, life isn't fair and nobody should expect it to be. But I hate with a passion people who lie while trying to pass themselves off as fair minded sources of unbiased information.

 

[Jack Douglas]

There is a difference between lying and inadvertently misrepresenting the truth or making an incorrect statement out of ignorance, etc. I think we should be considerate in how we describe other folk relative to motives. Especially those who are saddled with somewhat thankless jobs.

OTOH, Nothing wrong with strongly pointing out shortcomings assuming they are true. IMHO

Jack

 

[privatebydesign]

There is a difference between lying and inadvertently misrepresenting the truth or making an incorrect statement out of ignorance, etc. I think we should be considerate in how we describe other folk relative to motives. Especially those who are saddled with somewhat thankless jobs.

OTOH, Nothing wrong with strongly pointing out shortcomings assuming they are true. IMHO

Jack

Jack, I agree, that is why I chose my words and gave specific instances and links (as Rishi requested). I simply don't believe many of the comments in DPReview are borne from ignorance, nor inadvertent misrepresentation.

 

[rishi_sanyal]

May I ask where in particular you felt that our reviewer(s) might have had a lack of perspective when it came to Canon?

Thanks,
Rishi

My biggest issue is the misinformation, half truths, and blatant lies.

Things like:

In the 1DX MkII "examined-in-depth" piece (http://www.dpreview.com/articles/0676551873/canon-eos-1d-x-mark-ii-first-impressions-review?slide=16) at least three times you say things like "not as versatile as Nikon's class-leading 3D tracking" and "given the pinpoint precision Nikon 3D tracking is capable of ", yet in your Nikon D5 examined piece (http://www.dpreview.com/articles/9189851572/hands-on-with-nikon-d5?slide=11) you say nothing about Canon, not even where specs of already available Canon cameras vastly out strip it.

It's fairly futile for me to try and respond to someone who goes back and copy & pastes a comment he made in February of this year, which didn't even stand up to scrutiny. If you remember: you weren't even able to defend your statement when I asked you 'what specs outstrip the D5 that I didn't mention?' You weren't able to name a single spec where existing Canons 'vastly' outperformed the D5, save for the one I did mention: Live View / Movie AF.

By the way, we still fully stand behind those phrases on subject tracking, and they're not lies - they're the results of our very careful experiments. We don't just repeat spec, by the way, else we'd talk about how the 360k-pixel RGB+IR metering sensor of the 1DX II blows away the mere 180k-pixel RGB metering sensor of the D5. On the contrary - we know from actual extensive testing that Nikons are far more effective at using their metering sensors for subject tracking. Unsurprisingly, many who've switched from Canon to Nikon know this as well.

As a scientist, I have to always admit we may be wrong, so: we may be wrong. But I highly doubt it, from our repeated and vetted tests. In fact, our results are apparent to anyone we ask to actually do a side-by-side. What's odd is that when I then ask: 'have you tried the two side-by-side' to people such as yourself who claim that we're lying, the answer is invariably 'no'. That's pretty telling.

This gets back to the point I made earlier: you may not even know you're missing something if you never had it at all.

Or things like the 5DSR "impossible-to-control background scene of high contrast, and exposing to retain the sky meant that shadow brightening to make the foreground anything but a sea of black resulted in noise and banding" which you personally couldn't avoid, but can be avoided easily!
http://www.dpreview.com/reviews/canon-eos-5ds-sr/12
http://www.canonrumors.com/forum/index.php?topic=29255.msg582690#msg582690

Example images like that, with accompanying 'explanations', show you to be the biased and corrupted source you have become.

I'm not even going to address the argument that somehow more dynamic range isn't better because DR limitations can be 'avoided easily' because you can just 'expose properly!' - this argument has been addressed, and refuted, countless times and, frankly, I'm pretty sure you know better anyway. More dynamic range is more dynamic range, whether or not you know how, or need, to use it. Period.

I will say this though: saying things like: "This shot has more DR than the DPR image, how do we know that? The sun is still above the horizon." doesn't actually prove anything. In fact, ironically, views toward the sun after the sun has sunk below the horizon are higher DR than ones pointed at the sun while it's still above the horizon (unless you're trying to keep the actual sun from blowing, which is a fool's errand), because the sky above the sunset is still bright, while the foreground barely has any light. That's why most of our 'real world' dynamic range tests are done a half hour after sunset, or a half hour before sunrise.

Like this real-world, properly ETTR'd real-world test vs. the D810, where we demonstrate the literally stops of dynamic range advantage of the D810 over the 5DS R: http://bit.ly/1NL0aXY. I'm not sure what your isolated example demonstrates when our controlled comparison actually demonstrates the difference between these cameras - which is representative of what you may be leaving on the table when choosing a camera with lower dynamic range. Unless you are so presumptuous as to believe your scene and processing is representative of the limit of what any photographer in the world might ever want to do with their camera... your isolated example is, well, just an isolated example. We, OTOH, are just showing you the difference, and showing you a real-world examples where the limited DR did affect our image. Are we forcing you to care? Not by any means. But we're showing you what the implications, and differences compared to competitors, is. If that makes us liars: so be it.

And this is all before we address the obvious noise reduction (whether you purposefully applied any or not) in your image, or the fact that you haven't even worked up the image - actual contrast editing and grading exacerbates noise. The whole reason that pros find larger sensor cameras to provide more flexible images, by the way.

Again, we're back to the 'you may not know the advantage of something you haven't had'. If you don't understand, or need, the advantages, of increased dynamic range, that's fine. But to suggest we're intentionally deceiving is highly disingenuous. The sunrise or sunset scene you present in your thread, as I alluded to above, doesn't even represent a high dynamic range scene a professional landscape photographer may be wont to produce. Nor is it a controlled representation of the actual noise cost an ISO-variant (lower base ISO DR) camera will display vs. an ISO-invariant (higher base ISO DR) camera. These are often noise differences on the order of magnitude of stops, yet it's OK to argue over 1/3 stop noise improvements in high ISO? Interesting...

Increased noise from lower DR can even be apparent in more modest pushes, yet some of you like to act like there's some 'threshold' - like the extra noise isn't noticeable enough until > 3 stop pushes. That's just not true - the noise is there, and it may or may not affect your photography, but it's there on a lower dynamic range camera and we're showing it. It may even become far more evident than our studio test scene for Raw DR even shows when you actually add some contrast back in.

Also, why are we talking about dynamic range in relation to Canon only? Perhaps you missed our recent article on the Nikon D5's dynamic range: http://www.dpreview.com/news/9402203921/nikon-d5-shows-drop-in-dynamic-range

That's a harsher title than we've ever used on a Canon dynamic range article, which is odd considering your hypothesis that we somehow intentionally lie to favor Nikon and diss Canon.

Interestingly, you conveniently ignore when we praise Canon, claiming the 80D 'broke new ground' for Canon low ISO dynamic range: http://bit.ly/1YZe0GR. Could it be we're more interested in actual technology that helps photographers, and less interested in the brand?

Don't get me wrong, I don't give a darn about fairness, life isn't fair and nobody should expect it to be. But I hate with a passion people who lie while trying to pass themselves off as fair minded sources of unbiased information.

You are free to 'hate with a passion' all you want. But next time you call us a liar, it may strengthen your case to have at least one controlled test that disproves even one point we've made.

 

[rishi_sanyal]

There is a difference between lying and inadvertently misrepresenting the truth or making an incorrect statement out of ignorance, etc. I think we should be considerate in how we describe other folk relative to motives. Especially those who are saddled with somewhat thankless jobs.

OTOH, Nothing wrong with strongly pointing out shortcomings assuming they are true. IMHO

Jack

Jack, I agree, that is why I chose my words and gave specific instances and links (as Rishi requested). I simply don't believe many of the comments in DPReview are borne from ignorance, nor inadvertent misrepresentation.

That's the one thing you've written that is 100% true.
-Rishi

 

[Sporgon]

This gets back to the point I made earlier: you may not even know you're missing something if you never had it at all.



That's why most of our 'real world' dynamic range tests are done a half hour after sunset, or a half hour before sunrise.

Aha, so I got it spot on then when I tried to recreate what you had done. At that exposure I estimated it was half an hour after sunset that was required to get that effect - with under exposure.

http://www.canonrumors.com/forum/index.php?topic=29255.msg582690#msg582690

I recon you were about two to three stops under where you could have been and still hold highlights in the evening sky. It's old news that until this very latest generation, Canon dslrs were (are) not forgiving of severe under exposure. However the 5Ds would normally cope easily with two stops and dark rgb values in the range of 25, but what killed it is the fact that the light density was by then so thin. If you had done this same thing with a D810 you would not have had the noise but you'd still have very poor tonality, so still way off what a competent photographer would be able to produce, who wasn't specifically trying to demonstrate an issue.

 

[scyrene]

I look at Ken Rockwell for a rounded opinion

LOL

 

[Don Haines]

So if a review praises the camera dozens of times and criticizes it a few times then it is biased against it? I don't think so......

I have found that DPR does the best reviews of any of the review sites. Could they be better? Of course they could..... there is always room for improvement as perfection is something to be strived for, yet never obtained....

I thought it was a very well done review. My only complaint with it is the fascination that the camera industry has with ISO 100 and would have liked to see more tests done at high ISO like 3200 or 6400 so the readers get an idea of the relative merits of cameras when you start to push things a bit.....

But overall, it was well done, well organized, and easy to read.

 

[Don Haines]

...
I thought it was a very well done review. My only complaint with it is the fascination that the camera industry has with ISO 100 and would have liked to see more tests done at high ISO like 3200 or 6400 so the readers get an idea of the relative merits of cameras when you start to push things a bit.....
...

What is this fascination that only canonrumors has with ISOs like 3200 or 6400?

The info for shooting at ISO100 is great for landscape photographers and those shooting with relatively short lenses in good light, but as the quantity of light drops, lenses get longer, and subjects start to move quickly you find that you need high ISO.

Let's say you are shooting at 600mm on a crop camera.... you generally need twice the focal length for the shutter speed so you are shooting at 1/1200th of a second, or double that if it is moving quickly, like a bird in flight, sports, or the bride after 12 cups of capuchino.... Since you are shooting with an 80D, it is a fair bet that you did not get the $18,000 Canon F4 lens and instead are shooting with a Sigma or a Tamron 150-600 so your minimum aperture is 6.3..... but more likely 8 to get a bit more sharpness out of the lens.

At ISO 100, 1/2400th of a second, and F8, you will need 18EV of light..... and since shooting in a snow covered background in full sun will only get you as far as EV16, your choices are to bring along a flash powerful to light up that bird 200 meters away, hope for the sun to turn supernova, or crank up your ISO.

shooting in overcast conditions or early and late in the day gives you EV12... and OMG! all of a sudden you have to crank your ISO up to 3200 to keep your aperture and shutter speed....

most arenas are lit to EV9, sometime 10 for pro stadiums.... crank the ISO up further.

One of the big reasons people go for more capable cameras is to be able to handle more difficult conditions. It would be nice to see how they perform. This is not a "fascination that only canonrumors has", it is basic information that any photographer should have, be they shoot Canon, Nikon, Sony, Olympus, or even (gasp) iPhone......

 

[rishi_sanyal]

This gets back to the point I made earlier: you may not even know you're missing something if you never had it at all.

That's why most of our 'real world' dynamic range tests are done a half hour after sunset, or a half hour before sunrise.

Aha, so I got it spot on then when I tried to recreate what you had done. At that exposure I estimated it was half an hour after sunset that was required to get that effect - with under exposure.

http://www.canonrumors.com/forum/index.php?topic=29255.msg582690#msg582690

I recon you were about two to three stops under where you could have been and still hold highlights in the evening sky. It's old news that until this very latest generation, Canon dslrs were (are) not forgiving of severe under exposure. However the 5Ds would normally cope easily with two stops and dark rgb values in the range of 25, but what killed it is the fact that the light density was by then so thin. If you had done this same thing with a D810 you would not have had the noise but you'd still have very poor tonality, so still way off what a competent photographer would be able to produce, who wasn't specifically trying to demonstrate an issue.

No, our real-world comparisons are perfectly ETTR'd - there is not even 1/3 EV headroom. We bracket hundreds of shots and take the one that is just short of clipping tones we wish to preserve in the Raw file, as explained repeatedly every time we present our results.

"If you had done the same thing with a D810 you would not have had the noise but you'd still have poor tonality" -- not any poorer tonality than a slightly higher ISO shot (a 4 EV push on the D810 gives you ~ISO 1000 quality shadows). Are you saying no self-respecting photographer shoots at ISO 800 or so on a D810?

As for your shot - I'm not sure you're aware, but completely defocusing the background decreases contrast severely = decreased DR. Even then, your image shows considerable noise (somewhat suppressed by noise reduction in DPP it looks like) that a D810 wouldn't have.

Old news, new news - I'm not sure what that has to do with anything. When we're reviewing a new camera, we're trying to show magnitude of different aspects of image quality. APS-C cameras have more high ISO noise than full-frame - old news, should we retire our studio scene?

 

[Mt Spokane Photography]

Yes, we've had many, many thoughts, conversations, brainstorms with industry leaders about how we could design meaningful AF tests. What we have now in reviews is just a stop-gap; we are tirelessly working on how to actually build a repeatable set of AF tests that covers a wide variety of shooting situations. Needless to say, it's ridiculously difficult, which may partially be why no one - to my knowledge - has succeeded just yet.

Rishi

Good luck on finding a meaningful AF test Rishi. I ran a aerospace laboratory before retiring, and as a engineer, I know that even seemingly simple tests can stir up controversy. I know just enough about autofocus to realize that there are just two many parameters involved, and one system may be more sensitive to one or more of the parameters than others. Who's to say which is more important? The last thing I'd want to see is a poor specification for autofocus measurement where innovation was limited by the need to score high on that particular test. I suspect that manufacturers feel that way as well. I spent many hours and days attending SAE meetings where representatives of manufacturers had big lists of reasons to stop the user community from implementing a specification that their product line did not meet. Then, everyone would band together to prevent ISO from implementing specifications designed to give other nations a advantage. Everyone ganged up against China. (They would all deny this, of course)

Some of the parameters involved in autofocus are color, distance, available light, ambient temperature, lens aperture, a lens itself, the list is probably much longer.

It would be nice to actually have meaningful measurements, but, for now, I am relying on reading the thoughts of multiple experienced shooters who handle multiple brands. Then, I pick a system that seems to excel at what I do.

I personally think that on-sensor autofocus is going to be the standard going forward, but its not quite there yet.

Although some CR members are hard core Canon shooters, I've owned Canon, Nikon, and Minolta DSLR's and even more brands of P&S bodies.

I welcome the thoughtful comments by shooters of all brands. Insights into operation, strong and weak points are something we should all be interested in.

 

[Don Haines]

Yes, we've had many, many thoughts, conversations, brainstorms with industry leaders about how we could design meaningful AF tests. What we have now in reviews is just a stop-gap; we are tirelessly working on how to actually build a repeatable set of AF tests that covers a wide variety of shooting situations. Needless to say, it's ridiculously difficult, which may partially be why no one - to my knowledge - has succeeded just yet.

Rishi

Good luck on finding a meaningful AF test Rishi. <snip>

Some of the parameters involved in autofocus are color, distance, available light, ambient temperature, lens aperture, a lens itself, the list is probably much longer.

I doubt that there will ever be a "fair" comparison of AF between brands..... AF is a system function and can not be isolated to a camera body or a camera lens....and it is impractical to test all possible combinations. Just sticking with current Canon EOS, you have 15 bodies and 78 lenses, that's 1,170 tests.... and that's before teleconverters and third party lenses......

Let's say you decide on a particular focal length as the standard..... odds are that the brand with the newest lens wins top marks, but change focal lengths and now another brand wins.....

You could decide to use a common lens, lets say a Sigma XXmm lens..... the tests would show how well the various cameras worked with that lens, but the data would not be indicative of how it works with lenses native to the system.

In the end, you would have a complex system where people fixate on perceived faults because "that's not the way I do things", and human nature being what it is, will never manage to move past that....

Don't make the mistake of DXO and try to dumb things down to a single number. Any attempt to represent a complex system used by diverse operators for diverse goals under diverse conditions is doomed to failure. You might already be as good as things get and further attempts at quantifying may be counter-productive

 

[Loswr]

I recon you were about two to three stops under where you could have been and still hold highlights in the evening sky.

No, our real-world comparisons are perfectly ETTR'd - there is not even 1/3 EV headroom.

That woooshing sound you hear is Sporgon's point flying over your head.

 

[Orangutan]

Yes, we've had many, many thoughts, conversations, brainstorms with industry leaders about how we could design meaningful AF tests. What we have now in reviews is just a stop-gap; we are tirelessly working on how to actually build a repeatable set of AF tests that covers a wide variety of shooting situations. Needless to say, it's ridiculously difficult, which may partially be why no one - to my knowledge - has succeeded just yet.

Rishi

I know just enough about autofocus to realize that there are just two many parameters involved, and one system may be more sensitive to one or more of the parameters than others.

<snip>

The last thing I'd want to see is a poor specification for autofocus measurement where innovation was limited
by the need to score high on that particular test.

I have a lot of respect for your opinion, and I mostly agree with what you've said above. The two points I would make are these: imperfect criteria should not stop testers from trying to develop more objective tests; and human experience is far more subject to error than imperfect tests. Confirmation bias is unavoidable in anything that involves this much human experience.

With a sufficiently large budget it would be possible to create an objective and reasonable test rig; I just don't know that any review site has the cash for it.

 

[Loswr]

...
I thought it was a very well done review. My only complaint with it is the fascination that the camera industry has with ISO 100 and would have liked to see more tests done at high ISO like 3200 or 6400 so the readers get an idea of the relative merits of cameras when you start to push things a bit.....
...

What is this fascination that only canonrumors has with ISOs like 3200 or 6400?

We don't blame you for your lack of comprehension, as we all know there's always ample light and subjects never move...in dilbertland.