July 29, 2014, 08:12:45 AM

Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.


Messages - horshack

Pages: [1] 2 3 4
1
Third Party Manufacturers / Re: DXO uh-oh?
« on: July 27, 2014, 09:52:59 PM »
The margin of error in her D3s measurements were very small; she included them for completeness, and was open about not understanding their source. The variance was small enough to be immaterial to the results.

Sorry, but your statement does not align well with hers:

I re-tested both of my D3 bodies, plus the new D3s, for this - just to make sure I produced a valid comparison. For some obscure reason - sunspots or moon phase or other strangeness - photons are behaving better today, and I achieved higher FWC results for my D3's than I have before. Because of this discrepancy, I am only going to report relative performance between the D3s and D3, instead of giving absolute measurements.

She states the discrepancy was significant enough that she would not report the absolute values.  If the source of the discrepancy could not be identified, it cannot be assumed to be a systematic error, i.e. one which would affect the measurements of the new D3s with similar magnitude and direction as it would the old D3 bodies. 

Inconsistent data, flawed assumptions...bad science.

She has a high standard for what she publishes. Her relative D3s vs D3 results still match DxO's results, and her absolute results are very close as well.

'Black box' methods...bad science.
 
I think we're done here.

Seems we couldn't come to an agreement but I appreciate the discussion.

2
Third Party Manufacturers / Re: DXO uh-oh?
« on: July 27, 2014, 09:03:26 PM »
The margin of error in her D3s measurements were very small; she included them for completeness, and was open about not understanding their source. The variance was small enough to be immaterial to the results.

As for the exact formula DxO uses for their composite scores, I have not seen them published. If you look at the scores for a cross-section of cameras and then relate them to the individual data points DxO publishes (SNR, DR, color selectivity), you can get a general idea of their weighting, but yes, the precise formula is not published. If it were I imagine we would instead be discussing how the weighting unfairly favors one camera over another, which is the natural consequence of any subjective composite score, and why DxO publishes the individual data points for those wanting to look behind the curtain.

3
Third Party Manufacturers / Re: DXO uh-oh?
« on: July 27, 2014, 07:37:47 PM »
as a multi-system user I have to agree with DXO guys, they are honest, much more so than DPR or any other unscientific review sites online.


The problem is that DXO's "science" is in dispute. How can you trust something that produces inconsistent and obviously incorrect results?


Perhaps they should submit their 'science' to the Journal of Irreproducible Results.  They may even be worthy of consideration for an IgNobel Prize.

DxO documents their sensor testing procedure here:
http://www.dxomark.com/About/In-depth-measurements/DxOMark-testing-protocols/Noise-dynamic-range

DxO results have been independently reproduced at various times. For example:
http://www.dpreview.com/forums/post/33806693
http://www.dpreview.com/forums/post/33833501


Your DxO link describes one of their Measurements, which as I've stated on multiple occasions (at least dozens, if not hundreds on these forums) I find generally well done and useful (except when they make errors and deny it, which seems to occur mainly in their lens tests).  The problems are not with their Measurements, but with their Scores.  Can you provide a link where DxO explicitly describes how their Scores are calculated from the Measurements?  No, because they don't disclose the specifics of how those Scores are calculated.  Nor do they explicitly describe the bias inherent in their Scores.

You can read about the methodology of their scores here:
http://www.dxomark.com/About/Sensor-scores

And I describe in detail their low-light score, the score which typically produces the most Canon vs Nikon controversy in online debates:
http://www.dpreview.com/forums/post/41265241

As for your 'independent reproduction,' I clicked your first link but to be honest, I stopped reading after, "For some obscure reason - sunspots or moon phase or other strangeness - photons are behaving better today, and I achieved higher FWC results for my D3's than I have before."  Sorry, but independent verification of poor pseudoscience with worse pseudoscience is even less valid than two wrongs making a right.

She wrote that as tongue 'n cheek, and it actually represents a sign of humility and willingness to be open to contrary points of view, signs of a good engineer/scientist. As for her credentials, if you follow her posts on dpreview you'll see she one of the most informed technical minds for camera sensor info. To cite a specific example, she reverse-engineered Nikon's long-exposure noise algorithm, identified serious problems with it, devised a much improved alternate algorithm which was relayed to Nikon by Thom Hogan and then later adopted by Nikon in subsequent camera designs.

4
Third Party Manufacturers / Re: DXO uh-oh?
« on: July 27, 2014, 05:41:12 PM »
as a multi-system user I have to agree with DXO guys, they are honest, much more so than DPR or any other unscientific review sites online.


The problem is that DXO's "science" is in dispute. How can you trust something that produces inconsistent and obviously incorrect results?


Perhaps they should submit their 'science' to the Journal of Irreproducible Results.  They may even be worthy of consideration for an IgNobel Prize.

DxO documents their sensor testing procedure here:
http://www.dxomark.com/About/In-depth-measurements/DxOMark-testing-protocols/Noise-dynamic-range

DxO results have been independently reproduced at various times. For example:
http://www.dpreview.com/forums/post/33806693
http://www.dpreview.com/forums/post/33833501

5
Third Party Manufacturers / Re: DXO uh-oh?
« on: July 25, 2014, 06:42:45 AM »
This really isn't a surprise. DxO and Nikon are inseparably joined at the hip. Plus, all this really means, particularly the new 14.8 stops Print DR number, is that Nikon is cooking their RAW files EVEN MORE. Nikon/Sony's biggest "cheat" is the fact that they clip to black point, instead of offsetting to black point. Nikon cameras just throw away a lot of low-level signal information. The Sony Exmor sensor gives them more room to do that, for sure, but they are still throwing away information.


Clipping the back point does not affect the DR measurements because DxO's methodology (and other testers whose independent results match DxO's) account for the clipping. Also, Nikon stopped clipping blacks starting with the Sony Exmor in the D5300 (see here: http://www.dpreview.com/forums/post/52493166).

6
Third Party Manufacturers / Re: Anyone own both Canon and Nikon
« on: July 08, 2014, 12:12:47 AM »
If you're not in a rush I would wait for Photokina and see if Canon comes out with a body that tickles your fancy.

7
If you think through what typical High ISO shooting represents then you'll reach the conclusion that there is no detail advantage for higher-density sensors in this application. Specifically, High ISO is typically used in hand-held situations for scenes that are shutter-speed limited, ie ISO is bumped to achieve the minimum shutter speed necessary to avoid motion/camera shake. Higher-density sensors require faster shutter speeds to achieve the same pixel-level sharpness as lower-density sensors - this same pixel-level sharpness is required in order for the higher-density sensor to have a detail advantage over the lower-density sensor, even for downsampling cases, otherwise you're just oversampling motion blur/camera shake. Without a faster shutter speed the higher-density image will have the same acuity as the lower-density sensor at equivalent viewing sizes (but not worse). Because the High ISO image requires a faster shutter speed, the absolute exposure (roughly ISO) will be lower on the higher-density sensor, normalizing the detail advantage over the lower-density sensor due to higher noise.

Most High ISO comparisons are done with tripod-mounted setups, which fails to account for the necessary shutter speed adjustments since a stabilized setup can achieve pixel-level sharpness at any shutter speed (at least for camera shake if not motion blur).

8
Reviews / Re: DxO reviews Sony A7s: king of low light photography?
« on: June 18, 2014, 03:37:49 PM »
The intent of DxO's composite low-light score is to measure the total sensitivity of the sensor, which includes noise, color noise, and dynamic range. The problem with the score is that it applies subjective thresholds to each of these elements (I describe this in more detail here: http://www.dpreview.com/forums/post/41265241). In general I agree with their premise to include color noise/selectivity in the score since reducing color performance is one method sensor designers have used to improve High ISO, and boosting the saturation in PP to match a more color-selective sensor increases noise and thus would make the IQ comparison unfairly penalize the better color perfomer. However their threshold for selectivity is arbitrary. Their threshold for dynamic range is even more arbitrary (and underweighted in the score IMO), which is why cameras like the 1DX/6D (and now even the A7s) are rated lower than they should be since High ISO DR (shadow noise) is one of the most important components for subjectively pleasing High ISO performance.

9
Reichmann is a bit optimistic in his appraisal of the A7s's High ISO capabilities. That said, the A7s High ISO raws posted at http://www.dpreview.com/forums/post/53852362 demonstrate what to me looks like a 1EV improvement in High ISO over the current best FF sensors (1DX/6D/D4s/Df). You can see the converted JPEGs here: http://www.dpreview.com/forums/post/53855688

10
Thanks horshack.  I think I have the AFMA at the best possible setting, based on multiple tests, it just isn't quite as sharp as when focusing with Live View.
The DOF increments associated with each AF tune increment are pretty thin, so if you're not getting AF results that match LV then that might suggest you haven't arrived at the optimal tune value just yet. In your testing, is every AF photo consistently not as sharp as your LV at your arrived tune value, or is it that there is shot-to-shot variability at your tune value? Do you see the problem at all focal lengths on the zoom?

11
Make sure when you AFMA you're using the extreme ends of the focal range (70mm and 200mm) and set the AFMA for each end appropriately. Any attempts to AFMA with the zoom in between those extremes will produce unpredictable results since the 5DM3 firmware interpolates the AF tune value between the AFMA values of the extreme ends. Additionally, I would give DotTune a shot:
DotTune: Autofocus fine tuning in under 5 minutes Small | Large

12
What is the potential risk with ML anyway? I believe I've read something about frying your sensor (!) when using Dual-ISO. Is that a big risk and if so, why?
No risk. ML runs on your memory card, the moment you replace the memory card, there is no ML ... use it without fear.

My understanding is that once you install it on the 5DIII you cannot undo the boot flag. This means that if you don't like it (slow boot times, or something else), you can't "reset" your cmaera to factory performance. That is a barrier to me. I'll give it a try when I see that it is completely reversible.

That was a limitation of the previous 5DM3 ML releases. It's been fixed in this release - the bootflag on the body can now be cleared.

13
Lenses / Re: Strange AFMA results with canon 50 f1.4
« on: February 25, 2014, 11:49:09 PM »
The 50 is one of my favorite Canon lenses but it's micro-USM motor makes it a rather finicky and imprecise focusing lens (shot-to-shot variability). I've had the most success with it by tuning at infinity - that produces good results at all distances whereas the opposite is not true (tuning at 50x and expecting it to focus well at middle to infinity distances).

14
EOS Bodies / Re: Patent: Microadjustment Automated
« on: January 13, 2014, 08:33:34 PM »
I don't think CDAF's lack of predetermine directionality would matter in this case. All that is necessary is for CDAF to achieve focus as a reference point. Once a reference point is attained (and CDAF CAN indeed achieve very good focus once it's done going through all its gyrations), you save the focus group position of the lens, then all you need to do is test PDAF at a distribution of AFMA settings until you zero in on the one that most closely matches the CDAF position, and ensure that setting produces repeatable results.

The PDAF cycles aren't required. Once CDAF has found the optimum contrast point, the AFMA value can be determined almost instantly by evaluating the phase differential and then applying the appropriate AFMA adjust value to bring that differential to its minimum. This is one method outlined in Canon's patent.

15
EOS Bodies / Re: Patent: Microadjustment Automated
« on: January 13, 2014, 12:00:33 AM »
I'd have to read through the patent to see if there are any unique elements to it but generally speaking the idea of using CDAF to auto-tune PDAF has lots of prior art online.

Pages: [1] 2 3 4