My frustration with the program is beyond words. So I just keep putting out there that when you use FoCal, you still need a way to verify the AFMA such as with a spydercal. Dont "trust" that FoCal has worked as expected. You need to verify what it's produced and at that point you might as well just use SpyderCal or LensCal in the first place.
How do you verify your SpyderLensCal results? Personally, I never found that 8-10 shots were enough with a LensAlign Pro.
As I said, I get equivalent results with FoCal and LensAlign when I do compare them, but the LensAlign is a lot more work. Perhaps it's that digital image analysis (of microscopy images) was part of my day job for many years, but I know that a properly-written algorithm can pick the sharpest image from a group better than a human eye, particularly when the images are close. Gauging the edges of the DoF on the angled ruler was not always obvious (particular with slower lenses at longer distances). I'll also take a quantitative analysis over a qualitative analysis any day of the week.
I'm honestly not sure why you're getting inconsistent results with the setup you describe. But perhaps I am getting consistent results because of how I use FoCal - for image analysis only. They do seem to have worked toward predictive algorithms with each release, so that fewer shots are required per test. It may be that their algorithms aren't all that predictive - undersampling with a system that has both systematic and random error can easily lead to erroneous results. Since I got the 1D X before there was any FoCal support, I was forced to use Manual Mode - and I found it easy and very robust. I still do it that way. Granted...I know that I'm oversampling (a lot), but my curve fits are solid, and the conclusion is visually evident. Before FoCal, I wrote a Matlab script to do a similar analysis - FoCal is easier, with a nice front end GUI (something I have no idea how to code), and it works on jpgs instead of requiring conversion to tifs.
My setup is pretty simple - I leave a target taped to the basement wall (I've got enough room down there for up to a 300mm lens; outdoors I tape a target to the fence), set up the camera on a stable tripod, align as best I can (using the camera's level and my eyeball alignment), and shoot two shots per even AFMA value over |10| (defocused in opposite directions) and three shots from -10 to +10 (two defocused opposite then one without defocusing). Setup takes less than 5 minutes, taking the 83 shots per test takes about 10 minutes. It's quite obvious what the correct result should be, based on the curve. In the software, I can click on any point on the plot and see the image from which the data point was derived. What's to verify?