And some are more sensitive in cross or X pattern. This is good in theory but the target should be better defined and quite a bit larger than the ladder itself to prevent the camera from weighing on it. Otherwise it's just as good or better than Focal.
The latest version 1.8.1 failed on all of my lenses (5-15, 100L 70-200L II, 300 2.8L 600F4L and the latter 3 with and without 1.4XIII and 2XIII extenders) but ironically the older 24-70L. After letting focal do a quick cal as well as a full call on all of my lenses, a check with my spydercal revealed that focal had incorrectly calculated the AFMA on all but the 24-70.
This is quite disappointing as I would have expected an improvement in later versions and instead it has gone the other way. 1.4 seemed to give the best results.
Another interesting thing that happened with 1.8.1 when calibrating my 70-200 2.8L IS II is that on the 200 end at AFMA -20, focal gave it a higher result than at 0. The images were garbage compared to the same with AFMA 0. Even with removing those test points the AFMA calculation was considerably off. Again using the SpyderCal, -3 put the AF dead on 8 out of 8 shots.
To answer the OP question, yes AFMA matters if your lenses are not already dead sharp at 0 AFMA. And you would never know if you dont test them. Software and dot tune are not ideal. Both seem problematic to me and the only sure way to verify everything is to use SPyderCal or LensCal type device, homemade or not.
Then again, there's the cheap-o DIY version
A 2x8, a step ladder, and any kind of target (in this case, a cardboard box) - see setup.
I take a few photos, manual defocusing before using AF, and estimate the number of inches front or back focusing. Adjust the micro adjust and repeat.
But ... where is the camera actually trying to focus? You can't tell by the red box, because the aF area is larger, and it makes a difference based on exactly where the camera focused. A Camera sensor tends to grab horizontal lines in preference to vertical, and that can have a effect.
There can be lots of gotchas that can lead to wrong conclusions, or they could be right ones, the problem is in knowing which is which.