elford,
I will concede all of those points and more, after the intended purpose for the lens is well defined. If the need is shallower depth of field sure, low-light where a prime length is ok without lens switching/carrying too much equipment, sure that too. But then once the needs are well defined, the even faster, more expensive lenses, or rentals of them if they are only occasionally needed, might be better options. Also, while I agree with you completely, the 24-105's added IS does give it some extra low light capability if your subject needs can still work with slower shutter speeds, and in many cases they can. That's why I am suggesting to push the existing lens as far as possible before getting anything else. Actually, upon determining needs, with the limitations/requirements of intended purpose, an EX Flash might be a better purchase than a faster prime lens of either the highest end price and quality or the middle/cheap of the road step.
Also you are probably even more correct than the various reviews available. I wasn't in low-light, but a borrowed 50mm f/1.4 shot at f/1.4 was excellently sharp to me. I do hear (again from these reviews) that the f/1.2 preforms better until f/2-f/2.8 so again if the need, is low light, perhaps the greater investment or a flash is a better solution than getting something without knowing why you need it first.