....all true...and then there is that 70-200 f/2.8 II if you really need 70-105 mm range.Of slightly more concern are the ~30% of shots taken between 70-105mm
I would think that the higher sharpness at 70mm, especially stopped down to f/4, would give you the latitude to crop your photos to at least 85 or 90mm and still match the sharpness of the 24-105. This starts getting pretty close to matching the 24-105's usefulness as a walkaround lens, especially if you're not worried about shutter speed.
I have been having similar thoughts and a potential plan for myself is to up grade to the 24-70 f/2.8 II in about a year and sell the 24-105. My concern is at the other end, I like to use IS and the equivalent (just moved to FF from crop) of 24 mm to hand hold waterfall shots. Of course, my other thought is to keep the 24-105 as a general purpose/travel lens and buy primes.