I'm debating the same thing. I own a 24-105 and am relatively happy with it, but from reading the reviews and hearing testimonials here, its clear that the Canon 24-70mm 2.8 II and Tamron 24-70 2.9 VC have superior IQ and sharpness. While this is just a hobby, I'd like to have the best equipment possible for what I shoot the most (landscapes, family and portraits).
Since I have a 70-200mm 2.8 II and 85mm 1.8, I have the 70-105 range covered (albeit with a lens change...). The vast majority of my pictures with the 24-105mm are landscapes at the wide end (24-35mm), with a scattering of shots at various other focal lengths. Not that many between 70 and 105.
I think I'll probably pull the trigger and purchase one of the 24-70's at some point, but I need to figure out which one first.
Good luck with your decision.