I've been asking myself a very similar question: get the 24-70mm II or the 70-200 2.8 II? I've never had prior versions of both, and my current collection of L lenses includes the 17-40 f/4, the 50L and the 100L macro 2.8 IS.
After hearing all sides of the responses, I can surmise that it really depends on your subjects, expected shooting distances and typical lighting conditions vs available light.
Here's a question to throw into the mix: if the OP wants flexibility of the 24-70 with some extra reach but without the needed bulk, what about investing in the 24-70 II + a 3rd party extender like the 1.4x or 2.0x (Canon's own extenders are not compatible)?
I'd of course presume that with this combo, the excellent sharpness, lack of CA and slight distortion of the new 24-70 will all be degraded slightly with the extender, but the penalty's not that severe hopefully. Extenders from makers like Kenko or Sigma are suggested. Anyone has actually tried such pairings before?