I would go for the 70-200 because I don't like the 24-70 (soft at 2.8!). Just sold mine and bought the 35L, couldn't be happier.
I have the cheap 70-200 F4 NON IS and I love it, it's great for sports, nature, portraits. I would like the 2.8 IS II, but it's not my first priority (might buy the 135L instead). But it all comes down to what you want to shoot, something wide like landscape and group portraits or something with a bit tighter view to isolate a subject for portraits, sports, nature.