I had the same dilemma when I picked up the f/2.8. I ended up keeping the f/4 because it's just as sharp and WAYYYYY easier to carry when you don't need the speed/DOF. I love them both.
I will also have a 70-200 f/4 is. But, I can't wait to get the f/2.8. And to think, the price actually beat the refurb (after rebates) price and was less than better condition used lenses. I'll finally have a new resting place for my extra QR plate!

For now, I will keep both. I have had the f/4 for a while and it is substantially smaller and lighter when the 2.8 is not needed. I figure, if I want to sell the old one down the road, the price should hold up for a while.
Oh, and I also concur. CPW is a great (albeit dangerous) tool.