Well it's been interesting reading all the comments and today was judgement day. I finally got to try the f2.8 against the f4 and whilst it wasn't ideal conditions (indoors under fluorescent lighting with some daylight from the side) I could at least take "like for like" shots. I won't post images since there are plenty sites showing comparisons made under more scientific conditions.
My (admittedly limited) findings:
1. The f2.8 is much more substantial in size/weight than the f4. The f4 is quite dinky in comparison! As I often use the 70-200mm as an "always in the hand rather than on a strap walk-about" lens I'm conscious of the fact the added weight would be an issue after a while. Hand cramp is a cruel mistress. (Cue comments about the best strap to use.)
2. Focusing with the f2.8 was maybe
a bit quicker but not definitive. We're talking tiny fractions of a second here and my perception - you want to believe a newer, more expensive lens will focus quicker.
3. Sharpness (top requirement in my view) - yes I've pixel-peeped (not a crime, yet) and I'd say the f4 is a tiny bit sharper at like-for-like apertures. I think if you saw two images side by side you'd be hard pushed to say which lens took which shot. Yes both are a bit soft wide open but once you get to f5.6 and smaller they are both excellent. Sure you can use micro adjustment but I didn't have all day to play.
4. Colour - both allow you to take colour images
No obvious fringing horrors. Both were good enough for me (I'm going through the obligatory convert-everything-to-black-and-white phase).
5. IS on the f2.8 is much quieter than the f4. Shhhh!
6. The f4 has more vignetting (duh!) but nothing that would ruin any shot. The miracles of modern software can cure many ailments.
So I reckon the f2.8 at twice the price and twice the weight is not necessarily twice the lens. However if I couldn't live without that wider aperture I wouldn't shed a tear as I parted with the money. The f2.8 is very good but not essential to me. Your mileage may vary. Contents may settle in transit. Offer void in Utah.
At least now I can put that money aside for the 42mp 1D XMF due in Feb 2013 (oops, did anyone see a cat getting out of a bag?).
If you feel strongly that I've missed a point or my opinions are complete poop, I reserve the right to ignore you!