My 2 cents. I shoot video so the camera is generally supported, even for stills on my day iff I'll use a monopod at least:
F2.8 pays dividend for af speed. All EOS DSLRs benefit from F2.8 or faster lenses in terms of AF. A camera with great Af like my 7d becomes brilliant, a camera with mediocre af becomes quite good (like my 600d, in crntre spot select at least) regardless of actual shooting aperture.
F2.8 gives me an extra stop. Obvious. But for video when you are using a fixed shutter this can be incredibly helpful.
Depending on the tests you trust the non-is f2.8 is marginally sharper.
Yes its heavy, but if you are carrying a large 6d and an 70-200 f4 the difference isn't that much really. Why not buy an ixus instead?
Even handheld, the magnification of a 70-200 is such that even with IS taking care of the camera shake, subject motion blur is going to warrant a faster shutter anyway. Negating to some degree the IS.
One selling point for the f4IS of f2.8IS... Weather sealing, which the f2.8 non IS lacks.
My take might not be right for you, but it works for me.