I've owned all three of the canon 70-200mm f/2.8 (more than one of each model), and one f/4 IS. All are very good, but for pure IQ, I'd rank them like this. I haven't owned the f/4 non is, but it is not as good as the IS version according to all the lens testers.
Its really hard to go wrong, but if you want to use a TC at 200mm, I'd avoid the f/2.8 IS MK I.
1. f/2.8 IS MK II - Supurb
2. f/4 IS - Suburb (close Second)
3. fl2.8 Non IS - Excellent +
4. f/2.8 IS MK I - A very fine lens, weakest at 200mm