It sounds like you're really debating between two lenses, not three. I traded up to a 70-300L from a 70-200 f/4 IS. Yes, up. After testing both side by side, color, clarity, contrast, IS, size (packed) was better on my 70-300L. Both lenses are f/4 at 70mm and my 70-300L was sharper there. Both lenses are 5.6 at 280mm and my 70-300L was sharper there. One lens has snappier AF at a native 300mm, the other doesn't; one is sharper with no TC-induced flaring/veiling at 300mm, the other isn't.
Maybe my 70-200 f/4 IS was substandard (I don't think so), but my 70-300L was better at most things with no need for an extender. So I kept it instead. If you can test both side by side, that would be best. They are both very good lenses and choosing between them is very difficult.