70-300L owner and loving it...
besides the 'main use' comparison (ie, wide open at f/2.8 for faster shots in lower light vs the extra 100mm for extra reach), think about: (from here
70-300L is 1050g, 70-200 2.8v1 is 1470. (that's nearly 50% more weight)
70-200 comes new with a free tripod ring (does your 2nd hand one? it will add even more weight though, to 1670g)
70-300 is 15cm long (20cm zoomed in), 70-200 is 20cm long all the time (does it fit in your bag or will you need a new one?)
70-300 takes 67mm filters (cheaper), 70-200 takes 77mm (or do you already own one size?)
70-300 focusses closer, better MM (70-200 has better MM with 25mm tubes though).
and 70-200 f2.8 will focus on any body with 1.4x or 2x extenders (to 280mm f4 and 400mm f5.6), do you own either already? 70-300 won't fit either canon-brand extender without smashing glass. apparently some of the Kenkos fit, which i intend to buy one day, but then they lose waterproofing ability.
the 70-200 f2.8v1 plus 1.4x TCv2 (f4 and f5.6@280mm) looks a lot softer than the 70-300 (f5.6@300mm) here
i took a lot of time deciding between the 70-300L and the 70-200 f4 IS, the 300 won just the tiny bit better IQ, lots more contrast, and the extra cost (and hassle) of extenders (i really did want the extra reach). so convenience is also a factor if you are going to want more than 200mm, even occasionally...