I have just bought the 70-300L to replace my 70-300 non-L. I can't emphasise enough how much I think it is worth upgrading to the L version (except to say I just registered so I can post this!)
The L version is much sharper at 300mm, the CA is better, contrast and clarity is much improved, IS I'd say is marginally worse but nothing you'll miss, it's manufactured with much tighter tolerances so everything fits/moves better (my non-L feels wobbly, especially the barrel). Most significantly at 300/5.6 I took two photos of the same target, at the same distance, under the same lighting, with the same settings (1/40) using both the L and the non-L version. I found that not only was the L lens yielding a much sharper image but that it produced a much brighter image too. In fact, to get an image of equal brightness I was able to shoot with the L at 1/60 instead of 1/40. So, although it's not a proper test, I'd say it's a brighter lens too.
I like to photograph birds with my 7D so having a lens this much sharper @ 300mm is great. I tried a friends 100-400 and although it's a good lens, I found it slightly too cumbersome to carry round. The 70-300 L is heavier than the non-L but it's only 400g or so. Unlike the 100-400, it doesn't feel too heavy to have hanging off the camera mount or to have hanging off my Blackrapid strap. I find it quite portable. If all you're used to is the plastic entry level cameras/lenses it will be quite different but you'll get used to it.
In short, get it!!!! You definitely won't regret it!