@smithy : I'm using it on a 5D and a 7D so I've got a 40mm lens on one and a 65mm on the other I guess.
It's a weird one to say that I prefer the 2.8 to 1.4. Personally, I find the 50mm f1.4 soft and unpredictable at 1.4 and I tend to use it at 2.0 or 2.2. When I think of it like that then the 40mm wins the battle - as it's lighter, sharper and the difference between 2.2 and 2.8 is negligable.
I think I want the 135mm L, that's the real problem! Unless I ditch the 50mm then there will be no 135mm!
Softness at f/2 or wider was my experience too when I borrowed a friend's 50 f/1.4. If you don't like it wide open, then you'll be happier with the 135 than your 50 f/1.4. If you have something like the EF-S 17-55 or 24-70 already, then you're right that less than 1 stop won't make a big difference. I found the 35L and 50L to be much better wide open than the 50 f/1.4. If you're looking for better wide open performance near 50mm, then you'd have to look at the Sigma 50 f/1.4 and the Canon 35L and 50L.