This is a really difficult question indeed CHL! I have the mk II version of the 70-200 2.8 zoom and the IQ is preatty much on par with the 135L, minus the extra stop that the 135 provide you. I am getting the 135 anyway because I am looking for something lighter to carry.
I dont have experience with the version I of the 70-200mm zoom but I am told (and you cant read in these forums) the current version was a big improvement in the IQ. If you already find you use the 70-200mm zoom a lot for your purposes, I would think you might be better getting the mk II first instead of getting the 135L.
Keep in mind if you buy it before Dec 31st at B&H you get $400 off! ($1974). This is a good buy! (the link is on the main CR page).
Hope this helps.
Fabulous on the 5DII, the 135L is that bit smoother than the 70-200 II. IQ of the 135 is better but not with the huge gap there used to be. The 135 makes a good street lens as it doesn't leap out at you and is surpisingly small. Minimum focus distance is small too!
Thank you both. I am going for the 135L because of less weight and it is more discreet compared to the 70-200 2.8. I have also come to the conclusion that the 35L and the 135L will make a very good combo for me on my 40D and 5DMKII. I live in Europe so I can not take advantage of any good deals with B&H - unless I travel to New York which is not a bad idea in itself :-)
Congratulation CHL, you will not regret the 135L. Now that I have it, I wonder how I could live without it for so long! Cheers,