If I could only choose one lens to do photography with, it would most likely be a 50mm f/1.4, and ideally the upcoming 50mm f/1.4 ART.
However, the truth is I am blessed to be able to choose any lenses I want to. And the fact is that I have used the current Canon 50mm f/1.4 almost never in the past three years, despite shooting half a million photos each year of a wide range of subject matter. There is simply something more effective for almost every niche and specific photographic circumstance than the 50mm.
When I look at the lens lineup that you have right now, I believe that the 135mm f/2L will add much more photographic power to you than would the purchase of a 50mm f/1.4, even the ART. So the 135mm f/2L is my strong recommendation to you, especially given your penchant for portraits.
5DIII, 5DII, 7D x5, 6D, T2i, T3, 1D X, 10-22mm, 16-35mm II, 18-55mm II, 18-135mm IS x2, 70-200mm f/2.8L II, 24mm f/1.4L II, 50mm f/1.4, 50mm 1/1.8 II x2, 85mm f/1.8 x2, 100mm f/2 x2, 135mm f/2L x2, 200mm f/2.8L II x2, 1.4X III, 2.0X II, 60mm f/2.8 Macro, etc. only had room to list a few Canon items