I know you're asking about lenses, but how are you doing about lighting? You mentioned your 580exii, but portraiture is one that usually calls for more complex set-ups. We all know pictures are made or break by their light.
TL;DR summary: Lots of options, depends on your style and what you want to prioritize.
I agree with this. I'd say at least one more speedlite to be used off-camera would be huge. That probably also means getting a light stand, umbrella, some flash gels... the works. At least for formals.
As far as lenses go, a 100L is good for ring shots and other details. Other than that, while it does make a very nice portrait lens, it won't do anything your 70-200 won't (although it will be a bit sharper). The 135 is bokehlicious for portraits, and some other fast primes (even without spending on L lenses, like 50 1.4 or the sigma 30 1.4) could be useful for low-light situations.