Note: Of these lenses, the 85 is the only one I have, and I've not borrowed the others. Just saying what I'd do if I was in your shoes...
If it was me, I'd sell the 35L, buy a sigma 35, and use the leftover money for an 85 f/1.8 for portraits. I'm just not thrilled by any 50 on the market right now, but the sigma 35 looks awesome, and I've had a blast with my 85 1.8, which is a great buy for the money.