35mm. It's more versatile. The 50mm is such an overrated lens not to mention not a good portrait lens.People's heads will get distorted when shot with the 50mm (unless you're going for the distorted head look).
I agree with Peteroc. Keep the 35mm and get either the 85mm or 135mm.
Either you've never used one or you're... Im going to just go with you've never used one. I own and use all the primes below 135mm. They all work well for portraits, especially the 50mmL, 85mmL and 135L. They are tools, when you know how/when to use them they work fine.
Since moving onto the 5D3... the 50L is my go to lens for most shooting environments. The 35L was my fave lens until my first 5D3. The 35L is a great-walk around lens but now I gravitate to the 50L. 50mm is a great PJ focal length, and has been for many years. It also excels with portraits but… again, put it in the hands of a monkey and YMMV. Sure you can buy the 1.4 but it has a notorious habit of taking a crap when you need it... shabby build is why it's such a deal. If you don't mind that, then go for a 1.4. If your looking for a workhorse tank 50mm, the 50L is your lens.
50mm f/1.2 on a 5D3...

Nikkei Matsuri, San Jose 2012... by David KM, on Flickr

Robinson... by David KM, on Flickr

Lovers at the park... by David KM, on Flickr
I've used to own one and sold it. I notice you stand quite a distance away from your subjects. Get in closer and shoot just their heads and watch their heads get distorted and how unflattering they become.
The 50mm is not a close up headshot lens, knowing the limitations of the lens can still lead to great portraiture.
That's like saying I hate my 135mm for portaits because I need to step back too far to include the surroundings.
The 50mm is a great environmental portrait lens, great for including surroundings and giving more of the story, not all portraiture are just headshots, as you can see by those great examples!