They're both great lenses, and the 16-35 II is the only zoom I have left out of the ones I've had in the past (70-200 2.8IS, 2 different 24-70mm). But the 50L was the first Canon lens that I truly loved. It's easy to "blow it" with the 50L, but when you nail a shot you really, really nail it. The 16-35mm is great but not that sharp wide open. You may think that the 16-35mm is more versatile since it's a zoom, but I'd say the 50mm may be just as useful (if not more). They're really two different beasts. The 16-35mm is great for landscapes, 50mm is great for just about anything.
Now if you can live without AF, you should seriously consider checking out the Zeiss ZE 50mm f/2 Makro. I was shocked at how much sharper it is than my 50L, I mean night and day difference, especially on the edges. I love Zeiss glass and have been trying to bring myself to get rid of all my Canon glass for almost a year so I can make a complete switch over. But I have sentimental attachments to my L lenses so it's been tough.
Either way I think you'd be happy, but I feel like I've gotten some really special images from my 50L.