I own the 100mm L macro myself so I'm a little biased.
I'd go for the macro due to the IS. The IS system of the macro works wonder. It is really useful for tight portraits and whatnot. The AF on the macro is very fast (I've used it for indoor sports). The 100mm is a useful focal length and gives more distance between you and your subject.
The 85L on the other hand, is much heavier than the 100. The AF is very slow, so it is possible that you could miss an expression or force the model to wait a little bit before the picture can be taken. Also, given the ridiculously shallow DOF, even the slightest handshake can affect your image.