I own the 85mm f/1.2L II and I can't really complain. When I bought it, I knew going in that the AF speed would pale compared to the 135mm f/2L, 24-70L (I or II), and the 70-200mm f/2.8L IS II, or even my Sigma 50mm f/1.4 (all of which I already owned or rented).
Most of the time, I find the AF speed to be perfectly acceptable. Though my uses are usually in situations where I can "pre-focus" (focus, then recompose and focus again when my subject moves slightly). If the lens only has to hunt for focus in a very limited plane (hence why I pre-focus), then I don't notice much of a difference between considerably faster focusing lenses. But if it has to hunt across the entire focal range, then it's quite slow... definitely the slowest lens in my bag.
But I didn't buy this lens for its AF speed. :-)