I am asking myself the same thing. So I asked.
Not too many answers though.
Those answers are fairly representative for those that have kept the 50L. It is true that the 50L does not outresolve the other EF 50mm AF options by a large/significant margin (see 50mm shootout article by LensRentals). The reviews use MF/LV focusing, which does not take into account AF performance, which was a big negative of the Canon 50 f/1.4 I used. The 50 f/1.4 AF was inconsistent wide open to about f/2.8. It was accurate at f/2.8 but if I was using it stopped down so much, I'd opt for a 2.8 zoom. The 50L is much more consistent, which is why people that shoot wide open a lot tend to drift toward the 50L. It also does better on better AF Canon bodies (i.e. 5D III). Are images softer at 100% than the 35L and 85L wide open? Yes, but no 50mm EF optic is going to do better wide open (f/1.2 or f/1.4). Are the colors and bokeh better? Yes, the images look like L images and the bokeh is a bit smoother.
Is it worth it? For most people, then answer is no. It should not be a user's first fast L prime, and definitely not a user's first L lens. The 24L II, 35L and 85L II are all more forgiving than the 50L. It is not a good landscape lens (there are plenty of cheaper and sharper options). It is not a good detail lens (not a good performer at MFD if it has been AFMA'ed at typical portrait distances). However, it is a good portrait lens.