A lot of the unique 'look' a lens gives is with the bokeh, how the back/foreground is rendered. Almost all the L lenses produce great bokeh, smooth with good CA control and not too much 'distortion', although this isn't necessarily true with every lens. Beyond that there's color reproduction, the L lenses bring out good colors, like reds and blues, though that's not considered so important these days as so much can be altered in post for color.
Then contrast and sharpness do help, but yeah, they don't do much in giving a certain look.
But, other lenses compete, or are just as good as the L primes. Try telling apart Sigma's 85mm to the 85L, or Sigma's 50mm to the 50L, they're very close in overall look, the only real teller is that both Ls can go F/1.2, and that has a special feel to it, and super shallow. Really, pros these days will choose L for the build quality, weather sealing and reassurance that they will work and get the job done. Sharp all around and will last a good while