I agree the loss of 1-2 stops can be costly in shutter speed or ISO performance in some situation. Originally I had started to shoot with circular polarizing filter whenever I shoot outside in the sun. I since then dialed back on their use to only use them in some situation, mostly for landscape or when I dont need to "freeze" the action.
With the polarizing filter on, I missed several shots where I needed to a higher shutter speed...They are a good tool, but I would not recommend them on all the time.
1DX, 24mm f1.4L II, 50mm f1.2L, 85mm f1.2L II, 135mm f2L, 24-70mm f2.8L II, 70-200mm f2.8L IS II : D800, D4, 24 1.4G, 35 1.4G, 50 1.4G, 85 1.4G, 24-70 2.8, 70-200 2.8 VR II