Used a Canon and a B+W filter on my 17-55 and never had an issue with either of them. I'd suggest trying another filter brand (at a camera store or a friend's) just to see if it is a lens issue or not. Tolerance stack ups vary from lens to lens, so yours might be more sensitive naturally to filter variation. Another consideration is that longer lenses have longer focal lengths which may lessen the impact of bad filter. There are many videos showing that lenses with slightly damaged front elements don't perform noticeably different than those with perfect front elements. However, at the shortest focal lengths (i.e. fisheyes), dirt on the front element can be visible. You might be able to test this hypothesis by seeing if the image IQ hit is worse at the wide end than the tele end, but in any case, I'd suggest trying another filter.