Regarding the title of the post:
I think people are getting a little caught up on the number of lens elements in the optical path, which is really a separate issue to how IS works (in Canon lenses at least). Yes, IQ is affected by the number of lenses in the path, but bear in mind that multiple lenses are used to correct multiple optical issues. If you didn't have those lenses, then spherical or CA (or other issues) would degrade the quality. Its a balance.
IS on the other hand, is done by moving an element or elements on and off axis. This is going to create aberrations such as astigmatism. The effect would be much greater in lenses that are fast, as the latitude for astigmatism would be much smaller.
As an example, for those who wear glasses, hold them an inch or 2 in front of your eyes and move them around. As you will see, the image moves accordingly. This is, in effect, the same approach as IS in a Canon lens. Pushing it further, while holding the glasses in front, move the glasses so you are looking through the outer edge, not the center. In some cases, the IQ will drop, and the image blurs. This is the extreme example of an off axis lens.