Yeah I agree that the stabilization effect is more noticeable with a 70-200 than with the wider focal lengths but with my 17-55 @ 17mm I was getting fairly sharp useable shots at 1/8th and that has an older version of IS. The 24-105 has pretty decent IS too. Even the old 18-55 kit lens did alright.
What I find is that while IS cannot truly replace a tripod or monopod it does reduce the amount of camera shake to the point where it is at least acceptable for web use. The shake is still there and likely most of it from the mirror which makes me wonder actually ... I know the 70-200 IS has an anti mirror slap vibration function, maybe the 35 doesn't have that so there is a limit to how effective it can be?
Also, it may be a personal thing based on how you use it. I do agree that I was able to get usable 1/8 shots with the 35mm IS, but the issue for me is that they were not significantly more usable than when I just shot with IS disabled. It wasn't like one was razor sharp and the other was terrible, generally I found both were usable but not super sharp. Again, unlike the 70-200 which can be super sharp several stops down.
I thought IS was making a big difference with the 35 IS and slower shutter speeds until I turned it off and shot slower shutter speeds quite a bit without it, then I found the difference was not really significant unlike the 70-200. This of course was real world shooting and not myself trying to purposely induce shake into the shot to see the performance of IS. There was a big difference with a monopod, tripod, of course.
I am not sure the reason why for this, but the 35 IS is supposed to have the same advanced IS system the 100L does - which was supposed to be more advanced than the one in the 70-200. So, my guess is that Canon's IS system can only do so much when the shutter speeds get really slow, even if it is only 2 stops down... Which does sort of make sense if you think about the way optical IS works.
However, I do frequently use the IS in the 35mm to do panning shots. It assists in keeping the vertical axis stable and I have gotten some really cool and crisp panning shots in doing so. The 35 f/2 IS USM is my go-to panning lens due to the IS and flexible wide-normal focal length, I will pick it over the 24L, 50L, 16-35L II, 24-70L II, etc... I don't shoot video really so that part is less useful to me.
It is of course possible Canon will improve up on the system for the 16-35 f/4L IS - we will see. Personally I won't be able to test this lens though, as the 16-35 II is a better fit for what I shoot.