A lot depends upon how you would shoot with either. If you're using a stable tripod, mirror lockup, and a wired remote in good enough light to use very short exposure times with IS turned off (which is pretty much the way most labs test lenses) the difference will be much more noticeable than if you're handholding in poor light with slower shutter durations and IS actively engaged (which is pretty much the way many of us use 70-200mm f/2.8 zooms).
If you've got more blur in the image due to camera movement, subject movement, or IS compensation than the difference between the two under ideal conditions, then you won't notice much, if any difference. So whether you'll notice any difference is up to your shooting technique.
Just an aside about IS (whether IBIS or lens based): When the optical alignment is altered to compensate for motion the absolute image quality is also affected. With lens based IS it's actually an intentional slight misalignment of the lens. With IBIS its pulling the center of the sensor away from the center of the light circle and towards one edge, which will almost certainly be worse than the center in terms of optical aberrations. The reason we still use IS is because the blur and aberrations introduced by IS tend to be less than the blur introduced by camera motion if IS were not used in situations where IS is helpful.
If you're shooting in conditions where IS is beneficial, you'll notice a huge difference. If you're shooting in lab conditions you'll notice a significant difference. If you're using good handheld shooting technique in daylight at f/5.6 or f/8 and still getting shutter durations of 1/1000 or shorter you might not notice any difference.
If you're on a tight budget I'd recommend the Tamron 70-200mm f/2.8 Di VC USD G2 over the much older original EF 70-200mm f/2.8 L (non-IS). Also get the Tap-In Console USB dock and use the Tamron Tap-In Utility software to dial in the lens to you specific camera body. They've got pretty good tutorials on how to use it at the Tamron website.