So I played with this a bit yesterday. Set up my 5D3 with a highly contrasty scene - inside looking from darker room areas out to bright blue sky and fluffy white clouds. Set a middling manual exposure of aperture and shutter (can't remember exactly what, but say f8 and 60th), then took seven shots in 1 stop increments from 100 ISO through to 3200 ISO, all without changing the manual exposure settings.
Processed RAW images in Lightroom. Without any processing on import, the 100 ISO was very dark, while the 3200 was very bright. Then adjusted the exposure in LR (+3.0 stops for the 100 ISO, +2 stops for the 200 ISO, and so on to -3.0 for the 3200 ISO).
If the whole system (sensor, plus in-camera processing, plus LR processing) was ISO invariant, each image should have looked the same? Well they sure didn't.
The 100 ISO shot had MASSIVE noise in the pushed dark areas while the 1600 and 3200 ISO shots were not noisy in that area. The sky areas in the 100 ISO shot were nice blue sky and fluffy cloud detail. Same areas in the 3200 ISO shot was totally blown white, and almost as much in the 1600 shot. (White balance was also different at the same colour temp setting.)
After reading this thread and some other info, I had expected to see much less difference. And that would have made sense from the idea that the physical sensor's light gathering capability can't change based on what ISO you dial in. In fact, having seen the results of my simple test, I'm now wondering what the hell is going on in-camera to get this difference - even wondering if the camera can evaluate the scene, compare with your Av Tv and ISO settings and maybe adjust the settings a bit (without telling you or your RAW file) on the basis of an assumption of what you're trying to achieve. Seems unlikely, but I really am sceptical that such a difference would be evident just from the difference between the analogue and digital amplification steps.
Anyway, upshot is, if you got a 5D3, don't think you can set any 'ol number on the ISO dial and fix it later. Not quite sure how, but there's a big difference.