For something concrete to discuss, how much different can a picture be with a Canon 40mm pancake vs a Canon 24-105mmL? The 40 will take you down to f/2.8, but that's only one stop off the f/4.0 of the zoom. And the zoom has IS while the pancake does not. I'm seeing few instances these days where the 40mm gets mounted to my 5D3. The 24-105 usually gives me adequate results -- and a lot more versatility.
If you are looking at the centre of the frame (on FF) at apertures in the f 5 - 8 region then there is no perceivable difference, but as soon as you move out towards the four corners of mid frame there is a huge difference, and this makes a noticeable improvement on a landscape photo. On a zoom such as the 24-70 II you wouldn't see the same difference, and if you just had a central subject with the 24-105 then you wouldn't see the difference.
It depends on the application. Also the 24-70 II is a monster in order to achieve the same IQ as a tiny prime.
I understand the OP question, but I think it is possible to be disciplined with a zoom and work along as if you had primes.