I have, just as you, given it a thought whether or not et would be "better" to have the 24L II or the new 24-70.
For many purposes you probably would be better of with the zoom, as it is more versatile. And with todays crazy-high ISO levels, getting enough light, might not be as big of a problem.
Of course i know there is a difference between large apertures and high ISO's. But that's one of the things you'll have to decide with yourself, do you use your 24mm a lot at f 2.8 or faster?
For me there is a whole other aspect which haven't been mentioned yet.
As a street photographer, the feeling of using a prime. Getting to know it. Knowing it's exact framing without looking through the viewfinder, or at least only for a breif moment, is one of the things i enjoy most. This lets me take pictures faster then people get irritated by some random man(me) talking pictures of them. Which in some situations, means getting the shot.
Same goes for the size of the prime, which is not so aggressive and provocative, as it is often smaller.
So, which is most convenient for you?
Do you like walking around all day with one prime or are willing to have multiple lenses with you?
Last but not least, can you afford buying three primes, like a 24, 50 and 85, to replace one zoom lens like the 24-70?
I love my primes, and i will stick with them, they have something special upon them, which a like, including huge apertures
Just my 2 cents, hope it helps solving you matter.