In the same boat... What i did was look at what focal lengths i use the most (using LR).... For me, 60% of my shots with my 24-105 were at 24mm, 25% at 105 and the remaining everywhere in-between.
I suggest doing that! It can be very interesting.
I'm picking up a 24mm Prime. I have a 50 1.8, which I'm sure one day i will up-date, but on the long side of things, my 70-200 is doing me nicely...
My primes will be:
24, 50, x.
Another reason for 24 over 35.... you can always crop a 24 to 35 easily, but the other way round is more annoying (stitching)...
ps - I am also picking up the 40mm 2.8... so my need for a 35 is extremely low now!