If you're considering a three lens kit, then the 24/35/50 is a bit more natural than a (24 or 28)/40/50.
For architecture, a 16/17 lens is useful indoors and a 24 is useful for larger rooms and buildings, which favors your 17-40. If you're going for architecture, I'd suggest bringing your 17-40 over getting another lens even though it is a bit larger than what you're considering. The 17-40 will also be more flexible for landscapes.
For street, I prefer a 35 if I'm only going to use one lens. If you opt for the 35 f/2 IS, then you might be able to leave your 50 at home a lot of the time because f/2 would also work well in low light situations. You might want to consider swapping the 40 for the 35 f/2 IS, and seeing if that one lens will satisfy your needs for street photography. If you need wider, then you can consider the 24 f/2.8 IS and if you need longer, then bring along your 50 f/1.4.
FWIW, I recently picked up the 28 f/2.8 IS. It's nice: the colors are good and it's sharp enough. The IS is less useful than I thought it would be. I need to drop the shutter speed a lot before I can see IS making much of a difference (i.e. 1/15 or slower, at 1/30 I'm getting similar IS on/off results). I chose the 28 because I have plenty of options at 24/35/50 and I got it during a sale for 350.