Wait, wait, wait. Why would you sooner give up the 50mm 1.4 than the 40mm 2.8 if you are not doing video? If just for stills, I believe the 50mm is about as sharp as the 40mm at 2.8+, and of course you have the option of going to a usable 1.4 and a quite nice 1.8-2.2...
If you absolutely must make a decision because of finances, yes, you have to think hard; otherwise, you could buy your 24-70mm, use it for a month, and then you will really have a good feel for which of your lenses you want to give up.
As for me, I got the 24-105 for what seemed a steal during a kit sale, so I kept it after getting the 24-70mm. I was sure I'd want to sell it. But I use the 24-105mm for daytime outdoor events, for portrait sessions indoors where I'm using lights and want to work fast, and for when I scout a location for an upcoming session. I reduce the wear and tear on my 24-70mm, reserving it for lower-light, absolute best IQ during paid or portfolio sessions.
Even though I got the 24-70mm with a double rebate, I still see it as a big investment, making it hard to give up the 24-105mm.
Finally, if you don't shoot below f/2.8, period, giving up the 17-40, nifty 50, and the 40 makes sense.
[Yay! 100 posts!]