One thing you could do while pondering what to buy is attach your 17-40 to your 50D, set it to the equivalent of 35mm on FF (a bit less than 24mm) and spend a day wandering around using only that focal length and see how restricted you feel (if at all). I would find it too restricting, but that's me (I prefer details to scenes, so unless I expect to be inside my standard walkaround lens is more often than not my 70-200 f4 IS or 70-300L).
Regardless, though, I think buying the 24-105 as part of a kit makes the best sense financially, for all the reasons given so far by others. On FF you can get very nice background blur with it, by the way (though you'll want to be near 105 for optimal effect). Note that while the 35mm Sigma will make excellent background blur, you'll have to get pretty close to your subject, far closer than I would ever want to get to people (but again, that's me). If you do get the 24-105 first, wait and see how restricted you feel by *that*. You may be surprised at how well it performs, even in very low light (esp. on FF).
(I wouldn't overlook the new Canon 35mm IS either unless you're trying to minimize depth of focus.)