Well, first, I think there is a big misconception that f/2.8 lenses are "better" than f/4 lenses. This couldn't be further from the truth. As one example, look at Bryan Carnathan's ISO Charts and compare the 70-200mm f/4L IS vs. 70-200mm f/2.8L. The f/4L IS is sharper at all f/stops center frame in all comparisons, despite the latter being an L lens as well and an f/2.8 lens. The problem is that people think that just because a lens has a wider aperture it peforms optically better and this is just not the case. Yes it would perform better in low light, but guess what, they make tripods. I've shot many, many low light images with f/4L lenses on a tripod. Shutter speed and aperture can both be traded off depending on f/stop. I think the 24-105mm f/4L and 70-200mm f/4L IS lenses are superb lenses and I've shot some of my best photos with each! I am still amazed at how sharp the 24-105 is at f/4 during daylight. Those lenses are fantastic. Also, the 17-40 is fantastic. Even if you get the 24-105, keep it. I know this is not your scenario here, but I knew a pro who didn't buy the 16-35mm f/2.8L II because he saw better value in buying the 17-40mm f/4L and the 24 f/1.4L instead. Since you have a 35 f/1.4L, there's not much difference here. With your lenses, you will shoot fantastic images at a wide variety of focal lengths. If funding is available, I'd say try to keep the 100 macro too. If not, oh well, you're 70-200 will cover it.
