Adding a 24-70 to your existing outfit would add little to its capabilities, but would add a lot to its weight (especially if you go for an f/2.8 version). It would also add to the headache of deciding what to carry and what to leave at home, every time you go out. Sometimes less choice is better for your photography.
As for replacing your 24-105, that would be sensible only if you desperately needed the greater freedom from distortion (but for landscapes that is not the problem it would be if you were shooting (say) architecture), the Macro of the f/4 version or the greater action-stopping, background-blurring speed of the f/2.8 version.
Nevertheless, whatever Ken Rockwell might say (and he is no idiot, however much he might contradict himself, or change his mind, or rant about something, or hate Sigma for no good reason) a 24-70 is a very useful lens. 70mm and 24mm are hugely different angles of view. It is not always possible to "zoom with your feet" and in any case, doing that changes the perspective. Getting closer or further away with a 50mm lens does not give the same results as using a 24mm or a 70mm from the same spot.
When I returned to using Canon SLRs recently (after years of Leica rangefinders) I used only prime lenses (35mm f/2, 50mm f/1.8, 100mm f/2.8 macro). I quickly realised that a zoom would be more versatile and obtained a 28-105 f/3.5-4.5 USM. That is a wonderful lens, even if it gets a lot of bad press. It is light, focusses quickly and silently, covers a great range of focal lengths, and is almost as good in the corners as in the centre. It might not give the greatest resolution or sharpness, but if you focus properly and either keep the camera very still or use a very fast shutter, it can give terrific results.
Nevertheless I recently replaced it with the 24-70 f/4 IS USM Macro. I hardly miss the extra reach from 70-105. It is not so big a gap as it might seem. It is similar to going from 35mm to 50mm. The extra reach at the other end is useful, but by itself would not justify the change, especially as the new lens is not cheap, is much bulkier and not so nice to handle as the 28-105, and is getting on for twice the weight. It is a tiny bit sharper and resolves a tiny bit more detail, but that would not be worth the change either.
WHat makes it worth-while is the IS, which really does give the equivalent of 4 extra stops. For static subjects I can hand-hold it as effectively as an f/1.0 lens. The other great plus is the Macro mode. All over the web you'll see it criticised for not giving enough working distance. But if you are travelling relatively light then the ability to get excellent magnification (about 0.7x) with your main lens saves you from carrying a longer specialised macro lens.
There has been a lot of hand-wringing in this thread about covering all the focal lengths economically and practically. But you don't need to cover all the focal lengths. Recently I have travelled with just the 24-70 doing street and landscape photography and for those subjects I have never really needed anything longer. At first it felt odd, but I soon got used to it. I just concentrate on looking for good shots in the 24-70 range. If I really don't want to risk coming back without any images I'll slip a light backup SLR with the 50mm f/1.8 into the back pack.
And I get more good shots with just the one zoom than I ever got when I was younger and more foolish, and used to lug around a pair of film Nikons and 20mm, 35mm PC, 50mm, 105mm macro and 180mm f/2.8 lenses, or an even heavier set of zooms
