The TS-E 24 is not that big of a deal to shoot if you know how to use it. If you are taking a shot without tilt or shift, it is no different than shooting a non-IS lens manually. Think of it as a large, sharp, slow prime that can also be used as a tilt-shift.
I recommend using a tripod for landscape and architecture shots, if for no other reason than to utilize the amazing edge-to-edge sharpness of the TS-E. If you have any non-IS lenses, shoot one for a day manually and determine whether you would be willing to do this. I drink a lot of coffee too, and it is a non-issue for me.
I also advise staying away from the Rokinon because their quality control is terrible and the lens simply isn't that sharp.
It takes 82mm thread filters, so factor this in.
+1. Shift is relatively easy to do handheld, precise tilting of the focal plane is best done using LiveView and a tripod. Using the 24-105 at the wide end using manual focusing for landscapes/architecture is also a good idea. Drop the ISO down to 100 to maximize DR/sharpness and stop down the aperture to f/11 or f1/16 to maximize DOF. If you are ok using this workflow for landscapes and architecture, then the TS-E will be a great addition for you.