I'll share some of my experiences evolving and perfecting a hiking setup.
I hike what I would consider moderate length day hikes in varying terrains. I hike in the Appalachians in the fall and winter, and in Texas, Colorado, and Utah in the summer. I don't pack anything non-essential and aside from camera gear, that typically includes some food, water, a rain shell, first aid kit, portable binoculars, multitool, hunting knife, pepper spray, sunscreen, etc... As far as camera gear, I've been through several gyrations and combinations and various packs to carry it all in. I have just returned from 10 days in 4 states out west and did several 3-6 hour hikes with this setup and it works wonderfully.
I use a Black Rapid strap, over the left shoulder with the camera attachment hanging from my right side at arms length. Over that I wear my Lowepro Photo Sport 200AW. In the camera compartment (2 pockets) I carry either my 16-35mm f/4L or a 430EX II flash in the smaller pocket and my 6D with the 24-105 attached in the larger pocket. If I carry the flash, I carry my 35mm f/2 IS and 15mm f/2.8 fisheye in the upper larger compartment, but in Lowepro lens cases, which can be easily attached to the waist or shoulder strap of the pack. Access to the compartment for the lens is as ease as reaching behind my left side, unzipping the compartment with my left hand, pulling out the 16-35mm and swapping it with the 24-105mm and placing it in the compartment and zipping it back up. No need for the gymnastics of swinging the pack around the left shoulder to access the compartment (though this is precisely what I do if I want to retrieve or replace the camera into the pack, as it's easier). The camera hangs at my right hip, and I often just carry it in my hand by the grip, or lay it across my leg and use my hand to keep it there when needed.
The pack, loaded, with water bladder weighs in at approximately 12-15 pounds. Everything is easily accessible, I have mobility, and most importantly, it's comfortable.
I've spent the last 2.5 years with no less than 4 different attempts to get the right pack, the right combination of gear (body and lenses) and be able to hike comfortably with ease of access.
I realize this is a bit more detailed information than what the OP asked, but in reading the thread of conversation, there were tangents about weight and comfort and I simply wanted to share my experiences. If anyone would like, I'd be happy to take some photos of my setup, how it's packed and how I wear it.
In short, the combination I use most when hiking is the 24-105 lens on either my 6D or 7D. Which camera I select depends on if I want the slightly longer reach or not. I think the 35mm f/2 IS is the perfect companion to the 24-105mm for it's low-light capabilities. If the OP wanted to just carry one lens, it might be worth looking into a superzoom, like the Tamron 28-300mm. I've been considering that myself, but don't want to compromise image quality over what I have now. There are times where having that kind of range would be great.
Hope this helps.
Greg