I got back into SLR photography with the 10D back in 2003, and bought a (then recently-announced) 17-40L within a month to replace the 24-85 kit lens. I paid $770 at a local retail store, and got a very good copy. Used it on the 10D and then a 20D as my walk-around lens, and was quite satisfied with it.
When I moved to full frame with the 5D in 2007, I found the 17-40 less impressive in the outer edges of the enlarged frame. Still, I continued to use it, as I didn't need anything faster, there really wasn't a better quality UWA zoom available, and I don't shoot UWA all that often.
I sold that 17-40, still in excellent condition, with all original accessories and packaging, about 2 years ago for $600 on Craigslist. The buyer contacted me a few weeks later (after he had taken the lens with him on a trip to Europe) to tell me it was one of the best transactions he had ever had on CL.
Six months later (end of 2014), I picked up a new 16-35/4L IS in a Canon Price Watch sale for $996. It's still my least-used zoom (I confess to a bit of G.A.S.), but I find the experience of using it more satisfying when using its IS, and the results appreciably better than with the 17-40.
To the O.P.: If you can afford it, and you're going to keep the lens "forever," go ahead and spring for the 16-35. If funds are really tight (since you have your 20-35 to shoot with in the meantime), wait for a sale, a rebate or a refurb. If you go for the 17-40 and then find yourself displeased with the quality of images you get from it, you'll eventually move up to the 16-35, anyway.