While I don't think anything else compares to L glass, in terms of both optics and construction, you might consider Canon non-L lenses from the recent past.
While my experience, too, found the 28-135 IS a little wanting (with use, I noticed too much play in the barrel, especially when zoomed out to the long end, so the lens was never sharp).
But, a couple other lenses that cover close to the range of 24-105 gave me better results. Both the 24-85 and 28-105 are capable performers, especially considering their price on the used market (though I would want to test to make sure you have a good copy). While they don't offer IS, and their maximum aperture drops from f:3.5 to 4.5 at the long end, I've had more than adequate results.
No, they don't quite measure up to the build and optical performance of the 24-105 L, but, at a small fraction of the price, might work well for your use.