It's definitely possible to make a practical <2.8 zoom lens with a conservative zoom range (not too wide and not too long).
The Tokina AF 28-70mm f/2.6-2.8 AT-X Pro II is a (discontinued) full-frame lens that is a tad faster than 2.8 at the wide-end, and it's not very large (A bit larger than the 24-105 f/4 IS). Moreover it's fixed length (does not extend while zooming). Quite a nice piece of engineering.