... but I wonder why Sigma, Tamron & Tokina have not really ventured into that territory ... maybe the market for small primes isn't big enough for them?
I think you nailed the reason in a previous post ... most people want IS and USM (or equivalent designation), and these two technologies add considerable bulk to a lens.
Also, looking specifically at Sigma ... they have an "EF-S" fish-eye, but because the optics are shared with the Nikon version it is not a true 180 degree perspective on Canon APS-C cameras. So what's the point, Sigma?
Cheaper for Canon to manufacture just one lens with two functions.
But more expensive for the consumer who doesn't want that "second" functionality.
Relate this back to cameras ... nearly all DSLR's now have pretty decent video capabilities. Great, if you want and use it. Not and you're paying for capabilities (i.e. development and production cost) you don't need.
Or what about the DSLR videographer ... does (s)he need 61 cross-type, -3 EV-sensitive AF-points with multi-spotmetering and 1/8000 shutter-speeds? Nope. But (s)he's paying for it.