I guess it's because these non-L primes are from the old film days, when even the cheaper SLRs where FF.
I think that's exactly the reason. In the days of film, there were still cheap SLR bodies, and budget lenses had to be available for that segment. Now that APS-C is more common, that's the market segment for the less expensive lenses.
In particular wide-angle primes would benefit from going EF-S (price- and size-wise). I find it a bit perplexing that Canon hasn't put more effort into producing such lenses, I guess the reason is that zooms are relatively so much more popular in that sector. It shouldn't be impossible to make, e.g., an EF-S 10mm/2.8 lens, I look forward to the day we see one.
I'm sure it's possible - Canon makes a zoom that covers 10mm with a narrower aperture, and Tokina makes an f/2.8 zoom that goes to 11mm.
But, I think the main argument against it is the one you stated - the popularity of zooms. When some of the 'current' non-L wide and normal primes were developed, zoom lenses were still regarded as sub-par for image quality, although the gap was pretty narrow. Today, the IQ of a zoom lens can match a prime (e.g. the 70-200 II vs. the 135L). More importantly, zooms are convenient - and that accounts for their popularity with the consumer segment. It's conceivable that Canon will make a wide angle EF-S prime, but they'd be the first (at least, as I interpret it - an APS-C format prime lens at 22mm or wider, excepting the existing APS-C fisheye lenses).