I still do not see a reason for an EF lens to be built as a pancake though. Other than triggering "oh is this cute" reflexes.
....
Bleeding edge? Some examples for me are
* Nikon D800. Way better than the 1Ds III and the Nikon D3x at 40% of the price. Better than the 5D3 but $/€ 500 less. I have not heard Nikonians complain about the D800 price.
* Nikon AF-S 35/1.8. Excellent image quality. Small price.
* Nikon AF-S 85/1.8 bleeding edge image quality. Very affordable.
* Nikon 14-24 bleeding edge performance. Reasonable cost ... similar to the inferior Canon 16-35 II.
Canon? Bleeding edge? Yes some: superteles, TS/E, 70-200/2.8, possibly the 24-70 II ... definitely all bleeding expensive. Wide angle lenses? Not one.
The reason for building as a pancake is obvious ... to have a pancake. That means: it's really small. Smallness by itself is a great value. It's small in the bag, small in the hand, small in use. I love small lenses (when I don't need a fast aperture, that is). Yes, I would love a 40/1.8 too, but you can't have a full frame f/1.8 lens that is
this small.
For my purposes, the 5D3 is much better than the D800 and well worth the extra cost. I grant that for others, the D800 will be better. But I really don't need Canon to build cameras to match specific Nikon models.
The Nikon 35/1.8 and 14-24 don't have exact Canon equivalents ... yet. Perhaps they will one day; there's a rumor of a Canon 14-24 patent anyway. Nikon's 35/1.8 is excellent, but not full frame. Their 14-24 is excellent too, but not a lens I would find useful (or want to buy for $2K). The new 40/2.8 is actually much more useful for me, so I'm thrilled about it.
The point is, manufacturers don't follow the same lens roadmap. They each have different priorities at different times. That's to be expected. Remember how many years it took Nikon to introduce a 24/1.4 and a 35/1.4, or to build an updated 85/1.4 ... years. After Nikon cancelled their 28/1.4, years went by without a fast Nikon wide prime. I think the Canon 24/1.4 II is exquisitely good, and the 35/1.4 has been a workhorse for me for years, so I'm very happy with the fast Canon wides.
I think the bottom line is that whatever they build ...
someone will complain. I mean
any lens you can think of ... someone will complain that they didn't build another lens
instead. Photographers have different needs at different times and, not surprisingly, no manufacturer's product release schedules will exactly match our personal needs.