I probably shouldn't even wade into this, but I can't help myself.
I just don't get mirrorless. Well...I sorta do...I kinda get Fuji mirrorless, but that's about it.
I don't get interchangeable lenses on a mirrorless. If I want a smaller, fake Leica body I'd want a single, fixed lens moderate zoom (24mm-100mm for example). I'm not going to use a 200mm 2.8 lens on a mirrorless camera, much less a 70-300 or 100-400 zoom. No point. I'd rather pay a little more, get a stellar zoom that I can use under all conditions and save the interchangeable lenses for the DSLR, which is a lot more practical form factor for changing lenses.
I don't get interchangeable prime lenses on a mirrorless. If I have to carry two-three lenses what advantage does a mirrorless have?
I don't get electronic viewfinders for the sake of electronic viewfinders. Maybe the technology will get there someday, but it's not there today. That's one thing I like about Fuji. They found a nice way to blend optical and electronic.
You have a point, at least as far as FF mirrorless is concerned: if the point is to go small and light the lenses prevent that from happening unless Sony or whoever comes up with a range of small lenses, which presumably means no fast zooms of any focal length and no primes longer than c. 100mm. Maybe my perspective is odd because I often want to go much longer than that (my "go to" FF lens is 70-300L), but I wonder how appealing such limitations are. If size/weight really matter, m43 makes more sense to me, especially given how surprisingly well the latest models do in combination with the primes and better zooms.
But mirrorless has an appeal that has nothing to do with size. Once you've experienced the accuracy (for still subjects, at any rate, but that seems to be changing) and speed of focus points that almost cover the entire viewfinder/monitor (well, that's true of Olympus and Panasonic - their rivals don't seem to have caught up yet), you feel a bit limited when you return to a FF dslr which, no matter how many focus points it may have, limits them all to a fairly small proportion of the frame. And while I get why many don't like EVFs, not only are they improving simply as viewing devices (among other things, the latest Olympus has a viewfinder that's second only to the 1Dx in magnification, apparently) they make it much easier to control the image you're creating since you can see the effects of changes in exposure etc. while looking through the viewfinder. This may not matter to you, but some of us find it helpful.
So if Canon's next FF camera were around the size of a 6D or 5DIII, and otherwise performed at least as well as those two, and mirrorless, I would buy one without the slightest hesitation. Maybe noone else would.... (And if sensor technology were to improve in such a way that the difference between FF and m43 shrank further, I wouldn't object to that, either.)