As a Canon user, with both FF and crop cameras, and with a pair of Olys at work, I like to think that I have a fairly balanced perspective.
as far as size goes, Oly is hard to beat. There are lots of lenses for the system, everything from kit to high end. It is a much more complete system than the M
as far as ergonomics go, any Canon FF camera, the 7 series, and the XXD series beats Oly hollow for ergonomic. Even the Rebels beat it. The M series are about the same as Oly for ergonomics.
Image quality? FF beats crop. Period! As to which is better between Canon crop and Oly, it really depends on what lens you are using.
Features? Up until lately there have been 4 big areas where Oly beat the pants off of M. With the M6 II we can remove burst rate and “pro capture” from that list. The remaining two are multiple shot stitching into super hi res photos, and image stabilization. The M has optical stabilization, the Olys have IBIS and a few lenses with optical stabilization. For those few lenses and the newest bodies, Oly wins the stabilization contest, for everything else it is too close to call
ultimately, we vote with our wallets. I have stayed away from the M series because, for me, they just were not worth it as Oly was so much better...... but the M6 II makes the race a lot closer. That said, there is an M6 II in my future