As I see it there are three major differences between these cameras that would determine OP's choice.
* Viewfinder. M has none, X100 has one of the best. I find I do not miss a VF on the M, though to be fair I have DSLRs and use those with subject matter where I probably would miss a VF.
* Analog vs. touch screen control. This is a matter of personal preference, though admittedly most would prefer more physical controls. I like the M's touchscreen, but I do wish it had 2 dials for directly accessing shutter and aperture in manual mode.
* Lens interchangeability. Fuji X100s has one lens, the M has 3 dedicated and can take hundreds more via adapters. Notably, it is 100% AF/AE/aperture control compatible with Canon's EF/EF-S lenses via adapter.
Just depends on what you want out of the camera.
+1. These points summarize the key differences. I have owned an EOS-M since July. As with many others, I only bought it because of the discounted price and lens compatability. After using it I realized it for awhile, I realized the M had a lot to offer and the it quickly became my camera of choice when I want to go light, and my backup body for my 6D.
+ 100 for me too, I just let my wife use my two old Point and shoot/ Pocket cameras, And I use my 3 days old EOS-M from now as the Smart Pocket ( Large Pocket ) camera.