The 24L II is almost as heavy as the 24-70 II
I have sold all my primes because most of them are not as sharp or focus as accurately as either the 24-70II or the 70-200II (I have both).
As I recommended, keep everything and maybe invest in a good tripod system and use the MK3 primarily with this.
As FunPhotons said (and I'll reiterate), maybe it's just the big DSLR thing that bugs you (I know it bugs me).
I have a 40mm pancake, but I still don't think that a DSLR is much fun to lug around the streets.
A Fuji will slow you down, change the way you take photographs and inspire creativity in ways only old film cameras and Leica's can.
You can get the XE-1 with the 28-85mm IS lens for $1400, which is about the price of a prime lens.
FWIW, they are also due to announce the successor to the X100 on Monday and it's supposed to include Focus Peaking.
I know I reach for one of my Fuji's 99% of the time and considered selling all my Canon gear, but the money is spent and you will just lose it if you downgrade or add new stuff to it. That will just lead to a cycle of more remorse.
A 6D is not going to be any better than the 5D MK3 in this respect.
Also, the 5D MK3 is good for tracking fast moving subjects and the combination of 24-70II and 70-200II is probably all you need for capturing your growing family.
This is something the Fuji isn't good at.
I have a 19 month old daughter, so I kept my Canon setup for her.
I also think you'll be pleasantly surprised when you get images as good (or better) than your 5D MK3 from such a small, cheap package.
Anyway, I've been there recently and my advice is don't sell it and don't add to it.
Just try a different approach to your photography and see what happens.