I like the idea of going with smaller gear for certain situations, and the IQ is certainly very good now... but have you held on in your hands?
I held a friend's NEX a couple months ago and it cramped my hands inside a minute. Ergonomics are worth something I'd say.
When I decided to go digital, I had an older Minolta manual focus system, so I had no upgrade path to follow, and I looked at every brand and option. I went with Canon for both lens selection and how they felt in my hand.
So, if you try something new, try it in your hands before you sell the old gear, at the least.