Wow...there aren't many cameras on my list of cameras I wouldn't use even you gave it to me, but the X100S is one of them. There are several reasons I don't use a cell phone camera, but the biggest is that they have a fixed prime lens. To me, all the fixed prime lens cameras are just over grown cell phone cameras.
Don't get me wrong, I use primes, including a 35/1.4L on a 5D, but I never go out with just one, and I change focal lengths all the time. It's not at all uncommon for me to shoot with a 15mm fisheye and a 400mm on the same shoot in the same few minutes.