Basically they allow a thiner camera for a given sensor size with a interchangable lens. If you make a smaller sensor, you can also reduce the diameter of the lens, and make the overall size of the camera smaller.
You also lose some features that professional photographers value, like a optical view finder. Other pro features like phase detect AF are finding their way into a limited number of mirrorless bodies.
They are a quickly evolving product, and there is a lot of competition. Generally, new lenses are needed, but there are converters that let you use your old lenses if necessary.
One thing to consider, is that, like any new technology, there will be a fallout. Some models will not sell or be overtaken by other better ones, and go out of production. This could leave consumers with unusable high priced lenses. Manufacturers will not hesitate to drop a product line if it flops. Something to consider when buying one, I'd suspect that Nikon, and micro 4/3 are safe, Sony has dropped so many products in the past, that I avoid them, even though they seem to have some winning products ... for now.