If I'm "touring" and want to capture some street shots, I use my LX-7 and keep it slung over my shoulder, lens cap off, AV, and cued up. I have my shoulder strap adjusted so that the camera rests at the perfect angle for me to just press the shutter casually without having to raise it or compose. Of course, many shots will miss and often I will need to straighten and crop after, but I can get some great shots this way and no-one knows I'm even taking pics. For example, I'm in China at the moment and here's a pic I took a few days ago SOOC.
If I'm home based, and I want to shoot a protest or rally or even just some street, I'll use my DSLR gear and throw a name-badge lanyard on. I've got all types of badges from conferences and events at work - none of them have to do with photography. But nobody reads the badge, it just looks "official". People very rarely question what they perceive as "authority", no matter how informal. If I am questioned, I just state that I'm capturing some candid photos of "the event", "the local population", "the citizens of Chicago", and then I follow with "would you like to be in a shot?". I took some candids at a Native American rally a couple months ago and one of the guys actually gave me his facebook name so I could send him a shot. After that, I got about 10 candids of him throughout the rally and was sure to follow up by sending him the pics. He ended up reposting and changed his profile pic to one of my shots. It worked out great.
So, basically, what others have said. If you have to go stealth, use a nice compact. If you bring out your real gear, act like you know what you're doing, smile, talk, invite, and shoot.