When did photography become a crime?

In the vein of the thread here on brits and cameras ... I was walking along my street the other day, and somebody stopped and pointedly asked "why was I taking pictures of their houses". Recently I've been taking pictures of my kids school. We're not going to be there much longer and I want pictures of it (and am always trying to practice), and a policeman came by my door because the school is freaked that some guy is taking pictures.

What are these people trying to accomplish? Somehow harassing photographers will make them safer? In an age when cameras are on every bodies phone (ironically at the school somebody took a picture of me - taking pictures), the govt. has cameras everywhere including the sky, along with corporations and businesses. Suppose I'm a criminal - what will walking around obviously taking pictures do for me? I'll photoshop them, ignoring the billions of free photos of people available on the web? Seems rather stupid, and takes focus away from the real crimes that you never see.

I think we should put a stop to this right now, and all of us go out and take photos in public, at schools, in churches and in safe neighborhoods. Time to desensitize people from thinking that photography is a crime.