It's when you start adding elements not in the original capture, It's no longer photography.
I could not agree more, adding to the image changes it to photographic art; however, subtracting (the corner of a building, a flying bird, contrails, et cetera) is perfectly acceptable. Obviously anything you can do it a wet darkroom, you can 'legally' do the same in photoshop.
So I can take a nice portrait shot and delete the everything but the eyes, it's okay? Or do I have to run that by the edit police to determine what is deletable and what isn't?
BTW, by deleting ANYTHING you are adding as well--you are adding space or order or isolation, etc. Like the construction crane behind the building example--that's not reflecting reality, it's portraying an enhanced reality that is meets the photographer's tastes.
Even just enhancing contrast is a change to reality, yet so many of you are saying "deviations from reality that meet my arbitrary criteria are okay, but everything else is taboo." That's a bit egotistical, don't ya think? 
Good lord... photography is an art form... altered or non altered, it's an art form... In the film days did you not think they spliced film? overlapped film? Added density filters, dodge, burn, enhance, multiple exposure... I would hate to hear someone say ansel adams work wasn't photography because of all the manipulation he did in the darkroom... It is what it is... No one is going to look at a print ANY LESS if it was heavily manipulated vs OOC. Then you have photographers like Sal Cincotta and Dave Cross shoot some pictures just for post production if needed. I doubt any of their clients who are spending thousands for his services think his work isn't art/photography.