I have worked and lived in Saigon for quite some time and also traveled there from where I live now (Phnom Penh). I've lived in Hanoi also. First, Saigon is a fairly safe city depending on where you go. If you go to District 4 late at night there is no saving you, but staying around the city centre you will be fine. Just make sure to keep your equipment tight, there are boys on motorbikes ripping cameras and jewelry off of people even during daytime. There is plenty of interesting places to go around district one. The old colonial hotels, the opera house, the cathedral, the post office, the former presidential palace etc. Very obvious but also very nice. Make sure to get around to pagodas, there is an old one in District 11 called Giac Lam which is very beautiful. I did a portrait shoot there a couple of months ago with Vietnamese girls in the traditional dresses. By the river you can catch some nice scenery especially early in the morning. Try to catch some photos around the harbour. There are plenty of beautiful French colonial building still in use, they are often hidden away behind tree curtains, make the extra effort and you will be rewarded. Catch some classical Saigon scenery from the rooftop bar at the Caravelle hotel. District 5, China Town, is a place you want to go also.
About lens choices, not sure what to say. It's like any other city in those terms. If only one lens then the 24-105 covers most. The 135, I don't know what you expect to capture in a city except for candid portraits. Bring the 70-200 if you can. But remember, in June it will rain and it can be quite hot. In Phnom Penh we currently have around 40 degrees centigrade (104f), Saigon is just 250 km down the road.
Enjoy the town, it's fantastic!