I want to use that technique to shoot empty streets...was there many people coming and going in during shooting? (like stairs and bridge ones).
With a B&W 10 stopper (example or ND 400 Hoya) you won't see any people coming any going on a 4-5 minute exposure. This is the beauty of that dark glass extending your exposure time.
Just don't forget to do the MLU (mirror lock up) to avoid vibration.
I've read about the IQ degrading at F16 and F22 or higher (one of mine goes to F32) but I believe that only applies to shorter exposures example 1/15, 1/10, etc. if you have that lens open long enough the sensor will pick up more detail, colors etc which would somewhat mitigate the worry over IQ (excluding diffraction of course).