Photography against the clock is an exercise in futility.
I can't agree with you enough on that. Haha
When I'm out and about with people who aren't photographers or those who aren't as serious, I just expect not to get all my shots while being around them.. With the time it needs. i.e. Long exposures to name one. A lot of the times I'll just have a camera on a BR sling with a dslr and lens. That's all. Or use my iPhone. When I'm out, usually by myself, I'll take the whole party of joy with me. Haha 6 lenses, tripod, flashes... I'll leave out my softbox.
I learn that people enjoy the photos (granted it's not crap) but don't care for the effort and time put in it. Unfortunately, what others might not care for, that's what I enjoy. A good balance is always key for you and those around you. Happy shooting!