BRAVO!! We rate your comment a 12 out of 10... Seriously, fantastic comment. Nothing more to say.It is one of those things like most things in photography that doesn't have a simple answer... it depends. Obviously there are a lot of factors and totally scene [light] dependent, lens matters also and to some degree viewer discernment and who the images are for or purpose of them. It depends on the light and where it is coming from and obviously no-one can answer that without knowing the scene and knowing a particular lens behaviour. Some are really prone to flare/veiling/contrast issues etc and it doesn't take much to cause noticeable problems in lot of lighting conditions, some lenses may be more forgiving and only cause issues in extremes. edit: you can look up types of flare for comparissons and also review tests on the models you use that check for that deliberately to see. Plus some may be there but not apparent such as large area but mild veiling not dramatically visible until the extreme zones, just caussing a general fog that can hide high frequency detail and some loss of contrast but not a problem most would notice and if you aren't shooting for editors or clients who'd object who cares.
Also the degree and qualities of it matter even when present thus again is not so simple as any = bad. Well controlled only detected when pixelpeeping means good enough and I'd class that in the same camp as absent of flares. More this second point is how sometimes it may even be favourable. More so with cine glass than stills orientated glass where they don't try and obliterate such things so much as make it look good and can be used for creative reasons. For instance if you have massive generalised veiling hazy blob wiping out contrast in a landscape shot with sun coming from camera front/side and horrid looking blob artifacts it will look nasty where having a really nice well done cinematic flare starbursts and veiling is less general blob and nice "godray" style shafts it can look much much nicer than no flare at all. Generally for photo glass though it is preferably avoided as often doesn't look so favourable as less effort is put into things like pleasing flare for stills (or makign parfocal and so on thus cine glass oft cost more even if for stills it is equivalent image quality otherwise re: sharpness and general rendering).
If you only shoot scenes with light behind you with nothing too reflective close by and have lenses that generally don't complain unless you push them to extremes such as shooting into the sun or point lightsource unintentionally (ie. almost never) and don't already own hoods I wouldn't sweat it. That isn't me but sounds like it could be fine for you so I wouldn't rush out and buy hoods. Plus even if you did notice problems you can fix it with hand/moving closer to something to flag that offending light path and so on in most situations. If you constantly find yourself in scenes with light(s) at potential problem positions to the camera in scenes where you want nothing but image light getting to the sensor it is obviously bigger deal chancing it happening or not and troubleshooting when it happens. Thus airing on side of caution using a hood to help prevent it saves time and messing by avoiding the problem rather than waiting for problem then rectifying.
There are generally no downsides if you own hoods though if you own the right hoods as they oft reverse mount to lens for storage, as for vignetting even wide angles that could be prone to vignetting tend to have petal hoods made to match the lens fov thus avoid it. Generally as again there can be rare instances and lighting conditions.