The OP wrote "Probably 300mm+ on a full frame.
So the 70-200mm f/4 L (non IS) - while a great lens, doesn't meet his stated criterion.
Some time ago I was looking for a new telezoom to replace my Canon 100-300mm USM (which didn't have the IQ at 300mm, and also lacked IS).
I considered the Sigma 120-400, Sigma 150-500 and Tamron 200-500, as well as the Canon 70-300mm nonL and Tamron 70-300 VC. I would've gone for the Tamron, except I tried the Canon 70-300mm L, fell in love with it: was blown away with the IQ and usability (while still being portable) as well as getting a good deal on a new one.
When I tried the Sigma's, I found they had both a lack of contrast / sharpness wide open, not as sharp AF as I wanted, and the Canon 70-300mm nonL wasn't too bad, but not great in build quality, AF a bit dodgy and IQ generally quite good except wide open at tele-end. The Tamron 200-500 didn't meet my focal length requirements (but might the OP's) - but the IQ isn't the best
So in the end I thought the Tamon 70-300mm was the best pick of the bunch, decent IQ (lacking sharpness wide open at teleend, but not bad)... maybe the OP can obtain a good deal on a new one, or a decent 2nd hand copy. I'd steer clear of the variation incarnations of the older / cheaper Canon 75-300 ... they are plain poor. I'd much rather have the newer Canon 55-250mm (vI or vII)... There is nothing like an L tele lens though (either the good zooms or primes). I'm very happy with my Canon 70-300mm L... it really shines, but I realise it's probably out of the OP's planned budget!