...obviously, I wanted L......
Ok, but...why?
The lenses you currently have don't offer the best IQ, so presumably you want better IQ in that focal range? I think the best general purpose zoom for APS-C is the EF-S 17-55mm f/2.8 IS. Same price as the 24-105mm, but delivers better optical quality on the same APS-C camera, and gives both wide angle and short tele. FYI, wide angle is 35mm and wider (FF equivalent), so 24mm on APS-C isn't wide angle).
For the uses you indicate, the 17-55mm is ideal in its range; you obviously get more reach with the 24-105, but f/4 is slow for sports/street. The 17-55 also delivers better IQ than the 17-40, is longer, faster, and has IS.
For longer shots, given the uses you mention and the constraints of budget, I'd consider a fast prime. Not considering cost, I'd recommend the 70-200mm f/2.8L IS II, but it's quite expensive. So, for the lens after the 17-55mm, I'd look at your shots with the 18-200, and see what focal length would suit you best. Lenses I'd consider are the 85 mm f/1.8, 100mm f/2, 200mm f/2.8L II, and 135mm f/2L. FWIW, the 85/1.8 and 100/2 are two of the best values in the Canob lineup, great IQ for relatively low cost, and the 200/2.8 isn't far behind.