It's often a good idea, when you have a good idea, to wait a while & see if it was a genuinely good idea, or Lens Envy in this case, perhaps.
You own a couple of Canon's current best Lenses, the 24-70f/2.8L II & 70-200f/2.8L II, getting rid of them to replace them with Only Primes may not be in your best interests, You can certainly replace them with a selection of fast Primes 24f/1.4, 50f/1.2, 85f/1.2 L II, 135f/2, 200f/2, but the cost will be significant, and I'm not convinced you will see a Major difference in IQ.
I am of the opinion that Good Primes (The ones I mentioned Above) will generally produce better IQ at a given Length than a Zoom that covers the particular Length, in the 24-70 Range I have the 24-70f/2.8L II, 24f/1.4L II & the 50f/1.2 L, these Primes are excellent lenses, are they significantly better than the 24-70f/2.8l II ??, I don't think so, are you able to produce better Images with the Primes in this range, not better in my opinion, but different, mainly due to the ability to Blow out the Background more @ f/1.2 or f/1.4, But again, how often ??, not so much has been my experience.
Again in the 70-200 Range, I use the 85f/1.2 L II, 135f/2 & 200f/2, all again amazing Lenses, but they are pretty well particular Tools for again a certain job, primarily where you want to Blow the background, the only caveat I would add in this Range is I find the 85/135 & 200 Primes all produce a sharper image generally than the 70-200 at a similar Range, but it's at a cost, and it's at an inconvenience at times of carrying 3 Lenses instead of one.
Another factor to consider is weather sealing, in the 70-200f/2.8L II you have a well sealed Lens, that's not the case with the Primes until you get to the 200f/2, this may not be a major factor dependent on what you shoot, in my case, wildlife in Africa & Arctic/Antarctic environments, it's a major decider of what Kit I take to these Places.
Admittedly I'm fortunate in that I can afford to own &utilise both, the Zooms & the Primes, but having both, and using both, if I could only afford the one or the other, given the excellent IQ offered by the two Zooms in question, I'de stay with the Zooms, then set your sights on a particular Lens you feel you absolutely have to have, rent one for a while, if it's what you wanted, work on adding it to the Zooms you have and using it for that particular situation where the Prime will enhance your Photography.