You previously indicated your primary current subjects are macro (on rails), abstract and landscapes. I've had no issues with adapted EF lenses on my EOS R, but I've read similar reports that range from bad to great. However, it's important to keep in mind that the adapter has no optics, so from an image standpoint the adapter makes no difference. I could see there being a difference from a performance standpoint (AF performance, in particular), but for your chosen subjects I don't think blazing fast AF is a key need for you regardless.
IMO, the reasons to buy native RF lenses for R bodies vs. using adapted EF lenses boil down to the features offered by those lenses (e.g. 1.4x vs. 1x plus the SA control on the new RF 100mm Macro) and the cost of replacing an existing EF lens you already have.
Personally, given the forthcoming R3, I have no plans to buy any more EF lenses. I'm giving consideration to which of my EF lenses I may replace with their RF equivalents over time. I have the RF 24-105/4L and I would characterize it as you do the EF II version – satisfactory but not overly impressive. I will get the RF 100-500L, and I'll probably replace my 'workhorse' lenses (24-70/2.8 and 70-200/2.8) with RF versions. I likely will not replace the EF 16-35/4L IS or any of the primes (with the possible exception of the 100L macro, but I need to learn more about the SA control; the 1.4x magnification is of little interest since I also have the MP-E 65).
In your position, I would just use the adapted EF 24-105/4L IS II rather than getting the RF 24-105 non-L.