My point is what you say is true for the pixels, but not for the picture. Yes, if you’re examining your images at 100%, then the smaller pixels will look noisier. But if you’re viewing images from two full frame sensors with substantially different pixel sizes on the same monitor, they will look essentially the same as far as noise goes, even for images taken in low light. The higher MP image will be downsampled more, mitigating the additional noise.
This issue was beaten to death when the 5Ds came out. Image noise is determined by total light gathered, which is a function of sensor area, not pixel size.
If you 2x2 bin 80 MP down to 20 MP, the noise is the same as a 20 MP sensor, even in low light. That wasn’t always true, with old sensors the photosensitive area of each pixel was sometimes as little as 20-25% of the area and there was a big noise penalty for more pixels. But gapless microlenses obviated that issue, so now a 2x2 array of pixels delivers the same effective output as the equivalent single pixel when viewed at the image level.
Your point is relevant to pixel peepers, not so much to those of us who look at whole images. There are good reasons to choose a lower MP sensor over a higher MP one, but less image noise isn’t one of them.