My roughed out comparison to the 18-55 f/3.5-5.6 IS STM is below. It's not a perfect line up (used the mount to scale things), but it's probably close. That new lens is shorter.
18-55mm f/4 would make it bigger, yes. You just need more glass to pull off 55mm f/4 than you do for 55mm f/5.6.
To get a rough feel for this, you can compare a (full-frame) 24-105, which comes in both an f/3.5-5.6 version and an f/4L IS version. See below. The outward difference looks small (length about the same and filter diameter the same) but the non-L is a good 9-10 oz. lighter.