I've also struggled with this. I think metering and exposure are critical to getting good shots with on-camera flash. Whenever possible, I think on-camera flash works best when used to fill in shadows and not as the primary light source. The only exception is when you can bounce the light off nearby surfaces and diffuse the light.
With regard to flash toys, I don't think there is a silver bullet. All of the various manipulators seem to have their pluses and minuses. I've used cards, diffusers, bouncers, etc. Like most photographers, I have a flash toy graveyard.
I've also looked at ringflashes, but I feel like they are one-trick ponies (and the good ones are expensive). I wouldn't even call a ringflash "on-camera" because it's about as cumbersome as a tripod, imo.