I've owned DSLRs for about a decade, and have taken many many photos at the beach. Generally I don't hesitate to change lenses at the beach, but if it's windy I refrain from doing this (or I go to the nearest non-sandy area, eg walk 50 metres away from the sand of the beach). I do not try not to change lens too often, on the beach - but never at ground level, nor when sitting down (as sand is more likely to get pick up / kicked up at that level). So usually I do a change with the body hanging off my neck.
Having grown up in Australia and then spending about a decade in Europe, also visting various beaches when I lived there - and eg going on holidays to Thailand 3 years ago, I have taken my DSLR gear (and prior to that my P&S camera/s) with me wherever I go. Sand possibly did get into (or 'along the barrel of') one of my Fuji P&S cameras - that may have have caused issued (but I had taken over 100,000 shots with it, so that's not bad).
I found more issues with going from very cold conditions (eg below -20 outside) in Romania - to inside conditions (causing the lenses to humidify / 'fog up') was more annoying, and as others have stated, could possibly present more issues. I had the same condensation issue (but with less severity) in the tropics (even when it's been the off-peak / humid season).
I enjoy taking photos at the beach - and I love swimming, walking at the beach. So I have taken my DSLR cameras to the beach hundreds of times. When there is a small risk of me actually falling in the water (eg wading in water with waves) I use my older DSLR body / lens combination eg Canon 350D with 18-55mm kit lens, rather than my Canon 7D with 15-85mm.
Certainly do be careful changing lenses... but if you are careful - you should be right. I would also say that having an 'air-rocket' to blow away dust, sand particles, and also a soft (eg micro-fibre) cloth to hand - but away from sand itself, to wipe away any sand, dust, etc - is handy.
Best wishes - and yes, do enjoy your holiday (not just the photography side of it).