If it had a user replaceable date/time battery, it would have been mentioned in the user manual, same as the 5DmkIII.
Page 53 of the 5DmkIV user manual says "The date/time/zone settings may be reset when the camera is stored without the battery, when its battery becomes exhausted, or when it is exposed to below freezing temperatures for a prolonged period", which is when I would expect an internal capacitor to run out. If the camera 'forgets' those setting in other situations, my guess would be the capacitor is defective.
As Lithium ion batteries don't generally leak, I think its safe to keep the camera with a battery in.
I looked at Lens Rentals for a button battery soldered to a board or a large super capacitor. I decided that its more likely that they used just a ordinary capacitor, maybe the one shown in the photo that is located on the battery board.