There are ways altitude can cause issues, but not the kind of altitudes you are discussing. In extreme high altitudes, outgassing of silicones, lubes, plastics and the like can redeposit themselves where they care not wanted. You do not want them to deposit on your sensor or lens elements. That's why special modifications are made for equipment going into space. Part of my job used to be making sure that components used in spacecraft met outgassing specs.
Moisture is the enemy, rapid temperature change from warm humid to cold is the worst, since moisture condenses inside your camera. Taking a very cold camera into a moist warm room will condense moisture on the outside of the camera and lens where it is much less likely to be a problem.