As others have said, the 5D II has only a 98% viewfinder. The main reason for not having a 100% viewfinder isn't the cost of making it bigger, but the cost of aligning it accurately. Missing out 2% allows for some minor misalignment without the viewfinder seeing part of the frame beyond what the sensor would capture.
Are you able to see part of the frame in the viewfinder which is not captured by the sensor? If so, it suggests the alignment is out beyond the manufacturing tolerances, whereas if everything you see in the viewfinder is in the final image, just with an uneven amount of extra on the sides/above/below, then it is within spec, and something you'll probably have to learn to put up with.
Liveview uses the sensor exactly as taking the photo does - so no mirror/pentaprism/focus screen involved, and no chance of any alignment issues.
As you say the misalignment in the viewfinder is not varying from shot to shot, its unlikely to be anything loose, instead it will be something that's shifted. The user replaceable focusing screen might be out of place. What about re-seating that? If that doesn't work, has your second hand 5D mk II got any signs of drop damage, especially around the pentaprism? If so, you could try getting a quote out of Canon.