"That he may not come at all times into the holy place"—Leviticus 16:2.
The high priest is forbidden from entering the Holy of Holies any time other than Yom Kippur, and only when he is then performing the special Yom Kippur service.
The regular priest is forbidden from entering the Holy Temple's sanctuary unless he is discharging a service there [e.g., kindling the menorah, offering incense, or simply entering to prostrate himself before G‑d].
"And you shall not go out the door of the Tent of Meeting"—Leviticus 10:7.
It is forbidden for a priest to leave the Temple while he is in middle of performing his service.
A high priest may not leave the Temple even if he is an onen, i.e. he is informed that one of his next of kin has passed away.
"They shall send out from the camp all those afflicted with tzara'at or with a male discharge, and all those unclean through [contact with] the dead"—Numbers 5:2.
We are commanded to expel from the Temple area all those who are ritually impure.
"They shall not defile their camp"—Numbers 5:3.
All ritually impure individuals are forbidden from entering the Holy Temple—i.e., any area in [the sanctuary and] the Holy Temple Courtyard, starting from the Nikanor Gate.
"He may not enter this camp"—Deuteronomy 23:11.
All ritually impure individuals [with the exception of individuals impure due to contact with a corpse] are forbidden from entering the Temple Mount.