Earth 土 above a hand 寸: land held and administered. 寺 first named a government office, not a religious one. When Buddhist monks arrived from India in the first century they were housed in one of these offices, and the word followed the guests rather than the original tenants.
Every Buddhist temple in China is a 寺 — 少林寺 the most famous of them. Because the character is quiet ground held in hand, it also lends its shape to 时 (time), 持 (to hold), 诗 (poetry) and 待 (to wait).
Land 土 held in a measuring hand 寸 — an office that became a TEMPLE.
寺 meant a government office first. Buddhist monks arriving in China were put up in one, and the building's name stuck to them instead.