黄 most likely pictured a ceremonial jade pendant worn at the waist — yellow-brown jade being the royal stone. The emperor later claimed the color outright: yellow robes, yellow roofs, yellow everything.
黄 is imperial: only the emperor could wear 黄色 robes; the Forbidden City's roofs still blaze imperial yellow. Civilization's cradle is the 黄河 (Yellow River), and Chinese call themselves descendants of the 黄帝 (Yellow Emperor). Modern quirk: 黄 also means adult content (黄色电影), so context matters. And when plans collapse, they 黄了 — 'went yellow', like a dead leaf.
The emperor's jade pendant — imperial YELLOW.
事儿黄了 'the deal went yellow' = it fell through — autumn leaves doing business metaphors.