This is going to blow your mind, but there's such a thing as metonym. Sometimes we use the word proxy. As ChatGPT puts it:
"using skin color to define racial identity is a kind of metonym—specifically, it’s metonymic shorthand where a physical trait stands in for a broader, socially constructed category."
You're confusing the shorthand for the concept. It's not that different from the way we talk about "big men" in basketball. There is obviously so much more than height that separates players, and even their play style, but "big man" is a shorthand to refer to the collection of traits we associate with tall players.
Of course, it's not that skin color is unrelated to race. It's just that race is so much more about skin color, even if we use skin color words as shorthand because other formulations are wordy. You would hoot and holler at a formulation like, "racial identities based on societally imposed feelings and perceptions of inferiority, often due to skin color or assumptions about skin color." So we say dark-skinned for brevity.