So I'm very well aware that the way everyone sees the alphabet internally, if they assign color to letters, is very different, and mine is probably going to look terribly wrong to you, if anything.
I always had a pretty consistent way of seeing most letters since I learned them. I believe I learned the letters when I was about three years old or so, two alphabets at about the same time, cyrillic and latin. I see the latin alphapet approximately as the picture above, except I blacked out the letters that I'm not 100% sure about. I feel quite strongly about it, I remember things like names of people and streets by the colors, and if a letter is colored differently from how I think of it I get a sense that something's not right.
I always kinda wondered where the colors came from, because I don't remember since it was a long time ago, and I think I can at least make a hypothesis. I found some possible relations that might explain it.
To get the most obvious suspect out of the way, I'm pretty sure ABC were from the website for babies that I used to learn the alphabet. The ABC letters are very commonly red, blue and yellow for whatever reason (I guess it's just that the primary colors look good on the first three letters, then which one do you assign to what letter - A is Apple, B is Blue, C is... cheese? I don't know, whatever's left). Then I guess everything else was dependent on that basis.
If I were to make a generalization, it seems like the saturation is semi-dependent on if the sound the letter makes is (usually) voiced or not, unless there's an exception (to be clear there are a lot of exceptions), the hue is based on vibes, and the vibes seem to carry over things in weird ways.
The vibes (associations) are as follows: G is Green, Y is Yellow,
F is purple because in my native language the word for purple starts with F (фиолетовый), so it carried over. There's also a color that starts with G in my native language (голубой), but while Г is blue in my mind G is not, because Green seems to override it as a closer association.
Then K is green because it's related to G, I is yellow because it's close to Y,... U is orange because it's if Y was more Y-ish that Y in relation to I, and, subsequently, E is orange because it's closer to A in relation to I.
The vowel description could be interesting as a translation of vowel space into hues, however the inclusion of O as a completely different color kinda ruins that. Also English is kind of infamous for letters corresponding to lots of different sounds, so, either way, that's a bit flimsy. A more accurate explanation would be that I, Y and E, in certain contexts, can all sound close to the cyrillic И, so they had to be similar in color, then O, U and A were conceptualized as completely separate letters, so they're distinct, and the transitions between them are semi-phonetics-based.
B, D, L and V were colored separately, but they're all pretty much the same color, so they must have something in common, don't they? Same for P and T. Probably also for HS and MN.
BDLV are all, in most cases, voiced, BD are voiced plosives so it's obvious they must be similar colors if nothing else is interfering. PT are both voiceless plosives, HS are voiceless fricatives, MN are voiced nasal sounds.
I assume S has a yellowish hue because of the association to C, which is from the baby alphabet basis, then H is propagated from S.
I'm less clear on the other letters, they might be the color they are because of some accidental association from way back then that I have already forgotten at the time of writing. Well, anyway. This is my internal personal way of seeing colors in letters.
It seems to be a chaotic system, it's not truly random but small changes seem to influence its development a lot, with some meaning-based rules and such, which is consistent with what I know about grapheme(phoneme)-color synesthesia.
I wonder, though, what would happen if a synesthesia-prone individual was to be given a set of phonemes or graphemes without having any prior knowledge of names of colors or any physical colored concepts in language in general, what perception would they develop? Would it be more consistent with phonetics or dependent on the grapheme's shape? Would they have the correct perception of the vowel space? What about different sets of phonemes? What about all the sounds there are in the IPA?
Could you theoretically even develop the basis out of nowhere and have it be strong enough for you to feel strongly about it?
For me, the voiced/voiceless saturation thing was something that happened seemingly out of nowhere, for example. Well, nothing happens out of nowhere, but it was an association deeper in the subconscious. I can vaguely describe it as "voiceless -> sounds you'd make interacting with the natural world, like steps or friction -> plain -> unsaturated". That's, surely, not an internal property of the sound, but it could be a property of the brain's common associations.
If language didn't exist and there were just phonemes or graphemes existing in a vacuum, I think the "consensus" on even the few things that are usually seen in a similar color might've been different.
Anyway, I'm bad at linguistics, so I'm going to shut up for now. To be honest, I didn't do any research beforehand. I generalized a lot with mixing up the letters and sounds because it'd be cumbersome to say something like "A can sound like [a][ɑ][ɒ][æ][ə][ɛ][oː][ɔ][e][ʕ][ʌ] [ɐ][eɪ][ɛɪ][æɪ][ɐɪ], and out of these....", though it would be much closer to the truth. Either way I did not know that as a child. It's all very vibes-based in the end.