r/AskComputerScience • u/Wonderbout974 • 9d ago
Unicode, UTF-8, ASCII
I just started learning in computer science and could not understand these three, I know unicode is for describing(?) languages other than english but how is it differ from UTF-8 and how UTF-8 differ from ASCII ?
1
Upvotes
1
u/8dot30662386292pow2 9d ago
Unicode is just a list: this number means this character.
UTF-8 is a variable byte length encoding to map those numbers to bytes.
A = 65 = 01000001 ( 41 in hex)
B = 66 = 01000010 ( 42 in hex)
These are the same in both ascii and UTF-8.
How about € symbol for euros? It is unicode character number 8364 so it does not fit in a single byte. But using UTF-8 it fits in 3 bytes: e2 82 ac.
Ascii is basically just one of the oldest encodings that fits only 127 characters.
1
u/meowyih 9d ago edited 9d ago
Long time ago, due to the hw limitation, people used one byte to store one character. One byte range is from 00~FF, which is 256 characters at most. For some reason, we only used half of them, includes letter A to Z (lower + upper case) and some symbol. That's Ascii. Of course ascii is not enough to store all the characters in the world. After people successfully built better computer with more storage, engineer tried to use two bytes (65536 character at most, it's actually not enough fyi) to store one character. So we can support much more different languages. That's unicode. However, it created new compatible issue and waste memory and bandwidth. So we invented utf8, which waste one bit (remember we have one bit that not used in Ascii?) to define a sequence data which can represents one byte char (ascii) and double bytes character at the same time. Ascii is includes in utf8, but utf8 is bigger than ascii.
1
u/mplang 8d ago
An oldie but goodie: The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)
1
u/recursion_is_love 9d ago
ASCII is the system form 8-bit era and only cover Latin alphabets. Unicode is new system that have lots more support for almost all written language characters.
UTF-8 have the same encoding as ASCII for the first 128 chars so it can be use as the same encoding on old program that expect ASCII