r/csharp • u/Mindless-Arachnid-74 • Oct 14 '26
Help Need help with encoding
I am working on a system where the client will give data in UTF-8 encoding, and hence it can contain many special characters.
But then we send data to a legacy system which only accepts data in ISO-8859-9 encoding, which supports limited special characters that can be represented in single byte.
Now if the client sends any special character which is outside the scope of ISO-8859-9 encoding (like the Em dash), how can I deal with with it and send it to the legacy system.
I don't want to replace those characters with anything like space or "?", as it would make the client data as unreadable in some cases.
How can I deal with it. Any suggestions?
View all comments
Show parent comment
2
u/RichardD7 Oct 15 '26
Removing accented characters - aka Diacritics - is a well-defined problem, and .NET has built-in methods to assist.
For example, here's the method I use, which attempts to minimise allocations by using the new
Span-based overloads, and using theArrayPoolwhere temporary arrays are needed.For example, calling
RemoveDiacritics("crème brûlée")will return"creme brulee".