0

My task is to get a json string from request body in utf-8 and encode it to win1251, so i could save it the db in win1251.

data, err := io.ReadAll(c.Request.GetBody())
if err != nil {
    panic(err)
}

enc := charmap.Windows1251.NewEncoder()
win, err := enc.Bytes(data)
if err != nil {
    panic(err)
}
_, name, _ := charset.DetermineEncoding(win, "application/json")
revel.AppLog.Debug(name)

I tried this, but i'm getting win1252(( Any ideas?

timelord
  • 11
  • 1

1 Answers1

0

Do not use charset.DetermineEncoding, you know encoding for your bytes without it.

There is no magic function that correctly detects encoding for any data.

charset.DetermineEncoding is not an exception. It detects a few obvious cases and then fallback to "windows-1252":

https://cs.opensource.google/go/x/net/+/1185a901:html/charset/charset.go;l=52

Dmitry Harnitski
  • 5,838
  • 1
  • 28
  • 43
  • sure, but i'm getting win-1252, cause i'm saving this data to db and getting wrong encoding, cause db in win-1251. So deetermineEncoding is correct here and why would it fallback to win-1252 when i encoded data to win-1251??? if it was not obvious cases, it would explain it, but i should have win-1251 – timelord Jul 29 '22 at 09:16