Cyrillic in redis

Question

Using redis-cli, I add data in Cyrillic to the redis queue, if using redis-cli to get the queue data, we will see "\xaf\xe0\xae\xa1\xa0".

In GO

msg, _ := redis.String(R.conn.Do("LPOP", key))

result

How can I get the Cyrillic alphabet? I tried it

encoder := charmap.MacintoshCyrillic.NewDecoder() // Windows1251 to
s, _ := encoder.String(msg)
fmt.Print(s)

it didn't work

It looks like you are using the Redigo client. Redigo does not transform or convert bytes sent to or from the Redis server. — Charlie Tumahai, Sep 20 '20 at 13:53
The `"\xaf\xe0\xae\xa1\xa0"` string corresponds to russian word _проба_ (test, sample) in `cp866` encoding. Tried in python using `b"\xaf\xe0\xae\xa1\xa0".decode('cp866')`. — JosefZ, Sep 20 '20 at 15:31
@JosefZ thank ```go encoder := charmap.CodePage866.NewDecoder() s, _ := encoder.String(msg) ``` it works — Артем Артемыч, Sep 20 '20 at 16:38
@JosefZ How do you know it's Russian, the same word can be Serbian, Belarusian, Ukrainian... — Z. Kosanovic, Sep 20 '20 at 19:14
@Z.Kosanovic It **is** Russian however I don't assert an _exclusive_ claim to… For instance, Google translate _detects_ `проба` as Bulgarian… Sorry… — JosefZ, Sep 20 '20 at 21:05

score 1 · Answer 1 · answered Nov 05 '21 at 02:34

When you see garbage symbols like that, the issue is very often that data (text) encoded with one format is being decoded as something else.

I use chardetect (python) to figure out what the input data is encoded as:

echo -e "\xaf\xe0\xae\xa1\xa0" | tee /tmp/j | chardetect 
<stdin>: IBM866 with confidence 0.99

uchardet works too:

$ uchardet /tmp/j
IBM866

To display the word correctly:

$ echo -e "\xaf\xe0\xae\xa1\xa0" | iconv -f IBM866 -t UTF-8
проба

Similar projects:

1 Answers1