How to decode a string encoded by C# Convert.ToBase64String in python 3

Question

The c# server side:

[HttpGet("{id}")]
public ActionResult Get(int id)
{
    var user = new User
    {
        Id = id,
        Name = $"User{id}"
    };

     using(var ms = new MemoryStream())
     {
         ProtoBuf.Serializer.Serialize(ms, user);
         var bytes = ms.ToArray();
         var str = Convert.ToBase64String(bytes);
         return Json(str);
     }
 }

the python client side:

async def foo():
    async with ClientSession() as session:
        async with session.get("http://localhost:57968/api/values/5") as response:
            json = await response.json()
            # how to get the bytes created by the server
            person = Person.create_from_bytes(bs)

How to get the raw bytes array created by the server using ProtoBuf.Serializer.Serialize(ms, user); in the python client.

If i do not wrap the raw byte array with base64:

Update: I worked it out in the python client side like this:

json = await response.json()
bs = json.encode("ascii")
b64 = base64.b64decode(bs)
person = Person.create_from_bytes(b64)
print(f"{person.id} {person.name}")

minor tip: `var bytes = ms.GetBuffer(); var str = Convert.ToBase64String(bytes, 0, (int)ms.Length);` - avoids an extra array alloc/copy; as for the base-64 decode: isn't that just `base64.b64decode(yourString)` ? or possibly this? https://stackoverflow.com/questions/39209872/decode-base64-string-to-byte-array ? What does `Person.create_from_bytes` take as input? have you already unwrapped the json? i.e. what is `bs`? — Marc Gravell, Jun 22 '17 at 10:30
@MarcGravell In python3,base64.b64decode can not take string as parameter. — HongyanShen, Jun 22 '17 at 10:50
@mjwills if the intent is to talk protobuf, then ... that's OK too; I would perhaps question the decision to wrap it as base-64 in json rather than just raw bytes in the http response, but: it is OK to use things that aren't json — Marc Gravell, Jun 22 '17 at 10:53
@code_farmer did you try the other link I suggested? also: to emphasize: base-64 costs bandwidth; if possible, I would recommend a raw http binary payload here, not base-64 inside json — Marc Gravell, Jun 22 '17 at 10:53
@MarcGravell It does not work,i can not conver the **json** response to proper unicode — HongyanShen, Jun 22 '17 at 11:11
@code_farmer there is no unicode here; base-64 is ascii, and the bytes you get from protobuf are not text — Marc Gravell, Jun 22 '17 at 12:12
@MarcGravell Thanks for your inspiring, i worked it out as updated. — HongyanShen, Jun 22 '17 at 12:37
awesome! You should probably move the last part (the "Update:") into an answer - it is perfectly fine to answer your own question — Marc Gravell, Jun 22 '17 at 12:45

score 1 · Answer 1 · answered Jun 22 '17 at 12:59

1

Finally, i worked it out in the client side by using:

json = await response.json()
bs = json.encode("ascii")
b64 = base64.b64decode(bs)
person = Person.create_from_bytes(b64)
print(f"{person.id} {person.name}")

answered Jun 22 '17 at 12:59

HongyanShen

1,435
2
14
23

Breno Medeiros de Oliveira · Answer 2 · 2020-01-09T18:36:48.793

I had some time to develop this Solution for Python3.7, considering as example:

'''        Argument =   [110, 13, 46, 136, 95, 66, 92, 132, 109, 217, 58, 112, 43, 8, 145, 
                         42, 233, 98, 40, 139, 165, 228, 52, 9, 89, 175, 146, 103, 227, 238, 233, 190, 
                         78, 175, 242, 224, 202, 138, 248, 103, 114, 98, 199, 252, 80, 86, 61, 174]
            return =    'bg0uiF9CXIRt2TpwKwiRKuliKIul5DQJWa+SZ+Pu6b5Or/Lgyor4Z3Jix/xQVj2u' 
'''

Like this:

import base64
import numpy as np


# Do the same as Convert.ToBase64String() from C#
def ConvertToBase64String(encrypted):
    return (base64.b64encode(textoUnicodeToUtf8Literal(("".join([chr(item) for item in np.array(encrypted, dtype=np.uint8)]))).encode('ISO-8859-1'))).decode()

# Do the oposite of Convert.ToBase64String() from C#
def ConvertToStringBase64(encrypted):
    return np.frombuffer(base64.b64decode(encrypted.encode()), np.uint8)


def textoUnicodeToUtf8Literal(encodando):
    return encodando.replace("\xc2\x80", r'\x80').replace("\xc2\x81", r'\x81').replace("\xc2\x82", r'\x82')\
        .replace("\xc2\x83", r'\x83').replace("\xc2\x84", r'\x84').replace("\xc2\x85", r'\x85')\
        .replace("\xc2\x86", r'\x86').replace("\xc2\x87", r'\x87').replace("\xc2\x88", r'\x88')\
        .replace("\xc2\x89", r'\x89').replace("\xc2\x8a", r'\x8A').replace("\xc2\x8b", r'\x8B')\
        .replace("\xc2\x8c", r'\x8C').replace("\xc2\x8d", r'\x8D').replace("\xc2\x8e", r'\x8E')\
        .replace("\xc2\x8f", r'\x8F').replace("\xc2\x90", r'\x90').replace("\xc2\x91", r'\x91')\
        .replace("\xc2\x92", r'\x92').replace("\xc2\x93", r'\x93').replace("\xc2\x94", r'\x94')\
        .replace("\xc2\x95", r'\x95').replace("\xc2\x96", r'\x96').replace("\xc2\x97", r'\x97')\
        .replace("\xc2\x98", r'\x98').replace("\xc2\x99", r'\x99').replace("\xc2\x9a", r'\x9A')\
        .replace("\xc2\x9b", r'\x9B').replace("\xc2\x9c", r'\x9C').replace("\xc2\x9d", r'\x9D')\
        .replace("\xc2\x9e", r'\x9E').replace("\xc2\x9f", r'\x9F').replace("\xc2\xa0", r'\xA0')\
        .replace("\xc2\xa1", r'\xA1').replace("\xc2\xa2", r'\xA2').replace("\xc2\xa3", r'\xA3')\
        .replace("\xc2\xa4", r'\xA4').replace("\xc2\xa5", r'\xA5').replace("\xc2\xa6", r'\xA6')\
        .replace("\xc2\xa7", r'\xA7').replace("\xc2\xa8", r'\xA8').replace("\xc2\xa9", r'\xA9')\
        .replace("\xc2\xaa", r'\xAA').replace("\xc2\xab", r'\xAB').replace("\xc2\xac", r'\xAC')\
        .replace("\xc2\xad", r'\xAD').replace("\xc2\xae", r'\xAE').replace("\xc2\xaf", r'\xAF')\
        .replace("\xc2\xb0", r'\xB0').replace("\xc2\xb1", r'\xB1').replace("\xc2\xb2", r'\xB2')\
        .replace("\xc2\xb3", r'\xB3').replace("\xc2\xb4", r'\xB4').replace("\xc2\xb5", r'\xB5')\
        .replace("\xc2\xb6", r'\xB6').replace("\xc2\xb7", r'\xB7').replace("\xc2\xb8", r'\xB8')\
        .replace("\xc2\xb9", r'\xB9').replace("\xc2\xba", r'\xBA').replace("\xc2\xbb", r'\xBB')\
        .replace("\xc2\xbc", r'\xBC').replace("\xc2\xbd", r'\xBD').replace("\xc2\xbe", r'\xBE')\
        .replace("\xc2\xbf", r'\xBF').replace("\xc3\x80", r'\xC0').replace("\xc3\x81", r'\xC1')\
        .replace("\xc3\x82", r'\xC2').replace("\xc3\x83", r'\xC3').replace("\xc3\x84", r'\xC4')\
        .replace("\xc3\x85", r'\xC5').replace("\xc3\x86", r'\xC6').replace("\xc3\x87", r'\xC7')\
        .replace("\xc3\x88", r'\xC8').replace("\xc3\x89", r'\xC9').replace("\xc3\x8a", r'\xCA')\
        .replace("\xc3\x8b", r'\xCB').replace("\xc3\x8c", r'\xCC').replace("\xc3\x8d", r'\xCD')\
        .replace("\xc3\x8e", r'\xCE').replace("\xc3\x8f", r'\xCF').replace("\xc3\x90", r'\xD0')\
        .replace("\xc3\x91", r'\xD1').replace("\xc3\x92", r'\xD2').replace("\xc3\x93", r'\xD3')\
        .replace("\xc3\x94", r'\xD4').replace("\xc3\x95", r'\xD5').replace("\xc3\x96", r'\xD6')\
        .replace("\xc3\x97", r'\xD7').replace("\xc3\x98", r'\xD8').replace("\xc3\x99", r'\xD9')\
        .replace("\xc3\x9a", r'\xDA').replace("\xc3\x9b", r'\xDB').replace("\xc3\x9c", r'\xDC')\
        .replace("\xc3\x9d", r'\xDD').replace("\xc3\x9e", r'\xDE').replace("\xc3\x9f", r'\xDF')\
        .replace("\xc3\xa0", r'\xE0').replace("\xc3\xa1", r'\xE1').replace("\xc3\xa2", r'\xE2')\
        .replace("\xc3\xa3", r'\xE3').replace("\xc3\xa4", r'\xE4').replace("\xc3\xa5", r'\xE5')\
        .replace("\xc3\xa6", r'\xE6').replace("\xc3\xa7", r'\xE7').replace("\xc3\xa8", r'\xE8')\
        .replace("\xc3\xa9", r'\xE9').replace("\xc3\xaa", r'\xEA').replace("\xc3\xab", r'\xEB')\
        .replace("\xc3\xac", r'\xEC').replace("\xc3\xad", r'\xED').replace("\xc3\xae", r'\xEE')\
        .replace("\xc3\xaf", r'\xEF').replace("\xc3\xb0", r'\xF0').replace("\xc3\xb1", r'\xF1')\
        .replace("\xc3\xb2", r'\xF2').replace("\xc3\xb3", r'\xF3').replace("\xc3\xb4", r'\xF4')\
        .replace("\xc3\xb5", r'\xF5').replace("\xc3\xb6", r'\xF6').replace("\xc3\xb7", r'\xF7')\
        .replace("\xc3\xb8", r'\xF8').replace("\xc3\xb9", r'\xF9').replace("\xc3\xba", r'\xFA')\
        .replace("\xc3\xbb", r'\xFB').replace("\xc3\xbc", r'\xFC').replace("\xc3\xbd", r'\xFD')\
        .replace("\xc3\xbe", r'\xFE').replace("\xc3\xbf", r'\xFF')

How to decode a string encoded by C# Convert.ToBase64String in python 3

2 Answers2

Linked