Questions tagged [encoding]

Encoding is a set of predefined rules to reversibly transform a piece of information in a certain representation into a completely different representation. The other way round is called decoding. This tag is rather generic, but it is mainly used for binary encoding schemes such as base 64 and hexadecimal.

There are a lot of different applications:

  • which is how the computer represents characters like a and , which humans can recognize, into bytes, which computers can recognize.
  • which is used to transform between videos and bytes.
  • which is used to transform between plain text and valid URIs. Also known as .
  • which is used to transform between plain text and valid XML.
  • which is used to compress/decompress bytes.
24174 questions
7
votes
1 answer

Questions about converting a mixed-encoding file to UTF8 in Perl

I'm in the process of converting files generated by the ancient DOS-based library program of our university's Chinese Studies Department into something more useful and accesible. Among the problems I'm dealing with is that the exported text files…
screen12345
  • 117
  • 1
  • 6
7
votes
2 answers

Sending an amp (&) using a post with Obj-C

I am sending a post containing text, numbers and data. The numbers and data work fine, but I'm having problems with the text, since it may contain an ampersand (&). For example page.php?text=Hello World & Space. Now I found that the "&" is received…
Max Z.
  • 801
  • 1
  • 9
  • 25
7
votes
3 answers

decoding shift-jis: "illegal multibyte sequence"

I'm trying to decode a shift-jis encoded string, like this: string.decode('shift-jis').encode('utf-8') to be able to view it in my program. When I come across 2 shift-jis characters, in hex "0x87 0x54" and "0x87 0x55", I get this…
ben
  • 71
  • 1
  • 1
  • 2
7
votes
4 answers

Making Win32 Application work on ANSI & UNICODE

I want to make my Win32 C++ application able to be played on any encoding version (UNICODE & ANSI). Now I am a little confused as to what exactly is the difference between the two(or more?) encodings? To make my Win32 application cross-encoding…
user593747
  • 1,484
  • 4
  • 32
  • 52
7
votes
4 answers

How can I check if char encoding is ASCII?

I would like to write the following function: int char_index(char c) { if (is_ascii) return c - 'A'; else return c == 'A' ? 0 : c == 'B' ? 1 : // ... } Is there a function like is_ascii in std? I'm…
cigien
  • 57,834
  • 11
  • 73
  • 112
7
votes
4 answers

Convert "\x" escaped string into readable string in python

Is there a way to convert a \x escaped string like "\\xe8\\xaa\\x9e\\xe8\\xa8\\x80" into readable form: "語言"? >>> a = "\\xe8\\xaa\\x9e\\xe8\\xa8\\x80" >>> print(a) \xe8\xaa\x9e\xe8\xa8\x80 I am aware that there is a similar question here, but it…
oeter
  • 627
  • 2
  • 8
  • 23
7
votes
2 answers

How to decode nvarchar to text (SQL Server 2008 R2)?

I have a SQL Server 2008 R2 table with nvarchar(4000) field. Data that stores this table look like '696D616765206D61726B65643A5472' or '303131' ("011"). I see that each char is encoding to hex. How can I read those data from table? I don't want…
xieergai
  • 165
  • 2
  • 6
7
votes
5 answers

MemoryStream from string - confusion about Encoding to use

I have a piece of code that converts string into memory stream: using (MemoryStream stream = new MemoryStream(Encoding.UTF8.GetBytes(applicationForm))) However I'm a bit confused if it's correct. Basically I'm always confused about .NET…
dragonfly
  • 17,407
  • 30
  • 110
  • 219
7
votes
1 answer

Invalid data when creating mkv container with h264 stream because extradata is null

I need to encode raw frames into an h264 stream inside an mkv container using ffmpeg's c api. All the examples I could find copy existing codec parameters from a decoder so I had to modify it a bit. #include #include #include…
pixelou
  • 748
  • 6
  • 17
7
votes
1 answer

Save file with ANSI encoding in VS Code

I have a text file that needs to be in ANSI mode. It specifies only that: ANSI. Notepad++ has an option to convert to ANSI and that does the trick. In VS Code I don't find this encoding option. So I read up on it and it looks like ANSI doesn't…
bluppfisk
  • 2,538
  • 3
  • 27
  • 56
7
votes
2 answers

C Thread safe locale/encoding when using sprintf and printf

This question is not related to something that i want to do at the moment, but something i wondered while reading the (GNU/Linux/ISO-C) Documentation for sprintf() printf() and setlocale(). The hypothetical problem: Imagine a multithreaded…
frhun
  • 160
  • 6
7
votes
4 answers

RuntimeError: Click will abort further execution because Python 3 was configured to use ASCII as encoding for the environment

I tried to deploy my flask application in a docker in ubuntu18.04, and I was using python3.5 in my pipenv. But when I run docker build -t flask ., at a step of RUN pipenv install,I got this error: RuntimeError: Click will abort further execution…
bohnsix
  • 111
  • 2
  • 4
7
votes
3 answers

Is Windows 1252 a subset of UTF-8 or not?

I just want to know if windows 1252 is a subset of UTF-8 or not? and what are the differences? Thinking of migrating my DB from windows 1252 to UTF-8, any thoughts, opinions?
samg
  • 311
  • 1
  • 8
  • 21
7
votes
1 answer

open-uri returning ASCII-8BIT from webpage encoded in iso-8859

I am using open-uri to read a webpage which claims to be encoded in iso-8859-1. When I read the contents of the page, open-uri returns a string encoded in ASCII-8BIT. open("http://www.nigella.com/recipes/view/DEVILS-FOOD-CAKE-5310") {|f| p…
mkhettry
  • 71
  • 1
  • 2
7
votes
1 answer

Difference Between ASCIIEncoding and Encoding

I understand that Encoding can be used to initialize object to perform any type of Encoding, ASCII, Unicode, UTF-8 etc. It appears to me that all these are sufficient for performing any kind of encoding, then what is the need for ASCIIEncoding?
Shamim Hafiz - MSFT
  • 21,454
  • 43
  • 116
  • 176