Questions tagged [utf-8]

UTF-8 is a character encoding that describes each Unicode code point using a byte sequence of one to four bytes. It is backwards-compatible with ASCII while still supporting representation of all Unicode code points.

UTF-8 is a character-encoding that can describe the set of unicode code points in byte sequences of one to four bytes.

UTF-8 is the most widely used character encoding, and is recommended for use on the Internet. It is the standard character encoding on linux and other recent unix-like operating systems. It was designed to be backwards-compatible with ascii while still supporting representation of all Unicode code points.

The algorithm for encoding code points in UTF-8 is described in RFC 3629.

Related tags

The character-encoding tag discusses the general concept of character-set encodings
The unicode character set can be represented in a variety of encodings, one of which is UTF-8
The ascii character set and encoding it generalizes
Other UTFs: utf-16 utf-32, rarely used: utf-7 utf-1 utf-18 utf-36 utf8mb4

22178 questions

votes

1 answer

Java 8 change in UTF-8 decoding

We recently migrated our application to JDK 8 from JDK 7. After the change, we ran into a problem with the following snippet of code. String output = new String(byteArray, "UTF-8"); The byte array may contain invalid UTF-8 byte sequences. The same…

java utf-8 java-8 regression

asked Jun 01 '15 at 13:59

Jiraiya

votes

4 answers

Laravel 5 charset not working correctly on the views. But it working well when I dump it from controller

I'm facing a charset problem here. I'm developing an app that uses a sql server database. The database was not created for this app, it exists before it and works very well. I can't change anything on the database because its too large and its used…

php laravel utf-8 character-encoding laravel-5

asked May 22 '15 at 14:20

Anderson Silva

votes

3 answers

How to send UTF-8 encoded email body with JavaMailSenderImpl?

I am sending an email this way: @Test public void testEmailCharacterSet() throws MessagingException, UnsupportedEncodingException { JavaMailSenderImpl mailSender = new JavaMailSenderImpl(); mailSender.setDefaultEncoding("utf-8"); …

java email encoding utf-8

asked May 17 '15 at 16:49

jabal

11,987
12
51
99

votes

3 answers

Default encoding of HTTP POST request with JSON body

What's the default encoding of HTTP POST request when the content-type is "application/json" with no explicit charset given"? It seems two specs are in conflicts: JSON spec says that "JSON text SHALL be encoded in Unicode. The default encoding is…

json http encoding utf-8 character-encoding

asked Apr 21 '15 at 02:37

Kwang Yul Seo

votes

3 answers

Migrating MySQL UTF8 to UTF8MB4 problems and questions

Im trying to convert my UTF8 MySQL 5.5.30 database to UTF8MB4. I have looked at this article https://mathiasbynens.be/notes/mysql-utf8mb4 but have some questions. I have done these ALTER DATABASE database_name CHARACTER SET = utf8mb4 COLLATE =…

mysql utf-8 migrate utf8mb4

asked Mar 22 '15 at 12:19

Banshee

15,376
38
128
219

votes

3 answers

php reading mysql bit field returning weird character

I am using mysql_fetch_assoc($query), one of the bit field returns out to be , which is supposedly to be true. The problem is that I also need to output this to xml and it's an illegal xml character. the charset for the db table is utf-8. why does…

php mysql utf-8

asked May 26 '10 at 16:10

user121196

30,032
57
148
198

votes

3 answers

How to handle example data in R Package that has UTF-8 marked strings

I would like to include an example dataset (of Twitter tweets and metadata) in an R Package I'm writing. I downloaded an example data.frame using the Twitter API and saved it as .RData (with the corresponding .R data description file) in my…

r twitter utf-8

asked Mar 14 '15 at 00:10

Rocinante

votes

3 answers

Publishing DACPAC file with MSDeploy, UTF8 characters in post-deploy script are lost

I have a DACPAC file that was built in Visual Studio 2013, for an SSDT project. This SSDT project defines a post-deploy script designed to merge some static data into the published tables, and one piece of data contains a copyright symbol. Now,…

sql-server utf-8 msdeploy sql-server-data-tools dacpac

asked Mar 13 '15 at 21:26

Michael J. Heier

votes

2 answers

Setting shell script to utf8

I want to write to following command line into a shell script: cat text.tsv | grep -Pvi '.\t.\t.*\bHotels|Гостиница|Готель|Отель|Хотел|ホテル|מלון|فندق|होटल|โรงแรม|숙박|호텔|宾馆|旅店|旅馆|酒店|飯店\b' | awk '{print $0,"\t","column1"} > Text2.tsv However when I put…

bash shell utf-8 ascii

asked Feb 12 '15 at 12:45

Vic23

votes

2 answers

How to easily detect utf8 encoding in the string?

I have string which fill up by data from other program and this data can be with UTF8 encoding or not. So if not i can encode to UTF8 but what is the best way to detect UTF8 in the C++? I saw this variant https://stackoverflow.com/questions/... but…

c++ windows string encoding utf-8

asked Feb 02 '15 at 02:23

ratojakuf

votes

3 answers

multi-byte characters in libc regcomp and regexec

Is there anyway to get libc6's regexp functions regcomp and regexec to work properly with multi-byte characters? For instance, if my pattern is the utf8 characters 猫机+猫, finding a match on the utf8 encoded string 猫机机机猫 will fail, where it should…

regex utf-8 glibc libc multibyte-functions

asked Jan 23 '15 at 17:52

bill_e

votes

2 answers

How to convert any possible format to UTF-8 using Iconv?

so for example this will turn 1251 into utf-8. $utf8 = iconv('windows-1251', 'utf-8', $ansi); But how to turn unknown (when it comes to us we do not know yet what format it is) ( in general any ) format (possibly known by Iconv ) to utf-8? (code…

php utf-8 iconv

asked May 10 '10 at 21:46

Rella

65,003
109
363
636

votes

5 answers

Remove characters not-suitable for UTF-8 encoding from String

I have a text-area on website where user can write anything. Problem happens when user copy paste some text or something which contains non-UTF 8 characters and submit them to server. Java successfully handles it, as it support UTF-16 but my mySql…

java mysql string encoding utf-8

asked Jan 06 '15 at 08:43

Abhinav

3,322
9
47
63

votes

1 answer

Django UnicodeEncodeError in rendering form ('utf-8')

I got an UnicodeEncodeError while rendering page using forms. UnicodeEncodeError at /individual/acc/ 'ascii' codec can't encode character u'\u0142' in position 2: ordinal not in range(128) Here's fragment of HTML (standard use of forms): …

python django forms unicode utf-8

asked Jan 05 '15 at 13:55

Mateusz Knapczyk

votes

3 answers

Open a text file with accents in python

I try to open a text file in French with Python 2.7. I used the command f=open('textfr','r') but when I use f.read() I lose accented characters: I get u"J'\xc3\xa9tais \xc3\xa0 Paris instead of J'étais à Paris, etc.. when in linux terminal, I do…

python utf-8

asked Dec 21 '14 at 18:10

Mostafa

1,501
3
21
37

Prev 1 2 3

…

100