Stack Overflow
Stack Overflow

Questions tagged [utf-16le]

UTF-16LE is the little endian variety of UTF-16 without BOM.

UTF-16LE is the little endian variant of . While text in UTF-16 might be expected to signal endianness by starting with a Byte-Order-Mark, text in UTF-16LE should not. UTF-16LE can encode all code points in two or four bytes, like UTF-16.

UTF-16LE is the encoding used for the API, and many frameworks there. Most stored text on Windows is actually instead . Text in both formats on Windows often start with a BOM, which can confuse software not expecting it.

For any more details, consider instead.

82 questions
2
votes
3 answers

How to deal with UTF-16LE encoded text file using Java? or convert it to ASCII?

I am sorry if it has been asked before. I am trying to process a text file using Java. The text file is exported from MS SQLServer. When I open it in PSPad (sort of text editor in which I can view any file in hex format), it tells me that my text…
Bhushan
  • 18,329
  • 31
  • 104
  • 137
2
votes
2 answers

Javascript string to Base64 UTF-16BE

I'm trying to convert a string into BASE64 and a charset of utf-16 Big Endian in order to send it using an sms API. I'm not being able to do so in Javascript. This is the original js string I want to send in the sms: const originalString = 'Teste…
Eunito
  • 416
  • 5
  • 22
2
votes
2 answers

How to use Ruby's readlines.grep for UTF-16 files?

Given the following two files created by the following commands: $ printf "foo\nbar\nbaz\n" | iconv -t UTF-8 > utf-8.txt $ printf "foo\nbar\nbaz\n" | iconv -t UTF-16 > utf-16.txt $ file utf-8.txt utf-16.txt utf-8.txt: ASCII text utf-16.txt:…
kenorb
  • 155,785
  • 88
  • 678
  • 743
2
votes
2 answers

Reading and Writing Text files with UTF-16LE encoding and Apache Commons IO

I have written an application in Java and duplicated it in C#. The application reads and writes text files with tab delimited data to be used by an HMI software. The HMI software requires UTF or ANSI encoding for the degree symbol to be displayed…
user7338252
2
votes
1 answer

OSX perl to batch write filename as first line in txt file in UTF-16LE

I found a really useful bit of perl here that writes the filename of a text file to the first line of the file. I am running this from terminal in OS X Yosemite: perl -i -pe 'BEGIN{undef $/;} s/^/\nFilename:$ARGV\n/' `find . -name '*.TXT'` With…
Irenabyss
  • 43
  • 6
2
votes
1 answer

Why Groovy file write with UTF-16LE produce BOM char?

Do you have idea why first and secod lines below do not produce BOM to the file and third line does? I thought UTF-16LE is correct encoding name and that encoding does no create BOM automatically to beginning of the file. new…
JukkaU
  • 55
  • 7
2
votes
2 answers

utf-16-le BOM csv files

I'm downloading some CSV files from playstore (stats etc) and want to process with python. cromestant@jumphost-vpc:~/stat_dev/bime$ file -bi stats/installs/* text/plain; charset=utf-16le text/plain; charset=utf-16le text/plain;…
cromestant
  • 652
  • 2
  • 10
  • 21
2
votes
2 answers

How can I convert a string to UTF-16LE and then to Base64?

I'm throwing together a program as a refresher for VB.net, and I figured I might as well make it do something that I have to do a lot anyways: Convert an input string into UTF-16LE and then into Base64. Now, in PHP, I can do it like…
Austin Burk
  • 930
  • 4
  • 15
  • 33
2
votes
1 answer

What to include when using Encode with PAR Packer

I've been working with PAR::Packer to create standalone executable files out of Perl scripts. The only difficulty I have with it is figuring out what extra DLLs I have to force it to include via the -l option. I'm now working with a Perl script that…
Nate Glenn
  • 6,455
  • 8
  • 52
  • 95
1
vote
0 answers

C++ Converting utf-16 LE BOM to utf-8

I am trying to read a .txt file which is encoded with UTF-16 LE BOM, and trying to display it in my C++ GUI. But the only options I found online are deprecated. Stuff I tried: Reading UTF-16 file in c++ How to set file encoding format to UTF8 in…
Zein
  • 11
  • 2
1
vote
1 answer

How to save txt file with UTF-16 LE BOM encoding in Python

I have files encoded with UTF16 LE BOM encoding and need to alter them and save. However I could not see such encoding option in https://docs.python.org/3.10/library/codecs.html#standard-encodings My code: with open("file.txt", mode='w',…
Mirek Hotový
  • 31
  • 5
1
vote
3 answers

Why does utf-16 only support 2^20 code points?

Well, I'm starting to study unicode now, and I had several doubts, at this moment I'm learning what a plane is, I saw that a plane is a set of 2^16 code points, and that utf-16 encoding supports 17 plans enumerated from 0 to 16, well my question is…
Yuri Bittencourt
  • 91
  • 8
1
vote
0 answers

Using Python in pandas.read_sql from ODBC - UnicodUnicodeDecodeError: 'utf-16-le' codec can't decode bytes in position 34-35: illegal UTF-16 surrogate

I get this error when I include certain fields in a query. I have looked at other solutions on the same error and not quite finding what I need or at least understanding how. Any other solutions to this problem? To give more context on what I am…
Wes
  • 11
  • 2
1
vote
1 answer

How to encode to UTF16 Little Endian in Dart?

I am attempting to manipulate some system variables used by a program using Dart. I have encountered the problem of dart's utf package being discontinued, and I have not found any way to encode to UTF 16 Little Endian for a File.write. Is there a…
Jack1221
  • 15
  • 4
1
vote
1 answer

Default Output File in VS Code being stored in UTF 16 LE whereas default is set to UTF-8

I am working on windows 10 system and using VS Code. VS Code -v : 1.48.2 gcc version : gcc (MinGW.org GCC-8.2.0-5) 8.2.0 I am compiling a c program using "gcc program.c -o a" (program.c is the name of the C file). I store the output of a C file…
this_is_ash_shar
  • 53
  • 7
1
2
3 4 5 6