Dart/Flutter UTF-16 encoding

Question

I need to read in a file that's been saved in Java Android Studio with this code

fos = context.openFileOutput(filename, Context.MODE_PRIVATE);
ObjectOutputStream os = new ObjectOutputStream(fos);
os.writeObject(json);
os.close();
fos.close();

It turns out that Java uses internally UTF-16 for encoding the characters. I now need to read this files in Flutter SDK (with the Dart language)

file.readAsString(encoding: Latin1).then((str) => {
       print(str)
     }).catchError( (e) => {
       print(e)
});

However, the Latin1 encoding doesn't work perfectly. So I want to read it in using UTF-16 but it seems that Dart does simply not have this functionality, or at least not that I can find.

Utf8Codec does exist but there's not Utf16Codec nor Encoding.getByName("UTF-16"). Utf8 is by the way giving exceptions so this is also no option.

So how can I still read in files in Dart which have been saved in Android Studio Java using UTF-16?

Based on your other question, I'd guess that the bytes are *not* in UTF-16, but just have a Java object stream prefix. — Richard Heap, Sep 18 '19 at 18:17
You probably would be interested in this related question: [UTF-16LE txt file decode as String in Flutter (dart)](https://stackoverflow.com/questions/57943867/utf-16le-txt-file-decode-as-string-in-flutter-dart) — jamesdlin, Sep 19 '19 at 05:47

score 4 · Answer 1 · answered Sep 18 '19 at 17:55

4

Use the utf package.

  var bytes = await file.readAsBytes();
  var decoder = Utf16BytesToCodeUnitsDecoder(bytes); // use le variant if no BOM
  var string = String.fromCharCodes(decoder.decodeRest());

answered Sep 18 '19 at 17:55

Richard Heap

48,344
9
130
112

3

that package is discontinued – Cliff Helsel May 07 '20 at 13:31
I discovered that it still works though. Lucky because it solved a big problem, Phew! – Kretin Aug 13 '20 at 03:52
utf package is discontinued – M.ArslanKhan Oct 04 '20 at 13:20

score 1 · Answer 2 · answered Dec 15 '22 at 20:27

  final File file = File(filePath);

  final bytes = file.readAsBytesSync();

  final utf16CodeUnits = bytes.buffer.asUint16List();

  final s = String.fromCharCodes(utf16CodeUnits);

I was able to turn a UTF-16 file into a String using these 4 lines to avoid using the discontinued package. Hope this helps!

score -3 · Answer 3 · answered Oct 04 '20 at 16:08

-3

Use Base64. Importing 'dart:convert'.

  var bytes = await file.readAsBytes();
  final encoded = base64.encode(bytes);
  debugPrint("onPressed " + encoded);

answered Oct 04 '20 at 16:08

M.ArslanKhan

3,640
8
34
56

3

Base64 has nothing to do with UTF-16. – jamesdlin Jun 02 '21 at 10:06

Dart/Flutter UTF-16 encoding

3 Answers3