4

I use Apache Tika to get encoding of file.

            FileInputStream fis = new FileInputStream(my_file);
            final AutoDetectReader detector = new AutoDetectReader(fis);
            fis.close();
            System.out.println("Encoding:" + detector.getCharset().toString());

I use Scanner to read values from file.

                Scanner scanner = new Scanner(my_file, detector.getCharset().toString());
                Map<String, String> values = new HashMap<>();
                String line, key = null, value = null;
                while (scanner.hasNextLine()) {
                    line = scanner.nextLine();
                    if (line.contains(":")) {
                        if (key != null) {
                            values.put(key, value.trim());
                            key = null;
                            value = null;
                        }
                        int indexOfColon = line.indexOf(":");
                        key = line.substring(0, indexOfColon);
                        value = line.substring(indexOfColon + 1);
                    } else {
                        value += " " + line;
                    }
                }

Scanner is unable to read text from files with encoding windows-1252, I get empty string.

UPDATE 2018.11.07. I have same problem in case of BufferedReader.

                    Map<String, String> values = new HashMap<>();
                    String line, key = null, value = null;
                    FileInputStream is = new FileInputStream(my_file);
                    InputStreamReader isr = new InputStreamReader(is, getEncoding(my_file));
                    BufferedReader buffReader = new BufferedReader(isr);

                    while (buffReader.readLine() != null) {
                        line = buffReader.readLine();
                        if (line.contains(":")) {
                            if (key != null) {
                                values.put(key, value.trim());
                                key = null;
                                value = null;
                            }
                            int indexOfColon = line.indexOf(":");
                            key = line.substring(0, indexOfColon);
                            value = line.substring(indexOfColon + 1);
                        } else {
                            value += " " + line;
                        }
                    }
plaidshirt
  • 5,189
  • 19
  • 91
  • 181

1 Answers1

0

Instead of reading lines, I would try reading characters instead using the following approach:

ByteArrayOutputStream line = new ByteArrayOutputStream();
Scanner scanner = new Scanner(my_file);

while (scanner.hasNextInt()) {
    int c = 0;
    // read every line
    while (c != newline) { // TODO: Check for a newline char
        c = scanner.nextInt();
        line.write((byte) c);
    }
    byte[] array = line.toByteArray();
    String output = new String(array, "Windows-1252"); // This should do the trick

    // We have a string here, do your logic

    line.reset();
}

This approach is ugly, but uses new String which has the ability to specify a specific encoding. I did not test or run this code at all, but at least it will show you if any content is actually read properly.

gi097
  • 7,313
  • 3
  • 27
  • 49