Latin Character Inbetween String

Question

I have a program to read in a file which contain latin words such as "\xed". These latin words can appear anywhere in between any line, as such I have program parsing these character. Is there any library that can do so?

Answer 1

The simple way I often do is InputStreamReader with "UTF8" format. For example:

         try {
            File fileDir = new File("c:/temp/sample.txt");

            BufferedReader in = new BufferedReader(
                    new InputStreamReader(
                            new FileInputStream(fileDir), "UTF8"));

            String str;

            while ((str = in.readLine()) != null) {
                System.out.println(str);
            }

            in.close();
        } 
        catch (UnsupportedEncodingException e) 
        {
            System.out.println(e.getMessage());
        } 
        catch (IOException e) 
        {
            System.out.println(e.getMessage());
        }
        catch (Exception e)
        {
            System.out.println(e.getMessage());
        }

Answer 2

If you mean that the text is in bytes and you have a byte with the hex value ED, then the interpretation of that byte depends on your code page.

Java stores all String's internally in UTF-16. This means that a code page conversion is pretty much always applied when reading and writing file (UTF-16 is not a common file encoding).

By default, Java will use the platform default character set. If this is not the correct one, you have to specify the Charset to use.

As an example of the problem, byte ED is:

• ISO-8859-1: í (unicode 00ED) US Windows
• Windows-1251: н (unicode 043D) Russian
• Code page 437: φ (unicode 03C6) US Windows command-line (Win 7)

To control the code page, read the file like this:

File file = new File("C:\\path\\to\\file.txt");
try (BufferedReader in = new BufferedReader(new InputStreamReader(new FileInputStream(file), "ISO-8859-1"))) {
    String line;
    while ((line = in.readLine()) != null) {
        // process line here
    }
}

Or with the newer Path API:

Path path = Paths.get("C:\\path\\to\\file.txt");
try (BufferedReader in = Files.newBufferedReader(path, Charset.forName("ISO-8859-1"))) {
    String line;
    while ((line = in.readLine()) != null) {
        // process line here
    }
}