GoLang tailing UTF16LE windows log file

Question

How can I combine these two golang scripts to tail an active log that is UTF16LEBOM?

I am working on a log parser for a game (Eve Online). In the game chat logs can be saved, and I would like to use GoLang to read the running log and flag on keywords "die" in "prepare to die scumsucker!"

I've found two examples that work separately. Combining them is a challenge I've been unable to figure out. First tail equivalent here: https://medium.com/@arunprabhu.1/tailing-a-file-in-golang-72944204f22b

Second reading the file so that is legible and doesn't have extra characters: https://github.com/TomOnTime/utfutil/

I have been trying to replace the code from utfutil with the "equivalent" but it seems it is missing several functions.

// EDIT Almost working. It doesn't look like utfutil is defering the close.

package main

import (
    "bufio"
    "fmt"
    "io"
    "time"
    
    "github.com/TomOnTime/utfutil"
)

func main() {
    logfilefile := "C:/Users/user/Documents/EVE/logs/Chatlogs/chat_20220709_022129_1006197774.txt"
    file, err := utfutil.OpenFile(logfilefile, utfutil.WINDOWS)
    if err != nil {
        return
    }

    defer file.Close()

    reader := bufio.NewReader(file)
    for {
        line, err := reader.ReadString('\n')
        if err != nil {
            if err == io.EOF {
                // without this sleep you would hogg the CPU
                time.Sleep(500 * time.Millisecond)
                continue
            }

            break
        }

        fmt.Printf(string(line))
    }
}

<cut white space>
        ---------------------------------------------------------------

          Channel ID:      local
          Channel Name:    Local
          Listener:        Steve
          Session started: 2022.07.07 16:18:21
        ---------------------------------------------------------------

[ 2022.07.07 17:11:44 ] Steve > hello world
[ 2022.07.07 17:11:48 ] John > hello world
[ 2022.07.07 17:11:51 ] James > hello world
[ 2022.07.07 19:36:53 ] Bob > hello world

Sorry - I cannot see a question in the above? (please take a look at [how to ask](https://stackoverflow.com/help/how-to-ask) and note that it you are looking for library recommendations then that is [off topic](https://stackoverflow.com/help/on-topic) here). — Brits, Jul 08 '22 at 05:10
Golang's strings are UTF8, you need to convert your logs first. https://stackoverflow.com/questions/15783830/how-to-read-utf16-text-file-to-string-in-golang — Dmitry Harnitski, Jul 08 '22 at 12:53
@brits Added specific question "How can I combine these two golang scripts to tail an active log that is UTF16LEBOM?" — Gaudard, Jul 09 '22 at 05:16
@DmitryHarnitski I can't convert the log because it is being written to. — Gaudard, Jul 09 '22 at 05:16
Almost working. It looks like it isn't using "defer file.Close()". My console window hangs, but doesn't not print any further logs. — Gaudard, Jul 09 '22 at 07:22

score 1 · Accepted Answer · answered Jul 09 '22 at 22:47

You can simplify your code by removing github.com/TomOnTime/utfutil (this is a very thin wrapper over golang.org/x/text/encoding/unicode). It's also often simpler to chain tasks rather than trying to do everything at once; here I'm borrowing tailReader (with a minor change) from this answer.

Note: I have only tested the below very quickly (and don't have an "Eve Online" logfile to hand).

package main

import (
    "bufio"
    "fmt"
    "io"
    "os"
    "time"

    "golang.org/x/text/encoding/unicode"
)

func main() {
    file, err := newTailReader("./Local_20220707_170827_1006197774.txt")
    if err != nil {
        return
    }
    defer file.Close()

    utf := unicode.UTF16(unicode.LittleEndian, unicode.IgnoreBOM)
    reader := bufio.NewReader(utf.NewDecoder().Reader(file))

    for {
        line, err := reader.ReadString('\n')
        if err != nil {
            fmt.Println(err)
            break
        }

        fmt.Printf(string(line))
    }

}

// Code copied from https://stackoverflow.com/a/31122253/11810946
// and modified to output contents of file before beginning to 'tail'
type tailReader struct {
    io.ReadCloser
}

func (t tailReader) Read(b []byte) (int, error) {
    for {
        n, err := t.ReadCloser.Read(b)
        if n > 0 {
            return n, nil
        } else if err != io.EOF {
            return n, err
        }
        time.Sleep(10 * time.Millisecond)
    }
}

func newTailReader(fileName string) (tailReader, error) {
    f, err := os.Open(fileName)
    if err != nil {
        return tailReader{}, err
    }
    return tailReader{f}, nil
}

GoLang tailing UTF16LE windows log file

1 Answers1