0

There is a compressed text file(.gz) has some set of warning, error and information details.

Sample content.

Oct 25 06:58:51 : For info, please visit https://www.det.org
Oct 25 06:58:51 : Copyright 2004-2018 Internet Systems Consortium.
Oct 25 06:58:51 : All rights reserved.
Oct 25 06:58:51 : ERROR: Host declarations are global.
Oct 25 06:58:51 : WARNING: Host declarations are global

I need to get the warning and error complete line from the text file without uncompressed using golang. How can develop an algorithm for this?

Ibjas Mohamed
  • 73
  • 6
  • 8
    Open the file, use the [`compress/gzip`](https://pkg.go.dev/compress/gzip@go1.17.6) package to decompress it (as a stream), use [`bufio.Scanner`](https://pkg.go.dev/bufio#Scanner) to read by lines and check for `ERROR:` or `WARNING:` substrings using [`strings.Index()`](https://pkg.go.dev/strings#Index). – icza Jan 18 '22 at 17:39
  • Reading line by line and using substring contain is not optimize way.. if i have 1 million line but i have only 2 warning. in this case this login is not fit. – Ibjas Mohamed Jan 18 '22 at 17:45
  • https://serverfault.com/questions/927956/how-to-print-the-last-line-of-a-gz-compressed-file-in-the-command-line additional reading about how this is impossible – erik258 Jan 19 '22 at 05:12
  • Thanks @DanielFarrell your suggestion is very useful and It opens the the way for different approach. we can use zgrep command also we can use "os/exec" in goalng to achieve open the file looping the content is not a optimized way. – Ibjas Mohamed Jan 20 '22 at 18:17
  • it's true - gzip is not optimized for random access. It's optimized for streaming compressin/decompression. I believe eg `zgrep` or `zcat` do the same thing - start at beginning, decompress all file – erik258 Jan 20 '22 at 19:57

1 Answers1

1

Here is the answer I solved above question.

cmd := exec.Command(`zgrep`, `WARNING\|ERROR`, filename)
stdout, err := cmd.StdoutPipe()

if err != nil {
    log.Fatal(err)
}

if err := cmd.Start(); err != nil {
    log.Fatal(err)
}

data, err := ioutil.ReadAll(stdout)

if err != nil {
    log.Fatal(err)
}

if err := cmd.Wait(); err != nil {
    
        log.Fatalf("cmd.Wait: %v", err)
    
}
lines := strings.Split(string(data), "\n")
for _, line := range lines {
    fmt.Println(line)
    
}
Ibjas Mohamed
  • 73
  • 6