Read in lines in a text file, sort, then overwrite file

Question

I am trying to write a go function that will read in lines in a text file, sort them (alphabetize), and overwrite them back to the file. Right now, I am able to essentially emulate cat, but I can't seem to be able to manipulate the contents of the elements in read_line.

func sort() {

    ff, _ := os.OpenFile(file, os.O_RDWR, 0666)
    f := bufio.NewReader(ff)
    for {
        read_line, _ := f.ReadString('\n')
        fmt.Print(read_line)
        if read_line == "" {
            break
        }
    }
    ff.Close()
}

when i use ReadString, how can i store each line into a slice (or is there a better way to store them so i can manipulate them)? Then I would use the sort package in a manner similar to this:

sorted := sort.Strings(lines)

then, to write to the file, i am using something similar to the following, although i have not included it because i have not yet gotten "sort" to work:

io.WriteString(ff, (lines + "\n"))

Thank you in advance for any suggestions!

peterSO · Accepted Answer · 2011-09-15T02:56:13.763

For example,

package main

import (
    "bufio"
    "fmt"
    "os"
    "sort"
)

func readLines(file string) (lines []string, err os.Error) {
    f, err := os.Open(file)
    if err != nil {
        return nil, err
    }
    defer f.Close()
    r := bufio.NewReader(f)
    for {
        const delim = '\n'
        line, err := r.ReadString(delim)
        if err == nil || len(line) > 0 {
            if err != nil {
                line += string(delim)
            }
            lines = append(lines, line)
        }
        if err != nil {
            if err == os.EOF {
                break
            }
            return nil, err
        }
    }
    return lines, nil
}

func writeLines(file string, lines []string) (err os.Error) {
    f, err := os.Create(file)
    if err != nil {
        return err
    }
    defer f.Close()
    w := bufio.NewWriter(f)
    defer w.Flush()
    for _, line := range lines {
        _, err := w.WriteString(line)
        if err != nil {
            return err
        }
    }
    return nil
}

func main() {
    file := `lines.txt`
    lines, err := readLines(file)
    if err != nil {
        fmt.Println(err)
        os.Exit(1)
    }
    sort.Strings(lines)
    err = writeLines(file, lines)
    if err != nil {
        fmt.Println(err)
        os.Exit(1)
    }
}

eric chiang · Answer 2 · 2015-05-30T02:19:11.943

This is a pretty simple way of doing it.

import (
    "bytes"
    "io/ioutil"
    "sort"
)

// allow [][]byte to implement the sort.Interface interface
type lexicographically [][]byte

// bytes.Compare compares the byte slices lexicographically (alphabetically)
func (l lexicographically) Less(i, j int) bool { return bytes.Compare(l[i], l[j]) < 0 }
func (l lexicographically) Len() int           { return len(l) }
func (l lexicographically) Swap(i, j int)      { l[i], l[j] = l[j], l[i] }

func SortFile(name string) error {
    content, err := ioutil.ReadFile(name)
    if err != nil {
        return err
    }

    lines := bytes.Split(content, []byte{'\n'})
    sort.Sort(lexicographically(lines))

    content = bytes.Join(lines, []byte{'\n'})
    return ioutil.WriteFile(name, content, 0644)
}

You don't check `scanner.Err`. You don't need to re-open the file, you can just call `file.Truncate(0)` (ignoring that not using a second file is dangerous in that it will lose data if the program stops). And you don't check the error from `file.Close()` (can be important when writing a file). — Dave C, May 26 '15 at 17:31

score 1 · Answer 3 · answered Sep 15 '11 at 01:53

since you are about to sort the lines, you pretty much need to read the entire file. you can either slurp the file with io/ioutil.ReadAll or you can just write a small slurp function. once you have the lines of the file, sorting them can be done with a call to sort.Strings. i'll add a perhaps overly verbose version which hopefully illustrates how it can be done. i also recomment reading this excellent explanation on how go's sort package works: Go's sort package

package main

import (
    "os"
    "bufio"
    "fmt"
    "sort"
)

// slurp file into slice of lines/strings
func slurp(f string) (lines []string, e os.Error) {

    var fd *os.File
    var line string
    var bufRd *bufio.Reader
    var keepReading bool = true

    fd, e = os.Open(f)

    if e != nil {
        return nil, e
    }

    defer fd.Close()

    bufRd = bufio.NewReader(fd)

    for keepReading {
        line, e = bufRd.ReadString('\n')
        switch e {
        case nil:
            lines = append(lines, line)
        case os.EOF:
            lines = append(lines, line)
            keepReading = false
        default:
            return lines, e
        }
    }

    return lines, nil
}

// test stuff out..
func main() {

    if len(os.Args) > 1 {

        lines, e := slurp(os.Args[1])

        if e != nil {
            fmt.Fprintf(os.Stderr,"%s\n", e)
            os.Exit(1)
        }

        fmt.Println("\n----- unsorted -----\n")

        for _, line := range lines {
            fmt.Printf("%s", line)
        }

        fmt.Println("\n----- sorted -----\n")

        sort.Strings(lines)

        for _, line := range lines {
            fmt.Printf("%s", line)
        }
    }
}

note that the sort is in-place, so it does not return anything

score -3 · Answer 4 · answered May 26 '15 at 15:05

Just wondering how convenient is using Unix's sort for this purpose. I know it's not possible to have this code working in many deployment scenarios, but I see it worth it to mention as an option:

package main

import (
    "os"
    "os/exec"
)

func main() {
    file := "file.txt"

    command := []string{"sort", file, "-o", file}

    cmd := exec.Command(command[0], command[1:]...)
    cmd.Stdout = os.Stdout
    cmd.Stderr = os.Stderr

    if err := cmd.Run(); err != nil {
        panic(err)
    }
}

Thoughts?

Read in lines in a text file, sort, then overwrite file

4 Answers4