Iterate over 2 strings in go

Question

I want to compare 2 strings rune-by-rune to see which one comes first in an arbitrary alphabetical order.

Right now I have this implementation which stores in map[rune]int a mapping representing the order of letters in my alphabet.

I have this working code. I'm well aware of the flaws in the current design, but this isn't the point of the question.

package main

import (
    "bufio"
    "log"
    "math/rand"
    "os"
    "sort"
)

type Dictionnary struct {
    content           []string
    alphaBeticalOrder map[rune]int
}

func minSize(w1, w2 []rune) int {
    if len(w1) < len(w2) {
        return len(w1)
    }
    return len(w2)
}

func (d *Dictionnary) comesFirst(a, b rune) int {

    return d.alphaBeticalOrder[a] - d.alphaBeticalOrder[b]
}

func (d Dictionnary) Less(i, j int) bool {
    wordi, wordj := []rune(d.content[i]), []rune(d.content[j])
    size := minSize(wordi, wordj)
    for index := 0; index < size; index++ {
        diff := d.comesFirst(wordi[index], wordj[index])
        switch {
        case diff < 0:
            return true
        case diff > 0:
            return false
        default:
            continue
        }
    }
    return len(wordi) < len(wordj)
}

func (d Dictionnary) Swap(i, j int) {
    d.content[i], d.content[j] = d.content[j], d.content[i]
}

func (d Dictionnary) Len() int {
    return len(d.content)
}

func main() {

    letters := []rune{'z', 'y', 'x', 'w', 'v', 'u', 't', 's', 'r', 'q', 'p', 'o', 'n', 'm', 'l', 'k', 'j', 'i', 'h', 'g', 'f', 'e', 'd', 'c', 'b', 'a'}
    aOrder := make(map[rune]int)
    perm := rand.Perm(len(letters))
    for i, v := range perm {
        aOrder[letters[i]] = v
    }

    file, err := os.Open("testdata/corpus.txt")
    if err != nil {
        log.Fatal(err)
    }

    corpus := make([]string, 0, 1000)
    scanner := bufio.NewScanner(file)
    for scanner.Scan() {
        corpus = append(corpus, scanner.Text())
    }

    if err := scanner.Err(); err != nil {
        log.Fatal(err)
    }
    file.Close()

    input := Dictionnary{content: corpus, alphaBeticalOrder: aOrder}

    sort.Sort(input)

    ofile, err := os.Create("testdata/sorted.txt")
    writer := bufio.NewWriter(ofile)
    for _, v := range input.content {
        writer.WriteString(v)
        writer.WriteString("\n")
    }
    writer.Flush()
    defer ofile.Close()
}

My question concerns the Less(i,j int) bool function. Is there a more idiomatic way to iterate over 2 strings to compare them rune by rune ? I am making a copy of data here which could probably be avoided.

EDIT: To clarify my problem is that range(string) can allow you to iterate over strings rune by rune, but I cannot see a way to iterate over 2 strings side-by-side. Only way I see it to convert the strings to []rune.

why not use the string comparison functions in the `sort` package? http://golang.org/pkg/sort/#Strings — Not_a_Golfer, Oct 15 '14 at 15:34
@Not_a_Golfer because he has a custom alphabet. `sort.Strings` will use `sort.StringSlice`'s implementation, sorting by the "normal" alphabet. — thwd, Oct 15 '14 at 15:36
@tomwilde doesn't look that custom to me `[]rune{'z', 'y', 'x', 'w', 'v', 'u', 't', 's', 'r', 'q', 'p', 'o', 'n', 'm', 'l', 'k', 'j', 'i', 'h', 'g', 'f', 'e', 'd', 'c', 'b', 'a'}` — Not_a_Golfer, Oct 15 '14 at 15:42
Maybe OP should clarify; I read "arbitrary alphabetical order" as any possible order. — thwd, Oct 15 '14 at 15:43
Well the alphabetical order in specified via a map[rune]int. So the alphabet can consist of any Runes sorted in any order. I provided an example for the sake of having a working code sample. — Félix Cantournet, Oct 15 '14 at 15:48
@FélixCantournet now I'm intrigued - why would you want to sort in a non standard way? — Not_a_Golfer, Oct 15 '14 at 15:48
@Not_a_Golfer Initially my goal was to create a big enough dataset to test an inverse algorithm (which guesses an alphabetical order from a sorted list of words) I got a little carried away. — Félix Cantournet, Oct 15 '14 at 15:54
To be fair there are possible applications to non-standard languages. Btw, I suppose the built-in `sort.Strings` only works on roman characters ? in which case there must be some package that handles locale that I could leverage here. — Félix Cantournet, Oct 15 '14 at 16:02
@FélixCantournet there is a unicode collation library for go (from the go team) that is not part of the standard library, which should deal with this. it has comparison functions - https://godoc.org/code.google.com/p/go.text/collate — Not_a_Golfer, Oct 15 '14 at 18:31
@Not_a_Golfer thanks. it looks pretty much like what I need. =) — Félix Cantournet, Oct 16 '14 at 12:34

score 2 · Answer 1 · answered Oct 15 '14 at 15:43

2

You could make the loop slightly more idiomatic using a range on one of the two words. This necessitates adding a check within the loop but you no longer have to perform the check at the final return.

// determines if the word indexed at i is less than the word indexed at j.
func (d Dictionnary) Less(i, j int) bool {
    wordi, wordj := []rune(d.content[i]), []rune(d.content[j])
    for i, c := range wordi {
        if i == len(wordj) {
            return false
        }

        diff := d.comesFirst(c, wordj[i])
        switch {
        case diff < 0:
            return true
        case diff > 0:
            return false
        default:
            continue
        }
    }
    return false
}

answered Oct 15 '14 at 15:43

Nate Brennand

1,488
1
12
20

Yep that looks a little better, but I think it performs worse. – Félix Cantournet Oct 15 '14 at 16:03
I'm going to select the 2nd answer which is closer to what I was looking for. – Félix Cantournet Oct 16 '14 at 11:41
Felix, did you measure the performance, or is that just a hunch? You might be right, but I can't tell just by inspecting the source code. – Rick-777 Oct 16 '14 at 12:32

peterSO · Accepted Answer · 2014-10-17T00:59:03.297

To iterate over two strings side-by-side in the Less method:

package main

import (
    "bufio"
    "log"
    "math/rand"
    "os"
    "sort"
    "unicode/utf8"
)

type Dictionary struct {
    content           []string
    alphaBeticalOrder map[rune]int
}

func (d Dictionary) Len() int {
    return len(d.content)
}

func (d Dictionary) Swap(i, j int) {
    d.content[i], d.content[j] = d.content[j], d.content[i]
}

func (d Dictionary) Less(i, j int) bool {
    wi, wj := d.content[i], d.content[j]
    jj := 0
    for _, ri := range wi {
        rj, size := utf8.DecodeRuneInString(wj[jj:])
        if rj == utf8.RuneError && size == 0 {
            return false
        }
        switch ao := d.alphaBeticalOrder[ri] - d.alphaBeticalOrder[rj]; {
        case ao < 0:
            return true
        case ao > 0:
            return false
        }
        jj += size
    }
    return len(wi) < len(wj)
}

func main() {

    letters := []rune{'z', 'y', 'x', 'w', 'v', 'u', 't', 's', 'r', 'q', 'p', 'o', 'n', 'm', 'l', 'k', 'j', 'i', 'h', 'g', 'f', 'e', 'd', 'c', 'b', 'a'}
    aOrder := make(map[rune]int)
    perm := rand.Perm(len(letters))
    for i, v := range perm {
        aOrder[letters[i]] = v
    }

    file, err := os.Open("testdata/corpus.txt")
    if err != nil {
        log.Fatal(err)
    }

    corpus := make([]string, 0, 1000)
    scanner := bufio.NewScanner(file)
    for scanner.Scan() {
        corpus = append(corpus, scanner.Text())
    }

    if err := scanner.Err(); err != nil {
        log.Fatal(err)
    }
    file.Close()

    input := Dictionary{content: corpus, alphaBeticalOrder: aOrder}

    sort.Sort(input)

    ofile, err := os.Create("testdata/sorted.txt")
    writer := bufio.NewWriter(ofile)
    for _, v := range input.content {
        writer.WriteString(v)
        writer.WriteString("\n")
    }
    writer.Flush()
    defer ofile.Close()
}

Nice ! that is exactly what I was trying to achieve. Thank you. I'll benchmark it against the otyher implementation. — Félix Cantournet, Oct 16 '14 at 11:41
For those interested, this performs much better. both in terms of speed and of course memory allocation — Félix Cantournet, Oct 16 '14 at 12:32

Iterate over 2 strings in go

2 Answers2