Creating a substring in go creates a new kind of symbol

Question

I am comparing strings and there is the following:

Please note that the " in front of NEW are different.

Now when calling my function like this:

my_func(a[18:], b[18:])

The resulting strings are surprisingly:

What do I have to do to cut this weird symbol away and why is it behaving like this?

Hint: UTF-8. Strings are made up of bytes; *runes* are made up of one or more bytes from a string. Slicing a string slices off bytes and if you slice off *part* of a rune, that's what you see. — torek, Jun 03 '21 at 22:25

score 7 · Answer 1 · answered Jun 03 '21 at 22:25

Because that type of quote is a multibyte character, and you are splitting the string in the middle of a character. What you could do is convert to an []rune and then convert back:

https://play.golang.org/p/pw42sEwRTZd

s := "H界llo"
fmt.Println(s[1:3])                 // ��
fmt.Println(string([]rune(s)[1:3])) // 界l

score 3 · Accepted Answer · answered Jun 03 '21 at 22:56

Another option is the utf8string package:

package main
import "golang.org/x/exp/utf8string"

func main() {
   s := utf8string.NewString(` 'Not Available') “NEW CREDIT" FROM customers;`)
   t := s.Slice(18, s.RuneCount())
   println(t == `“NEW CREDIT" FROM customers;`)
}

https://pkg.go.dev/golang.org/x/exp/utf8string

Creating a substring in go creates a new kind of symbol

2 Answers2