4

I am trying to compress the representation of a number into fewer bits. For example, right now I am using a float64 to represent a number such as 8.0. I only need 4 bits to represent 8.0, so I was trying to find a way to convert the float64 representation into a 4 bit representation. I know I can use an uint8 to represent 8 using only 8 bits, but that is not enough for my application. I need to squeeze those little extra bits in space.

I looked around the Go standard library, but did not find anything that would allow me to represent a number in less than 8 bits. Did I miss a package that can help me do this? How can I approach this problem?

Nik
  • 2,885
  • 2
  • 25
  • 25
MUAS
  • 519
  • 1
  • 7
  • 20
  • 3
    The smallest memory block you can address is a byte. You won't be able to use 4 bit integers without trickery which may make things slower. – Rafael Almeida Mar 28 '19 at 23:14
  • 5
    This sounds like an XY problem: you say you need to compress the number down to 4 bits. _Why_ do you need to do this? Are you trying to minimize storage? Is saving 4 bits (the smallest supported Go data type is a byte, which is most likely 8 bits) critical to your success? Does something you're working with want 4-bit values? There may be a better solution than using 4-bit numbers. – Joe McMahon Mar 28 '19 at 23:39
  • I am trying to do quantization for large ML models. Quantization is one of the strategies I am using to reduce the size of the model. This will hopefully reduce the communication cost of sending the model between peers in the system, while also being less taxing on the memory of machines storing the model. It doesn't need to be exactly 4 bits, but I want to test my system with this lower bit representation and measure its effect on performance and training error. Anything between 2 and 7 bits would be fine in my situation. – MUAS Mar 29 '19 at 06:22
  • This isn't about Go. Modern PC architectures will not allow you to address anything smaller than 1 byte. The only way to use 4-bit integers would be to store them in multiples in a larger datatype (e.g. store two in a byte, 4 int a uint16, etc). – Adrian Mar 29 '19 at 14:57

1 Answers1

3

The Go standard library does have an example of variable-length numbers.

package main

import (
    "encoding/binary"
    "fmt"
)

func main() {
    buf := make([]byte, binary.MaxVarintLen64)
    x := int64(8)
    n := binary.PutVarint(buf[:cap(buf)], x)
    buf = buf[:n]
    fmt.Println(x, len(buf), buf)
    y, m := binary.Varint(buf)
    fmt.Println(y, m)
}

Playground: https://play.golang.org/p/-p6M6OSHcMF

Output:

8 1 [16]
8 1
peterSO
  • 158,998
  • 31
  • 281
  • 276
  • 1
    But each byte is still 8 bits. I guess there is no way to use anything smaller than 8 bits in Go? – MUAS Mar 29 '19 at 06:29
  • 1
    Your question is not about Go, it is about [information theory](https://en.wikipedia.org/wiki/Information_theory). What algorithm do you want to program in Go? – peterSO Mar 29 '19 at 08:26
  • 1
    @MUAS if your concern is conserving storage space, you can write your own [binary marshaler/unmarshaler](https://golang.org/pkg/encoding/#BinaryMarshaler) for representing items != 8 bits - packing them in as tight as you like. Ultimately everything is stored in 8-bit bytes, but you can get very creative with bit usage. – colm.anseo Mar 29 '19 at 14:47