Golang abstract function that gets data from db and fills array

Question

I want to create an abstract function, that gets data from DB and fills array by this data. Types of array can be different. And I want to do it without reflect, due to performance issues. I just want to call everywhere some function like GetDBItems() and get array of data from DB with desired type. But all implementations that I create are owful.

Here is this function implementation:

type AbstractArrayGetter func(size int) []interface{}

func GetItems(arrayGetter AbstractArrayGetter) {
    res := DBResponse{}
    DB.Get(&res)
    arr := arrayGetter(len(res.Rows))
    for i := 0; i < len(res.Rows); i++ {
        json.Unmarshal(res.Rows[i].Value, &obj[i])
    }
}

Here I call this function:

var events []Event
GetFullItems("events", "events_list", map[string]interface{}{}, func(size int) []interface{} {
        events = make([]Event, size, size)
        proxyEnt := make([]interface{}, size, size)
        for i, _ := range events {
            proxyEnt[i] = &events[i]
        }
        return proxyEnt
    })

It works, but there are to much code to call this function, also there is some perfomance issue about copying events array to interfaces array.

How can I do it without reflect and do it with a short function call code? Or reflect not to slow in this case?

If you are having performance issues I'd suggest you try profiling. This code isn't too long - try and make a version that uses reflection and see how fast that is. It is unclear what kind of database you are getting your data from - does it follow the spec of sql.DB? Is the data layout known at compile time or are you dealing with dynamic data? — Michael Banzon, Sep 30 '17 at 18:06
I didn't compare the performance of this solution with reflect solution yet, but will. The main question is about right solution, seems it's not a right way: to copy array to get 1 level of abstraction. Also calling data getting from DB using 9 lines seems very bad. I'm using couchDB. Data layout is known at compile time, I just have a lot of different structs needed to load from DB, and I don't want to write a lot of code every time I need to get data from DB. — Vincento Law, Sep 30 '17 at 20:08

score 0 · Accepted Answer · answered Oct 05 '17 at 17:13

I tested performance with reflect, and it is similar to the mentioned above solution. So here is solution with reflect, if someone needs it. This function gets data from DB and fills abstract array

func GetItems(design string, viewName string, opts map[string]interface{}, arrayType interface{}) (interface{}, error) {
    res := couchResponse{}
    opts["limit"] = 100000
    bytes, err := CouchView(design, viewName, opts)
    if err != nil {
        return nil, err
    }
    err = json.Unmarshal(bytes, &res)
    if err != nil {
        return nil, err
    }
    dataType := reflect.TypeOf(arrayType)
    slice := reflect.MakeSlice(reflect.SliceOf(dataType), len(res.Rows), len(res.Rows))
    for i := 0; i < len(res.Rows); i++ {
        if opts["include_docs"] == true {
            err = json.Unmarshal(res.Rows[i].Doc, slice.Index(i).Addr().Interface())
        } else {
            err = json.Unmarshal(res.Rows[i].Value, slice.Index(i).Addr().Interface())
        }
        if err != nil {
            return nil, err
        }
    }
    x := reflect.New(slice.Type())
    x.Elem().Set(slice)
    return x.Interface(), nil
}

and getting data using this function:

var e Event
res, err := GetItems("data", "data_list", map[string]interface{}{}, e)

Golang abstract function that gets data from db and fills array

1 Answers1