"Too many open files" error with rethinkdb

Question

I used the golang driver provided by https://github.com/dancannon/gorethink. It is my understanding that i never close connection and reconnect. I can't close the connection because i don't know how to get the connection. What i know how is to get the session, and starting to think i don't know the right way to get the session.

So my question is:

How to fix the error?
How to close connection and reconnect?

EDIT:

@OneOfOne Not some, basically all of the related code:

// How i define session:
session, e := r.Connect(r.ConnectOpts{
    Address:  "localhost:28015",
    Database: "database",
    MaxActive: 0,
    MaxIdle: 0,
    // IdleTimeout: time.Minute,
})
//
// Inserting
inserts := map[string]interface{}{"something": something, "something1": something1, "something2": something2}
r.Db("database").Table("table").Insert(inserts).RunWrite(session)
//
// Updating
r.Db("database").Table("table").Filter(map[string]interface{}{"parameter": parameter}).Update(map[string]interface{}{"something" : somethingMore}).RunWrite(session)
//
// Get some row
row, e := r.Db("database").Table("table").Filter(map[string]interface{}{"parameter": parameter}).RunRow(session)
if e!= nil {
    //error
}

@neumino: This is what comes out when i hit ulimit -a

└─ $ ▶ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 30419
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 30419
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

I don't know whether this is too low or too high.

@VonC : That's one of the things that i don't get, I can't see how connection is inside the pool:

//http://godoc.org/github.com/dancannon/gorethink#Pool
type Pool struct {
    Session *Session

    // Maximum number of idle connections in the pool.
    MaxIdle int

    // Maximum number of connections allocated by the pool at a given time.
    // When zero, there is no limit on the number of connections in the pool.
    MaxActive int

    // Close connections after remaining idle for this duration. If the value
    // is zero, then idle connections are not closed. Applications should set
    // the timeout to a value less than the server's timeout.
    IdleTimeout time.Duration
    // contains filtered or unexported fields
}

Or did I just misunderstand what a pool is or what a connection is (or even what a session is)?

The sesion could be in a SessionStore (as in https://github.com/Lanciv/GoGradeAPI/blob/82fe7805225dcd5f1f1b784573c3a11ba6967094/store/session.go#L13 using https://github.com/Lanciv/GoGradeAPI/blob/master/store/defaultStore.go). The connections are in a Pool (http://godoc.org/github.com/dancannon/gorethink#Pool), but a SessionStore should retrieve the right session and close it (the rethink_test has just one to close: https://github.com/dancannon/gorethink/blob/916b03142cf131fae1584693a68f0d4b52feea18/gorethink_test.go#L54= — VonC, Jul 12 '14 at 19:51
Where are you: a) initialising the session pool? `init()`? `main()`, and then passing it around? Is it a global? A better understanding of your codebase would help here. — elithrar, Jul 13 '14 at 05:48
@elithrar Is session pool `session, e := r.Connect(r.ConnectOpts{})`? I initialize that in main(), then passing it to an inserting/updating/get function as one of the parameters. It's not a global variable. Basically like in [here](http://play.golang.org/p/o_I2FnSiQ4) — Joni Atif, Jul 13 '14 at 16:00
Yep, that looks okay to me. Do you suffer the issues when you are running under concurrency/stress testing? — elithrar, Jul 14 '14 at 02:08

score 3 · Answer 1 · answered Jul 12 '14 at 22:25

You can first look at your current limit, maybe it's excessively low? Run ulimit -a to see your settings

If you need to increase the number of open files, you can run ulimit -n <number of open files>.

I don't see any obvious problem in your defaultStore.go file. The cursor are closed, and the connection should be released. I gave a quick look at your code, and it doesn't seem that you create unnecessary sessions/pool, so I'm not sure where is the problem (or maybe I missed something?).

"Too many open files" error with rethinkdb

1 Answers1