0

I'm curious how should I structure JSON REST API server in Go language with Mgo library. I have dozens of collections related with each other. I've created the gist with sample part of file structure in my current approach.

It works great, but from time to time I encounter downtime caused by this error: "read tcp 10.168.30.100:37288: i/o timeout". I suppose that I handle mgo connection pool inapropriately. Are there any examples showing how should I create big applications based on mgo?

Rafał Sobota
  • 1,468
  • 1
  • 17
  • 32
  • Have you create any big applications in MongoDB in general? I mean may be the issue because you data model is not so good then the time spend for loading all data you need take to loong (just guest :D) – nvcnvn Nov 08 '13 at 12:17
  • In most of time it is super fast. After server restart this problem disappear. – Rafał Sobota Nov 08 '13 at 13:35
  • Are you trying to create new connection to mongodb every request (I mean cal mgo.Dial in the http handler). Maybe profiling your program will help you. – nvcnvn Nov 08 '13 at 13:45
  • I found some reply from the mgo author himself that could interest you: https://groups.google.com/d/msg/mgo-users/iZeiKgonGDU/2w4jf0V18TgJ https://groups.google.com/d/msg/mgo-users/oVJcXKPvNbU/L4AxMFDooD8J – Andrea Di Persio Nov 08 '13 at 13:50
  • I have a node.js application in production with 100.000+ users, but it also struggles with timeouts, so maybe solution for this could be more universal. This server in go most of time works on request in 20-200 ms. – Rafał Sobota Nov 08 '13 at 14:02
  • @nvcnvn I only create connection once on application start and use it in every http request handler. – Rafał Sobota Nov 08 '13 at 14:19

1 Answers1

5

This error message implies a roundtrip to the database took longer than the timeout period you defined. Just increasing that timeout should get rid of the problem, assuming you don't have any real issues that are causing the application to behave in a sluggish manner.

In general, this error doesn't imply you have any kind of scale issues, other than the fact maybe you have an increasing amount of data in some collections and certain queries may be getting too slow and need re-thinking (indexes, etc).

There's also no need to restart the application. You can either Refresh the problematic session, or Close and re-create the session in case you're using copies of a master session. The state of mgo and the pool of connections is still fine. It's just warning you that this specific session observed an issue on the wire, and so you have to acknowledge it before the session will be valid again.

As usual, also make sure to be using the latest release to avoid problems that have already been fixed, if any.

Gustavo Niemeyer
  • 22,007
  • 5
  • 57
  • 46
  • I create session once on application start and use it in every http handler, as in my gist - https://gist.github.com/rafalsobota/7369852#file-database-go-L13 . May it cause this problem? If so, can I simply refresh session after resolving http request or should I clone session for every http request and pass it to my model logic (it requires major refactoring of my code)? – Rafał Sobota Nov 08 '13 at 14:16
  • When this error occures in my app it repeats multiple times and after server restart everything backs to normal behavior. – Rafał Sobota Nov 08 '13 at 14:18
  • You're describing precisely the behavior I pointed out in the third paragraph. The error will not go away until you acknowledge it. One easy and good way to organize it is to Copy the master session for each handler, and defer Close it to make sure you don't have leaks. This way each new session will pick a good socket from the pool, or establish a new connection if necessary. – Gustavo Niemeyer Nov 08 '13 at 14:51
  • 1
    Thanks Gustavo. Is there any blog post how this connection pool works under the cover?. I don't think I understand everything and still have some questions. 1. Does this error mean that connection is lost and mgo doens't know about it yet and reuses it? 2. When I Close connection it doesn't back to the pool if it's broken? 3. Is it a good practise to Refresh session as soon as error occures and try again instead of return 5xx http status code? – Rafał Sobota Nov 08 '13 at 16:25
  • 1
    4. Can I use only one session in my entire application in multiple goroutines and only Refresh this global session when I detect this error? Why not? My existing queries would be killed? – Rafał Sobota Nov 08 '13 at 16:34
  • There are good recent threads in the mailing list on the topic. Please move there for a more open conversation. – Gustavo Niemeyer Nov 09 '13 at 00:35