10

I am running a map-reduce job from mgo. It runs on a collection with a little more than 3.5M records. For some reasons right now I can not port this to aggregation; may be later. So, map-reduce is the thing I am looking forward to. This job, when I run it from the original js files I have created to test the code and output, runs fine. I tried to put the map and reduce code inside two strings and then tried to call the mgo.MapReduce to do the map-reduce for me where I am writing the output in a different collection. And it gives me

read tcp 127.0.0.1:27017: i/o timeout

Though, as the job has been fired in back-ground it is still running. Now according to this thread here --- http://grokbase.com/t/gg/mgo-users/1396d9wyk3/i-o-timeout-in-statistics-generation-upsert

It is easy to solve by calling the session.SetSocketTimeout but I do not want to do this as the total number of documents on which this map-reduce will run will vary and thus, I believe, the time. So, I will never be able to solve the problem by that way I believe.

What are the other ways that I might have?

Please help me

SRC
  • 2,123
  • 3
  • 31
  • 44
  • Not really an answer but you can always set the timeout to something really long and see if it works, for example: `session.SetSocketTimeout(1 * time.Hour)`, also out of curiosity, how long does it take when you run it from a js file? – OneOfOne Jul 09 '14 at 11:42
  • Around 10 mints. I thought of that also what you told me. but as you mentioned this is not a "proper" solution. let me see what i can come up with. – SRC Jul 09 '14 at 11:46
  • 1
    I really don't think there is a proper solution for this specific problem – OneOfOne Jul 09 '14 at 11:55
  • Same I think also!! :( – SRC Jul 09 '14 at 12:00

1 Answers1

18

Moving my comment to an answer.

I believe the only way to fix this is simply setting the socket timeout to something ridiculously high, for example:

session.SetSocketTimeout(1 * time.Hour)
OneOfOne
  • 95,033
  • 20
  • 184
  • 185