0

I was trying to run an automl model on a linux server with 32 GB RAM using h2o in R on a data set having 5 million records and 70 features.

Here's a code that I was trying to run:

aml <- h2o.automl(y = y, x = x, training_frame = train, seed = 123)

I got this error:

|====                                         |   8%Error in .h2o.doSafeREST(h2oRestApiVersion = h2oRestApiVersion, urlSuffix = page,  :   Unexpected CURL error: Recv failure: Connection reset by peerError in .h2o.doSafeREST(h2oRestApiVersion = h2oRestApiVersion, urlSuffix = urlSuffix,  :   Unexpected CURL error: Failed connect to localhost:54321; Connection refused

And, then after restarting the R session and doing all the clean up when I tried to initialize h2o using:

h2o.init()

It gave me an error:

H2O is not running yet, starting it now...Error in system(paste(which, shQuote(names[i])), intern = TRUE, ignore.stderr = TRUE) :   cannot popen '/usr/bin/which 'java' 2>/dev/null', probable reason 'Cannot allocate memory'

Does anyone know how to fix it. I couldn't find any solutions online yet.

EDIT 1

By restarting the server; h2o got was getting initialized with h2o.init. And, by using keep_cross_validation_models = F in h2o.automl; model generation got completed to 100%.

But, I started getting this new error after h2o.automl progress bar reached 100%.

aml <- h2o.automl(y = y, x = x, training_frame = train, keep_cross_validation_models = F, seed = 123)

|======================================================| 100%ERROR: Unexpected HTTP Status code: 500 Server Error (url = http://localhost:54321/3/Jobs/$03017f00000132d4ffffffff$_8d3c6f4c4fd73944d1ce4624042d44a9) Error: parse error: premature EOF                                                            (right here) ------^

EDIT 2:

The problem got solved by rebooting the server and initializing h2o by adding max_mem_size = "12g" option.

sm925
  • 2,648
  • 1
  • 16
  • 28
  • 1
    How much RAM does your computer have? And also, what is the h2o model doing? Is it doing cross validation? Just training with known parameters? Something else? It's tough to help you debug without knowing anything about your system or the code you're running – Matt Kaye Apr 17 '21 at 03:24
  • 32 GB RAM Centos 7 linux server. I'm trying to run automl in h2o; added code in my question. – sm925 Apr 17 '21 at 15:34
  • 1
    Do you have any idea what h2o is doing in the background? My best guess is that it's doing cross validation and making a lot of copies of the data, which is overflowing your RAM. Is there a parameter in h2o to control that? I'm not familiar with h2o at all – Matt Kaye Apr 17 '21 at 16:00
  • Yes, by default it's 5 fold cross validation; when we don't specify that as a parameter with model – sm925 Apr 17 '21 at 16:11
  • Actually I came across this option in automl with which you can turn off keeping cross validation models by `keep_cross_validation_models = F`. But, before I can try that how do I initialize h2o again. – sm925 Apr 17 '21 at 17:39
  • That sounds promising. And re: initializing h2o, I'm not sure since I don't use it much. You said you're in a linux server? Can you just restart the server? Or maybe ssh into the server and restart the R session or something? – Matt Kaye Apr 17 '21 at 18:38
  • Tried restarting R; it didn't solve reinitializing issue. Restarting server might help. I'll try and see if that would help. – sm925 Apr 17 '21 at 19:48

0 Answers0