H2O error when calling as.factor on H2O data frame

Question

When I call the following reproducible doce:

install.packages("h2o", type = "source", repos =
           "http://h2o-release.s3.amazonaws.com/h2o/rel-ueno/8/R")
library(rsparkling)
library(h2o)
library(sparklyr)
library(dplyr)
library(DBI)
library(readr)
library(stringr)

conf <- spark_config()
conf$'sparklyr.shell.executor-memory' <- "460G"
conf$'sparklyr.shell.driver-memory' <- "460G"
conf$spark.executor.cores <- 32
conf$spark.executor.memory <- "460G"
conf$spark.executor.instances <- 8
conf$spark.dynamicAllocation.enabled <- "false"
conf$maximizeResourceAllocation <- "true"
conf$spark.default.parallelism <- 32

sc <- spark_connect(master = "local", config = conf, version = "2.0.2")
h2o_context(sc ,strict_version_check = T)
h2o.init(nthreads = -1)

NFLPlayers <- c('Von Miller','Christian McCaffrey','Joe Thomas')
salary <- c(21000, 23400, 26800)
Valuable = c(1,1,1)
Rdata <- data.frame(NFLPlayers, salary, Valuable)
system.time(Test <- copy_to(sc, Rdata, "Rdata"))
Test1 = as_h2o_frame(sc, Test, strict_version_check = FALSE)
Test1$Valuable<-as.factor(Test1$Valuable)

On an H2O dataframe I get the following error.

 ERROR: Unexpected HTTP Status code: 500 Server Error (url = http://127.0.0.1:54321/99/Rapids)

 Error: lexical error: invalid char in json text.
                                   <html> <head> <meta http-equiv=
                 (right here) ------^

This is brand new and it is not only happening in my computer but also on that of a colleague. I have restarted the cluster. Reinstalled sparklyr and Rsparkling. I have also changed versions and am constantly getting the same error. Any ideas?

Some of the logs:

06-19 22:46:24.991 127.0.0.1:54321       2158   #2868-244 INFO: POST /99/Rapids, parms: {ast=(tmp= RTMP_sid_8db5_1 (:= frame_rdd_58_b5fd354222706e76381948c109884d05 (as.factor (cols frame_rdd_58_b5fd354222706e76381948c109884d05 [8])) [8] [])), session_id=_sid_8db5}
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR: java.lang.NoClassDefFoundError: no/priv/garshol/duke/Comparator
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at water.rapids.Env.<clinit>(Env.java:278)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at water.rapids.Session.exec(Session.java:86)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at water.rapids.Rapids.exec(Rapids.java:93)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at water.api.RapidsHandler.exec(RapidsHandler.java:41)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at java.lang.reflect.Method.invoke(Method.java:498)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at water.api.Handler.handle(Handler.java:63)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at water.api.RequestServer.serve(RequestServer.java:448)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at water.api.RequestServer.doGeneric(RequestServer.java:297)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at water.api.RequestServer.doPost(RequestServer.java:223)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:503)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:429)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at water.JettyHTTPD$LoginHandler.handle(JettyHTTPD.java:183)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at org.eclipse.jetty.server.Server.handle(Server.java:370)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at java.lang.Thread.run(Thread.java:748)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR: Caused by: java.lang.ClassNotFoundException: no.priv.garshol.duke.Comparator
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
06-19 22:46:25.081 127.0.0.1:54321       2158   #2868-244 ERRR:     ... 37 more

Your post needs H2O version information and a reproducible example. — Erin LeDell, Jun 19 '17 at 22:59
Can you show the code that comes before? Ideally a fully reproducible example? Those weird 500 errors are normally something has got of sync (e.g. you delete some in the Flow interface, but you are still using the variable in your R session) and are often fixed by a restart. That you get it consistently suggests you have a logic bug in your script. (Or you are just hitting an H2O bug!) — Darren Cook, Jun 20 '17 at 09:49
Can you reproduce the issue in rsparkling on the latest sparkling water ? For spark 2.0 it's sparkling water 2.0.10 and h2o 3.10.5.1. Please see this readme how to set these versions https://github.com/h2oai/rsparkling. If that fails, could you please download the sparkling water manually and set as explained here https://github.com/h2oai/rsparkling#note-the-previous-command-requires-access-to-the-internet-1 ? Please let us know if that fails too. It's complaining about missing dependency which we might have forgotten to put somewhere. — Jakub Háva, Jun 21 '17 at 20:16
For me, the problem also arises when only trying to look at a column of data. For instance, with the example proposed above: `Test1[,1]`. Working with Ubuntu 14.04 using sparklyr 0.5.6, rsparkling 0.2.0 and h2o 3.10.5.2. — Sébastien Rochette, Jun 26 '17 at 11:31
For me, as suggested by @Jakub, the version of sparkling water downloaded from the web solved the problem. Follow the instructions of this page: https://github.com/h2oai/rsparkling#note-the-previous-command-requires-access-to-the-internet-1. Currently, you can download the last version here: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.0/10/index.html — Sébastien Rochette, Jun 27 '17 at 12:59
Thanks for bug report - It's work in progress - https://0xdata.atlassian.net/browse/SW-475, currently testing the solution — Jakub Háva, Jun 27 '17 at 13:29
This is fixed with the latest version of RSparkling 0.2.1 and Sparkling Water 2.1.10 and 2.0.11 — Jakub Háva, Jul 05 '17 at 12:31

score 0 · Answer 1 · answered Nov 09 '17 at 11:14

0

This was a bug and is fixed in latest Sparkling Water versions for Spark 2.0, Spark 2.1 and Spark 2.2

answered Nov 09 '17 at 11:14

Jakub Háva

228
1
5

H2O error when calling as.factor on H2O data frame

1 Answers1