0

Have tried this combination without Cloudera but failed.

With Cloudera, I tried following the tutorial sparklyr: a test drive on YARN

Wonder if anyone has success without need to use

/etc/hadoop/conf.cloudera.hdfs
/etc/hadoop/conf.cloudera.yarn

And how?

himanshuIIITian
  • 5,985
  • 6
  • 50
  • 70
Charlie
  • 11
  • 2
  • What part of the tutorial is giving you errors? Simply these files are needed but do not exist, or some specific error message that occurs at a specific step? Please help people help you by editing your question to make it more clear what is happening. – Paul Sep 13 '17 at 04:00
  • @Paul These are directories specialized to cloudera. I installed hadoop by homebrew, set YARN_CONF_DIR, HADOOP_CONF_DIR to /path/to/homebrew/hadoop/conf in R, do spark_connect("yarn-client"). It didn't work. Hadoop and Yarn are OK. If a use spark come with sparklyr, there is a error like this one: https://community.hortonworks.com/questions/85757/how-to-add-the-hadoop-and-yarn-configuration-file.html. Beyond my capacity to investigate, just wonder if any out-of-box solution without cloudera. – Charlie Sep 13 '17 at 19:53

0 Answers0