0

I am very new to Google DataProc

We want to run set of code via spark-shell or sparkR for testing purposes. Is it possible to connect to spark cluster and execute the commands in spark-shell or sparkR in google DataProc?

I checked the doc and it seems we can submit jobs using spark-submit but I don't find information related to spark-shell or SparkR.

sag
  • 5,333
  • 8
  • 54
  • 91

1 Answers1

3

Yes.

spark-shell and sparkR are installed on all VMs in the dataproc cluster and you can SSH into any of them (I usually use the master node) and run spark-shell or sparkR.

Patrick Clay
  • 1,339
  • 7
  • 5