How to use sparklyr spark_write_jdbc to connect to MySql

Question

spark_write_jdbc(members_df,
    name = "Mbrs",
    options = list(
      url = paste0("jdbc:mysql://",mysql_host,":",mysql_port,"/",dbname),
      user = mysql_user,
      password = mysql_password),
    mode = "append")

Results in the following exception:

Error: java.sql.SQLException: No suitable driver
  at java.sql.DriverManager.getDriver(DriverManager.java:315)

The .jar file is in a folder on the server where RStudio is running, config details below. We're able to access MySql via the RMySql package so MySql is working and accessible.

config$`spark.sparklyr.shell.driver-class-path` <- "/dev/shm/temp/mysql-connector-java-5.1.44-bin.jar"

score 1 · Answer 1 · answered Dec 19 '18 at 14:09

Even if the question is different, I think the answer still applies also to this:

How to use a predicate while reading from JDBC connection?

Code to connect to JDBC MySQL through sparklyr (I made some slight changes to simplify the code a bit, written by Jake Russ)

library(sparklyr)
library(dplyr)

config <- spark_config()
#config$`sparklyr.shell.driver-class-path` <- "E:\\spark232_hadoop27\\jars\\mysql-connector-java-5.1.47-bin.jar"
#in my case, using RStudio and Sparkly this seemed to be optional

sc <- spark_connect(master = "local")

db_tbl <- spark_read_jdbc(sc,
                  name    = "table_name",  
                  options = list(url      = "jdbc:mysql://localhost:3306/schema_name",
                                 user     = "root",
                                 password = "password",
                                 dbtable  = "table_name"))

How to use sparklyr spark_write_jdbc to connect to MySql

1 Answers1