Questions tagged [spark-jdbc]
78 questions
0
votes
0 answers
spark jdbc edit insert statement
Is it possible to inject parameter into spark JDBC insert statement?
I'm using
spark.sql("select * from my_table ").write.mode(SaveMode.Append).jdbc
for saving bulk dataframe to my DB.
In JdbcUtils the insertStatement is created.
Is it…
Arnon Rodman
- 511
- 2
- 6
- 21
0
votes
0 answers
jdbc read with partitionColumn - still see only 1 executor doing any work
I've been working on a pyspark based content pull / sync tool ( akin to sqoop but applying some transforms as a pipeline ). I'm creating a jdbc connection via:
datatype_boundry_mappings = {
#…
Bill Schwanitz
- 11
- 1
- 3
-1
votes
1 answer
how to use pyspark writing to JDBC without column name
My question is really really simple.
I'm use pyspark to export a hive table to SQL Server.
I found I exported column names as lines in the SQL Server.
I just want to do it without column names.
I don't want these columns in tables...
My pyspark…
SkyOne
- 188
- 3
- 15