I am new in Pyspark. I am trying to connect to Amazon Aurora from Databricks using Pyspark. Can anyone please share a sample code to connect to Amazon Aurora from Databricks using Pysparks.
Asked
Active
Viewed 1,443 times
0
-
What type of database are you using on Aurora? – danimille Feb 14 '22 at 16:14
-
@danimille, Thanks for your reply. I am using Amazon Aurora Postgres engine. – SK ASIF ALI Feb 15 '22 at 05:55
1 Answers
0
After doing some analysis I got the answer of own question. It is same as connecting to other oracle database.
Here we need to install post-gres driver as a pre-requisite.
SparkSession.builder.appName("Aurora_Database_Connection").getOrCreate()
driver_aurora = "org.postgresql.Driver"
query="your query"
df = spark.read.format("jdbc").option("driver", driver).option("url", url).option("user", username).\
option("password", password).option("query", query).load()
SK ASIF ALI
- 85
- 8