I want to read data from Google Storage Bucket using Google Session Token in Spark Application. Here instead of json.keyfile I want to use Google Session Key in spark conf option.
I tried with json.key file but Actually I am looking for Google Session Token approach. Is there any spark conf option way like below :
spark.conf.set("fs.gs.impl", "com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem") spark.conf.set("fs.AbstractFileSystem.gs.impl","com.google.cloud.hadoop.fs.gcs.GoogleHadoopFS") spark.conf.set("google.cloud.auth.service.account.enable","true") spark.conf.set("google.cloud.auth.service.account.json.keyfile","<key_file_path>")
Here instead of json.keyfile I want to use Google Session token
