I'm trying to connect Hadoop running on Google Cloud VM to Google Cloud Storage. I have:
- Modified the core-site.xml to include properties of fs.gs.impl and fs.AbstractFileSystem.gs.impl
- Downloaded and referenced the gcs-connector-latest-hadoop2.jar in a generated hadoop-env.sh
- authenticated via gcloud auth login using my personal account (instead of a service account).
I'm able to run gsutil -ls gs://mybucket/ without any issues but when I execute
hadoop fs -ls gs://mybucket/
I get the output:
14/09/30 23:29:31 INFO gcs.GoogleHadoopFileSystemBase: GHFS version: 1.2.9-hadoop2
ls: Error getting access token from metadata server at: http://metadata/computeMetadata/v1/instance/service-accounts/default/token
Wondering what steps I am missing to get Hadoop to be able to see the Google Storage?
Thanks!