0

I have created a PREMIUM trail Databricks account with AWS. I have setup AWS account with user access keys.

And for configuring AWS storage followed the below instructions in the URL(setup bucket policy as below in below URL).

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Grant Databricks Access",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::98765432101:root"
            },
            "Action": [
                "s3:GetObject",
                "s3:GetObjectVersion",
                "s3:PutObject",
                "s3:DeleteObject",
                "s3:ListBucket",
                "s3:GetBucketLocation"
            ],
            "Resource": [
                "arn:aws:s3:::my-databricks-user-bucket/*",
                "arn:aws:s3:::my-databricks-user-bucket"
            ]
        }
    ]
}

https://docs.databricks.com/administration-guide/account-settings/aws-storage.html

But, I am getting the error as below.

The provided S3 bucket is valid, but have insufficient permissions to launch a Databricks deployment. Please double check your settings according to the tutorial. Missing permissions: PUT, LIST, DELETE

In the above bucket policy which I used, PUT, LIST, DELETE policies are there. Still facing the above error.

Note: As trail and error, changed the Action as below which allows all actions. But, still getting the same error.

"Action": "*"
Praveen L
  • 937
  • 6
  • 13
  • Are you sure its about s3 policy, not [cross-account role](https://docs.databricks.com/administration-guide/account-settings/aws-accounts.html) that DB requires? – Marcin Feb 17 '21 at 11:22
  • I have completed `AWS account` setup in Databricks using `access keys` not `cross-account-role` successfully. But, after that when I am setting up AWS storage, I faced the mentioned issue. – Praveen L Feb 17 '21 at 13:53

2 Answers2

1

The above error is caused because of the mistake I did when I am setting up Databricks account with AWS.

As part of setting up AWS account details in Databricks, a cross-account-role should be created (alternative is through access key). When creating the role, AWS account id should be given(Databricks AWS account id). The value of that is 414351767826.

The mistake I did was I gave my AWS account ID instead of Databricks one. Following as it is in the below URL will work as expected.

The same issue I did when I am setting AWS storage. Following the documentation as it is will work perfectly.

https://docs.databricks.com/administration-guide/account-settings/aws-accounts.html

Praveen L
  • 937
  • 6
  • 13
0

I had the same problem and I did solve it with the following :

  1. Make sure your bucket name is globally unique : my-databricks-user-bucket !!
  2. Make sure to use exactly the same name in your databricks storage, EXACTLY THE SAME!!
  3. Make sure the region of your bucket and your databricks workspace's region are
    the same

I am sure this will solve your issue...

Alex Ott
  • 80,552
  • 8
  • 87
  • 132
Soheyb KHELIFA
  • 1