terraform backend s3 bucket creation returns 403 w/ Terraform 0.11.1

Question

How do I create an S3 bucket that has access to put a terraform.tfstate file? How do I get the tfstate into the bucket? What is the proper way to do this?

To preface, I have spent over 6 hours trying to figure this out. I saw the similar post with a problem caused by MFA. That's not my issue.

I'm using the same code to create EC2 instances, VPC and other resources just fine.

---[ REQUEST POST-SIGN ]-----------------------------
GET /?prefix=env%3A%2F HTTP/1.1
Host: tfstate-neonaluminum.s3.us-east-2.amazonaws.com
User-Agent: aws-sdk-go/1.10.36 (go1.9.2; darwin; amd64) APN/1.0 HashiCorp/1.0 Terraform/0.11.1
Authorization: AWS4-HMAC-SHA256 Credential=<CUT>/20171215/us-east-2/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=<CUT>
X-Amz-Content-Sha256: <CUT>
X-Amz-Date: 20171215T103755Z
Accept-Encoding: gzip

-----------------------------------------------------
2017/12/15 04:37:55 [DEBUG] [aws-sdk-go] DEBUG: Response s3/ListObjects Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 403 Forbidden
Connection: close
Transfer-Encoding: chunked
Content-Type: application/xml
Date: Fri, 15 Dec 2017 10:37:55 GMT
Server: AmazonS3
X-Amz-Bucket-Region: us-east-2
X-Amz-Id-2: UwWJlCAtabKny2Ncam+D7/s9Rozjw43ozsVNdtZgQqPSXoSvZbO3JnuPz3B9cmLcgYea6x8xPqQ=
X-Amz-Request-Id: 470C02B96A60090B

Here's the .tf code:

provider "aws" {
    access_key = "${var.aws_access_key}"
    secret_key = "${var.aws_secret_key}"
    region  = "${var.region}"
}

terraform {
   backend "s3" {
    bucket = "tfstate-neonaluminum"
       key = "terraform.tfstate"
    region = "us-east-2"
  }
}
resource "aws_s3_bucket" "terraform_state_bucket" {
  bucket = "tfstate-neonaluminum"
  acl = "private"
  tags {
    management = "terraform"
  }
}

Hope I didn't leave out any necessary details - new to asking questions on stackoverflow.

Does your provided key (`var.aws_access_key`) have write and list privileges to the s3 bucket `tfstate-neonaluminum`? — fishi0x01, Dec 15 '17 at 11:15
Is it actually possibly to use s3 as your backend with the bucket also being created by Terraform? I'm not sure either way but I would have expected the bucket is required to exist before Terraform can run, otherwise how will it pull in the state files? — timothyclifford, Dec 15 '17 at 11:40
Yes, I created the S3 bucket with terraform already. I assume I can use the same keys I created the bucket with to also write to the bucket? — Neal, Dec 17 '17 at 01:07

score 2 · Answer 1 · answered Dec 15 '17 at 13:59

2

Unfortunately that's the one caveat for using S3 as your backend to Terraform; you'll need to create the bucket manually. At times - a common pattern is that an organization will have multiple AWS accounts (For a variety of reasons - not just Terraform state) - this could be a possible option, to throw create your S3 bucket in that account, that way you can keep all of your other account's infrastructure defined by Terraform - but personally, I just create it by hand in my own account.

One thing you can do, is import the bucket after the fact with terraform import if you wanted to manage it with Terraform, just of course there's the chicken and the egg problem still. Or - use a local state initially while creating the bucket, and then define it later; however either way you would not be able to dynamically reference the bucket via the resource object.

But no - you cannot simultaneously create the bucket with Terraform while also setting it to your state.

answered Dec 15 '17 at 13:59

TJ Biddle

6,024
6
40
47

This makes sense. I actually have created a local state file and want to change the tf script to use a remote state file. – Neal Dec 17 '17 at 01:04
My S3 bucket is created remotely. Can I just drop the tf state file in S3 via the console? – Neal Dec 17 '17 at 01:06
@Neal Just change it from local to remote and run an apply - Terraform will do it for you – TJ Biddle Dec 17 '17 at 02:00
It's not really that difficult. You can have 1 terraform module w/ a local backend to create all of the state buckets for all of your accounts. Then the rest of the modules use the previously-created state bucket as normal. There is absolutely no need to do it manually – Brandon Mar 28 '22 at 20:18

terraform backend s3 bucket creation returns 403 w/ Terraform 0.11.1

1 Answers1

Linked