jsondecode fails when using for_each to pass variables to module

Question

I'm trying to use for_each with a terraform module creating datadog synthetic tests. The object names in an s3 bucket are listed and passed as the set for the for_each. The module reads the content of each file using the each.value passed in by the calling module as the key. I hardcoded the s3 object key value in the module during testing and it was working. When I attempt to call the module from main.tf, passing in the key name dynamically from the set it fails with the below error.

│ Error: Error in function call
│
│   on modules\Synthetics\trial2.tf line 7, in locals:
│    7:   servicedef  = jsondecode(data.aws_s3_object.tdjson.body)
│     ├────────────────
│     │ data.aws_s3_object.tdjson.body is ""
│
│ Call to function "jsondecode" failed: EOF.

main.tf

data "aws_s3_objects" "serviceList" {
  bucket = "bucketname"
}
module "API_test" {
  for_each = toset(data.aws_s3_objects.serviceList.keys)
  source  = "./modules/Synthetics"
  
  S3key   = each.value
  }

module

data "aws_s3_object" "tdjson" {
  bucket = "bucketname"
  key    = var.S3key
}

locals {
  servicedef  = jsondecode(data.aws_s3_object.tdjson.body)
  Keys        = [for k,v in local.servicedef.Endpoints: k]

}

Any clues as to what's wrong here?

Thanks

Answer 1

Check out the note on the aws_s3_object data source:

The content of an object (body field) is available only for objects which have a human-readable Content-Type (text/* and application/json). This is to prevent printing unsafe characters and potentially downloading large amount of data which would be thrown away in favour of metadata.

Since it's successfully getting the data source (not throwing an error), but the body is empty, this is very likely to be your issue. Make sure that your S3 object has the Content-Type metadata set to application/json. Here's a Stack Overflow question/answer on how to do that via the CLI; you can also do it via the AWS console, API, or Terraform (if you created the object via Terraform).

EDIT: I found the other issue. Check out the syntax for using for_each with toset:

resource "aws_iam_user" "the-accounts" {
  for_each = toset( ["Todd", "James", "Alice", "Dottie"] )
  name     = each.key
}

The important bit is that you should be using each.key instead of each.value.