File listing in Terraform

Question

I am trying to first generate in Terraform some files in a folder and then list those files. Unfortunately, I am not able to get the file list from the generation command since it is a shell script which does not outputs anything. As I understand Terraform tries to list files during the plan command. Is there a way to list files after they have been generated from the same terraform code?

resource "terraform_data" "files" {
  depends_on = [null_resource.generate_files]

  # Input argument is reflected to output
  input = [
    for f in fileset(local.root_path, "*.txt") : f
  ]
}


output "text_one" {
  value = terraform_data.files.output[0]
}

The output text_one value gives:

The given key does not identify an element in this collection value: the collection has no elements.

You could use the `local-exec` provisioner in `terraform_data` source to list all the files, but it's still unclear what yo want to achieve. Also, `terraform_data` is a replacement for `null_resource` and as such there is no need to use both. — Marko E, Jul 05 '23 at 12:43
I first generate some files, then I want to list them. I can also use only `terraform_data` as you said. Can you explain better the solution with `local-exec`? Do you mean to write the list of files in a file and then read that file? Or the list command from `local-exec` will be registered in some way so that I can reuse it? — sctx, Jul 05 '23 at 12:52
If it's Unix/Linux, you could just do `ls ${local.root_path}/*.txt` I guess, inside of the `local-exec` provisioner. The question is why do you need the file list? Will you reuse it in a different module or you are just trying to make sure the files are there? — Marko E, Jul 05 '23 at 12:57
Yes. Linux. I need to reuse the file list in the same module. I tried to separate in multiple modules, but it is not optimal in my case. — sctx, Jul 05 '23 at 12:59
I reuse in the sense that I need to open some of those files — sctx, Jul 05 '23 at 13:00
You need to access them in terraform or when terraform is done creating them? :) I'm asking so many questions because there might be better ways of doing that. — Marko E, Jul 05 '23 at 13:01
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/254380/discussion-between-sctx-and-marko-e). — sctx, Jul 05 '23 at 13:02

score 1 · Answer 1 · answered Jul 05 '23 at 18:32

The fileset function, like all of Terraform's built-in functions that interact with the filesystem, expects the files its interacting with to already be present on disk when you first run Terraform and to remain unmodified throughout the plan and apply process.

In any situation where you need to react to side-effects that occur during the apply process you will need to use either a managed resource (resource block) or a data resource (data block), and it must be the resource itself that is accessing the filesystem. In your example, although you've used resource "terraform_data" that resource type doesn't actually access the filesystem itself.

The hashicorp/local provider is the typical home for resource types that interact with the local system, including the local filesystem. However, at the time I'm writing this answer it only has resource types for interacting with individual files, and doesn't have any resource types with equivalent functionality to fileset. Therefore to solve this with Terraform you'll either need to find another provider that can perform this function or write your own provider that does so.

In the Terraform Registry I can see the community provider olidacombe/dirtree which has a data source dirtree_files that is documented to read files from disk. I've not used this provider myself and so I cannot vouch for it; if you intend to try it, you will need to study it yourself to make sure it doesn't include any unwanted or malicious behavior.

File listing in Terraform

1 Answers1