DVC - apply a pipeline to multiple source files

Question

I have 100 source files to push through a pipeline. Is there a pattern for looping over a list, generating a new source and output file name each time for the same pipeline?

Thanks!

Answer 1

If you can elaborate more, that would be helpful, but maybe something like this is what you are looking for?

dvc.yaml

stages:
  example:
    foreach: ${some_param}
    do:
      cmd:
        - echo ${item}

params.yaml

some_param:
  - files/file1
  - files/file2
  - files/file3