0

I have a large list-output and would like to run a separate task for each entry. For small outputs I can use below template. But If I increase output, e.g.

for run in {1..100000}; do

it crashes. How can I solve this problem? I tried to use artifacts, but for artifacts I cannot iterate elements of a list.

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: example-large-output-
spec:
  entrypoint: main
  templates:
    - name: main
      steps:
       - - name: create-large-output
           template: create-large-output-template
       - - name: iterate-large-output
           template: iterate-large-output-template
           arguments:
             parameters:
             - name: fp
               value: "{{item}}"
           withParam: "{{steps.create-large-output.outputs.result}}"

    - name: create-large-output-template
      script:
       image: debian:9.4
       command: [bash]
       source: |
          o='['
          for run in {1..100}; do
          o=$o'"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",'
          done
          o=${o::-1}"]"
          echo $o


    - name: iterate-large-output-template
      inputs:
        parameters:
        - name: fp
      script:
        image: alpine
        command:
          - sh
        source: |
          echo {{inputs.parameters.fp}}
olli
  • 59
  • 4
  • Something like this? https://stackoverflow.com/questions/62537126/how-can-i-work-around-argo-output-parameter-size-limit/62537127#62537127 – crenshaw-dev Feb 16 '23 at 19:39
  • @crenshaw-dev Yeah I found no other solution. – olli Feb 21 '23 at 13:17
  • It seems the only solution is [https://stackoverflow.com/questions/62537126/how-can-i-work-around-argo-output-parameter-size-limit/62537127#62537127](https://stackoverflow.com/questions/62537126/how-can-i-work-around-argo-output-parameter-size-limit/62537127#62537127) – olli Feb 21 '23 at 13:17

0 Answers0