GitHub action dynamic secrets among multiple jobs

Question

We have a CD pipeline which retrieves secrets from our Vault. Currently our action has multiple calls to the vault action for each job in the workflow. I want to make it cleaner so it calls it once in a job and uses the secret among all the jobs.

Our current workflow looks like this:

  build:
    name: Build
    runs-on: ${{ inputs.runner_environment }}
    steps:
      - name: Retrieve secrets
        uses: /workflows/actions/vault@v1
        with:
          namespace: ${{ inputs.vault_namespace }}
          auth_method: ${{ inputs.vault_auth_method }}
          role_id: ${{ secrets.vault_role_id }}
          secret_id: ${{ secrets.vault_secret_id }}
          auth_role: ${{ inputs.vault_auth_role }}
          auth_path: ${{ inputs.vault_auth_path }}
          secrets: |
            ${{ inputs.artifactory_username_vault_secret_path }} ${{ inputs.artifactory_username_vault_secret_key }} | ARTIFACTORY_USERNAME;
            ${{ inputs.artifactory_password_vault_secret_path }} ${{ inputs.artifactory_password_vault_secret_key }} | ARTIFACTORY_PASSWORD;


      - name: Checkout
        uses: actions/checkout@v3

      - name: Verify
        uses: /workflows/actions/verify_quarkus@v1
        with:
          maven_settings_path: ${{ inputs.maven_settings_path }}
          artifactory_username: ${{ env.ARTIFACTORY_USERNAME }}
          artifactory_password: ${{ env.ARTIFACTORY_PASSWORD }}

  docker_image:
    name: Build & Push Docker image
    runs-on: ${{ inputs.runner_environment }}
    needs: build
    outputs:
      tag: ${{ steps.docker_image.outputs.tag }}
      image_full_path: ${{ steps.docker_image.outputs.image_full_path }}
    steps:
      - name: Retrieve secrets
        uses: /workflows/actions/vault@v1
        with:
          namespace: ${{ inputs.vault_namespace }}
          auth_method: ${{ inputs.vault_auth_method }}
          role_id: ${{ secrets.vault_role_id }}
          secret_id: ${{ secrets.vault_secret_id }}
          auth_role: ${{ inputs.vault_auth_role }}
          auth_path: ${{ inputs.vault_auth_path }}
          secrets: |
            ${{ inputs.artifactory_username_vault_secret_path }} ${{ inputs.artifactory_username_vault_secret_key }} | ARTIFACTORY_USERNAME;
            ${{ inputs.artifactory_password_vault_secret_path }} ${{ inputs.artifactory_password_vault_secret_key }} | ARTIFACTORY_PASSWORD;
      - name: Download artifacts
        uses: actions/download-artifact@v3
        with:
          name: artifacts
      - name: Build & Push Docker image
        uses: /workflows/actions/docker_image@v1
        id: docker_image
        with:
          image_name: ${{ inputs.docker_image_name }}
          artifactory_registry: ${{ inputs.artifactory_registry }}
          artifactory_repository: ${{ inputs.artifactory_repository }}
          artifactory_username: ${{ env.ARTIFACTORY_USERNAME }}
          artifactory_password: ${{ env.ARTIFACTORY_PASSWORD }}

As you can imagine there are way more jobs inside but I trimmed it. I want to have a separate job called retieve-vault and use it's outputs in every other job but so far I couldn't manage it. Last thing I tried is:

vault:
    name: Retrieve Vault Secrets
    runs-on: ubuntu-latest
    outputs:
      ARTIFACTORY_USERNAME: ${{ steps.set_outputs.outputs.ARTIFACTORY_USERNAME }}
      ARTIFACTORY_PASSWORD: ${{ steps.set_outputs.outputs.ARTIFACTORY_PASSWORD }}
    steps:
      - name: Retrieve Vault Secrets
        uses: /workflows/actions/vault@v1
        id: vault
        with:
          namespace: ${{ inputs.vault_namespace }}
          auth_method: ${{ inputs.vault_auth_method }}
          role_id: ${{ secrets.vault_role_id }}
          secret_id: ${{ secrets.vault_secret_id }}
          auth_role: ${{ inputs.vault_auth_role }}
          auth_path: ${{ inputs.vault_auth_path }}
          secrets: |
            ${{ inputs.artifactory_username_vault_secret_path }} ${{ inputs.artifactory_username_vault_secret_key }} | ARTIFACTORY_USERNAME;
            ${{ inputs.artifactory_password_vault_secret_path }} ${{ inputs.artifactory_password_vault_secret_key }} | ARTIFACTORY_PASSWORD;

      - name: Set outputs
        id: set_outputs
        run: |
          echo "ARTIFACTORY_USERNAME=${{ env.ARTIFACTORY_USERNAME }}" >> $GITHUB_OUTPUT;
          echo "ARTIFACTORY_PASSWORD=${{ env.ARTIFACTORY_PASSWORD }}" >> $GITHUB_OUTPUT;

And use it like ${{ needs.vault.outputs.ARTIFACTORY_USERNAME }} but it doesn't work as well.

Any ideas?