3

I'm using GitLab pipelines to run e2e tests on various physical machines (these machines are connected to the test hardware in a 1 to 1 relation). On each machine, a GitLab runner is installed. The pipeline consists of three major parts:

  • prepare the test hardware (deploy, configure)
  • execute the e2e tests (on the test hardware)
  • clean up the test hardware

Currently I'm doing all of this in one job, by using the before_script, script and after_script keywords. But I would like to use multiple jobs (or even stages) for this.

The problem I'm facing is, that I can't be sure that all jobs/stages are executed on the same runner. So it might happen, that the prepare step is executed on runner1 and the execute step is executed on runner2 (even in parallel), which obviously is not what I want. The preparation is more than just creating artifacts, therefore I can't simply give it to the next job.

Tags also seems not to solve this, because a tag can only be specified for one job, not for multiple, or the complete stage.

I understand that this is not the way how runners are used normally, but I still wonder if there is a way to achieve this.

Or can someone point out another approach to solve this?

I'm using GitLab Community Edition 14.3.2.

Tobi
  • 61
  • 6
  • You could specify the same tag for every job in the stage, would that work for your purpose? I think also perhaps you're overthinking use of stages. The stages you describe I would think of as a single job -- as you are currently doing. Why do you want to use multiple stages? – sytech Nov 16 '21 at 17:06
  • Using tags was no option for me because I didn't want to hard code the tag in the pipeline. I wanted the pipeline to pick one available runner. But I didn't know that variables can be used in runner tags. I will give that a try. Maybe you are right and using on job for this is also okay, multiple stages just felt somehow cleaner to me. – Tobi Nov 17 '21 at 08:08

1 Answers1

1

I think you have two options here for how you can split this up -

  1. As sytech mentioned, you can tag each machine with machine-1, machine-2, etc, which will allow you to make your jobs sticky to each runner. Since you can use variables in runner tags, you could have a job at the start that checks which runner is not running tests, and sets RUNNER_TAG or something similar to that runner, so you don't have to hardcode your runner to a single box
  2. You could not have the test boxes run the jobs directly (presumably you're using a shell runner to do this today), and use SSH or winRM to access the box directly, and modify it from there. Then the state of your runner doesn't matter at all. This is likely the "cleaner" way to do it, so your test boxes don't have to share resources or state with the runner
Patrick
  • 2,885
  • 1
  • 14
  • 20