3

My application for this is to visualize the performance of my software. Therefore I briefly describe what I'm doing and where I'm stuck.

  1. I have my source code in GitLab
  2. Compile and run some tests in the CI for each commit
  3. Measure the time it took for the test run and save it to a file
  4. Upload the file with the time as an artifact

--------- From here on I don't know how to achieve it.

  1. Run some new job that reads all timing files of the previous artifacts
  2. Plot the times, probably with Python and save the image as SVG in the "main" repository
  3. Show the image on the GitLab start page (README.md should probably include it)
  4. Now I see which commits had which impact on my software's performance

No idea whether I'm asking for the impossible or not. I hope someone can help me as I'm not an CI expert. Maybe a single expression is already enough to google the solution but I don't even know how to formulate this.

Thanks everyone :)

Dominik Wernberger
  • 33
  • 1
  • 3

1 Answers1

0

Committing Images to Main

You can't just save an SVG image to the main repo from a pipeline job. You would need to make a commit. Not only would that pollute your git history and bulk up your repo, but it could also trigger a new pipeline, resulting in an endless loop.

There are ways around the endless loop, e.g. by controlling which sources/branches trigger pipelines or by prefixing the commit message with [skip ci], but it can get complicated and is probably not worth it. The truth is GitLab cannot do exactly what you want, so you will have to compromise somewhere.

Generate Metrics Graphs From Artifacts

You can collect metrics from past pipelines in a CSV file and save it as an artifact.

Add this to a reusable script called add_metrics.sh:

#!/bin/bash
HTTP_HEADER="PRIVATE-TOKEN: $YOUR_ACCESS_TOKEN"
URL_START="https://gitlab.example.com/api/v4/projects/$CI_PROJECT_ID/jobs/artifacts"
URL_END="raw/<path/to/artifact>/metrics.csv?job=$CI_JOB_NAME"

COLUMN_HEADERS=Date,Time,Branch,Commit SHA,Test Time(s),Code Coverage (%)

# download latest artifact
if curl --location --header $HTTP_HEADER $URL_START/$CI_COMMIT_BRANCH/$URL_END
then echo "Feature branch artifact downloaded."
elif curl --location --header $HTTP_HEADER $URL_START/master/$URL_END
then echo "Master branch artifact downloaded."
else echo $COLUMN_HEADERS >> metrics.csv
fi

# add data sample row to CSV
NOW_DATE=$(date +"%F")
NOW_TIME=$(date +"%T")
echo $NOW_DATE,$NOW_TIME,$CI_COMMIT_BRANCH,$CI_COMMIT_SHA,$TEST_TIME,$CODE_COVERAGE >> metrics.csv

# keep last 50 lines
echo "$(tail -50 metrics.csv)" > metrics.csv

Then call it from your pipeline in gitlab-ci.yml:

job_name:
  script:
    - TEST_TIME=10
    - CODE_COVERAGE=85
    - chmod +x add_metrics.sh
    - bash add_metrics.sh
  artifacts:
    paths:
      - metrics.csv
    expire_in: 1 month

Note: You will have to create a personal token and add it to a masked variable. I will also leave it up to you to populate the data metrics, like test time, code coverage, etc.

Explanation of Code

  1. Download the latest artifact for the current branch.
  2. The first commit of a feature branch will not find a "latest" artifact. If that happens, download the latest artifact from master.
  3. The first time the script runs, master won't even have a "latest" artifact, so create a new CSV file.
  4. APPEND the current sample to the end of the CSV file. Delete old samples to keep a fixed number of data points. You can add date, pipeline ID and other metrics.
  5. Store the updated artifact.
  6. To view the graph, download the artifact from the GitLab UI and view it in a spreadsheet app.

Publish to Pages

Using Python (pandas, matplotlib), you can generate an image of the plot and publish it to Gitlab Pages from your master branch pipeline. You can have a static HTML page in your repository referencing the same image filename, and keep replacing the same image from your pipeline. You can also add more useful metrics, such as code coverage.

DV82XL
  • 5,350
  • 5
  • 30
  • 59
  • 1
    I needed some time to verify your suggestion. Adding my current timing data to the previous ones and uploading the artifact works fine. As a next step I will try to publish my data via GitLab Pages, lets see how this works out. Thank you very much for your detailed answer. In the example, there is a `fi` missing at the end of the `if`-statement. – Dominik Wernberger Mar 27 '21 at 12:54
  • You're welcome and good catch! The filename in the 'tail' command was also wrong, so fixed that as well. Good luck with Pages. Let us know if you get stuck. – DV82XL Mar 28 '21 at 00:45
  • There’s a [skip ci] tag that can be added to a commit, to prevent the loop. – Fritz Oct 28 '21 at 13:29
  • @Fritz Good call out. Relying on commit message conventions may be prone to human error, e.g. if someone forgets the skip ci tag. If it's in a peer-reviewed script, maybe the risk is low. – DV82XL Oct 28 '21 at 14:17
  • @DV82XL I think the risk that the script messes up anything is pretty low. The only risks I see is changes in the CI env (e.g. changing the conventions). If a mistake happens despite that you could also stop the loop by canceling the task. However your solution is cleaner since it does not bloat the git repo and git history – Fritz Oct 28 '21 at 14:26