Github actions on Vercel deployment with multiple environments

Question

We have Vercel preview deployments setup for every pull request in Github. On every pull request the code gets deployed to a Vercel test / acc / prod environment which are coupled to different backends.

For every pull request we want to run some (Cypress) tests against it, but only against the Vercel test environment.

We have this working by using the deployment_status event and specifying it should only run when the environment is test.

jobs:
  cypress:
    if: github.event.deployment_status.environment == 'Preview – Test'

This will result in a skipped Github run for acc / prod and a pass/fail for the test environment. However Github only lists the last run in the PR Github checks, this can be either the skipped or the pass/fail run and is depending on which preview environment gets deployed last.

Is there a way to enforce that Github only lists the relevant run? I tried making that run dynamic and making the test one mandatory but the check still get's overridden if test was not the last deployed Vercel environment.

I have the same question but for a Playwright run and a Lighthouse run. — Jacob Bowdoin, Jul 05 '23 at 01:02

VonC · Accepted Answer · 2023-07-06T04:57:17.203

Vercel do handle deployments automatically for each PR.
The challenge with GitHub Actions is that there is currently no native way to trigger an action based on a comment made on a PR, and this makes it a bit challenging to extract data from the Vercel bot comments.

However, a workaround could be to create a GitHub action that is triggered on a pull_request event and within this action, use the GitHub API to retrieve the PR comments, filter for Vercel bot comments and extract the required deployment URLs for the Cypress tests.

That could look like:

name: Cypress tests
on:
  pull_request:
    types:
      - opened
      - synchronize
      - reopened

jobs:
  cypress-run:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3

      - name: Get comments
        id: comments
        run: |
          COMMENTS=$(curl -s -H "Authorization: token ${{ secrets.GITHUB_TOKEN }}" \
          "https://api.github.com/repos/${{ github.repository }}/issues/${{ github.event.pull_request.number }}/comments")
          echo "::set-output name=comments::$COMMENTS"
        shell: bash

      - name: Extract Vercel preview URL
        id: vercel_url
        run: |
          URL=$(echo '${{ steps.comments.outputs.comments }}' | jq -r '.[] | select(.user.login == "vercel[bot]") | .body' | grep -oP 'https://.*\.vercel\.app')
          echo "::set-output name=url::$URL"
        shell: bash

      - name: Cypress run
        uses: cypress-io/github-action@v5
        with:
          browser: chrome
          env: "CYPRESS_BASE_URL=${{ steps.vercel_url.outputs.url }}"

That will:

checkout the code.
use the GitHub API to get all comments on the PR.
extract the Vercel preview URL from the comments.
run Cypress tests on the extracted URL.

(Do replace https://.*\.vercel\.app with your actual regular expression that matches the deployment URLs)

This is a very basic example:

You might need to handle more complex scenarios, like multiple deployments per PR, in which case you might need to loop through the URLs, run tests against each URL, and so on.
You also need to make sure the Vercel deployment is complete before running the tests, which might require additional checks or delays.

See also "Cypress.io and GitHub Actions: A step-by-step guide" from Filip Hric, jan. 2023.

Github actions on Vercel deployment with multiple environments

1 Answers1