Jobs not connecting to MySQL service for self hosted GitHub Actions on ECS

Question

I have a self hosted GitHub Actions running in ECS and it has been working wonders, until I have to deal with service discovery.

The below is the workflow file

name: Checks before merging for core-service server

on:
  push:
    paths:
      - 'tons/of/paths/**'

jobs:
  build:
    runs-on: [self-hosted, linux, x64, nomad]

    #set a timeout just in case the tests hang
    timeout-minutes: 5

    services:
      mysql:
        image: mysql:5.7
        env:
          MYSQL_DATABASE: test-db
          MYSQL_USER: user
          MYSQL_PASSWORD: password
          MYSQL_ROOT_PASSWORD: rootpassword
        ports:
          - 3306
        options: --health-cmd="mysqladmin ping" --health-interval=10s --health-timeout=5s --health-retries=3

    steps:
    - uses: actions/checkout@v2
    - name: Use Node.js 14.16
      uses: actions/setup-node@v1
      with:
        node-version: 14.16
    - name: Install dependencies
      run: npm ci
      working-directory: some-dependency/location
    - name: Check if build is passing
      run: npm run build
      working-directory: more-dependency/more-places
    - name: Run tests
      run: npm run test
      env:
        DATABASE_URL: mysql://user:password@127.0.0.1:${{ job.services.mysql.ports[3306] }}/test-db
      working-directory: core-service/server

The below is the error I'm getting

Using existing mysql database from: DATABASE_URL
Error: connect ECONNREFUSED 127.0.0.1:32777

Any comments will be appreciated. Have a nice day~

Where is the DB relative to other assets in the stack (is it running in `RDS`)? If so, have the security groups been properly configured to allow traffic from all external components in the stack? Another thought: your config file shows all ports indicated as `3306` but your error is on port `32777`. Can you provide more details on that? `ECONNREFUSED` is often the result of network access architecture, or DB user and permission privileges missing or incorrect. — dusthaines, Jun 28 '21 at 19:21

Cameron Wilby · Accepted Answer · 2023-05-28T18:01:34.017

Service discovery within GitHub seems to have implementation details that have not made it to self-hosted runners.

I found that self hosted jobs should access the MySQL service via 172.17.0.1 (the host address for default networking in Docker..?), and use service discovery to fetch the port.

More information regarding accessing the gateway host in ECS can be found here: https://github.com/aws/containers-roadmap/issues/165

name: Build

on: [push]

jobs:
  test:
    runs-on: self-hosted

    services:
      mysql:
        image: mysql:latest
        env:
          MYSQL_DATABASE: acme_test
          MYSQL_ROOT_PASSWORD: secret
        ports:
          - 33306:3306
        options: >-
          --health-cmd="mysqladmin ping"
          --health-interval=10s
          --health-timeout=5s
          --health-retries=3

    steps:       
      - name: Verify MySQL connection
        run: |
          sudo apt update
          sudo apt install -y mysql-client
          mysql --version
          mysql \
            --host 172.17.0.1 \
            --port ${{ job.services.mysql.ports['3306'] }} \
            -uroot
            -psecret -e "SHOW DATABASES;"

Jobs not connecting to MySQL service for self hosted GitHub Actions on ECS

1 Answers1