1

Basically, I have a Nodejs project which connects to a postgres database via knex and I use CircleCI for my tests. It works, but occasionally a build will fail due to “Error: connect ECONNREFUSED 127.0.0.1:5432”, and I can’t figure out why; it works without issue on my machine, there must be a difference in the builds.

The file that is run to create the test database is below:

'use strict';

require('dotenv').config();

const connectionOptions = require(`${process.cwd()}/knexfile.js`)[`${process.env.NODE_ENV}`];

const knex = require('knex')(connectionOptions);
console.log('New connection to default postgres database made');

// Remove all other connections to test database
knex.raw(`select pg_terminate_backend(pid) from pg_stat_activity where datname = '${process.env.PG_TEST_DATABASE}'`)
 .then(() => {
  console.log('Removed all other connections to the test database');
  // Drop test database if it exists
  return knex.raw(`DROP DATABASE IF EXISTS ${process.env.PG_TEST_DATABASE};`);
 })
 .then(() => {
  console.log('Dropped test database (if it existed)');
  // Create test database
  return knex.raw(`CREATE DATABASE ${process.env.PG_TEST_DATABASE};`);
 })
 .then(() => {
  console.log('Test database created');
  return process.exit();
 });

When it occurs, the error will happen on the first attempt to do anything with postgres after I connect, whether I try to drop a db, remove other connections, etc. Currently, it happens when I try to do the ‘select pg_terminate_backend’ line.

My settings for Postgres are:

const connectionOptions = {
 client: 'pg',
 version: '7.4.1',
 connection: {
  host: '127.0.0.1',
  user: process.env.PG_USER,
  password: process.env.PG_PASSWORD,
  database: process.env.PG_DATABASE,
  port: parseInt(process.env.PG_PORT) || 5432
 },
 pool: {
  min: 2,
  max: 10
 },
 migrations: {
  tableName: '_migrations',
  directory: './migrations',
 },
 seeds: {
  directory: './seeds/development'
 }
};

And my circleci yml file is below:

version: 2
jobs:
  build:
    working_directory: ~/project
    docker:
      - image: circleci/node:8.9.4
      # The below environemnt is where you set the .env variables used throughout node
        environment:
          NODE_ENV: test
          PG_USER: jerrodq2
          PG_DATABASE: freelancing_project
          PG_TEST_DATABASE: freelancing_project_test
          PG_PORT: 5432

      - image: postgres:10.3
        environment:
          POSTGRES_USER: jerrodq2
          POSTGRES_DB: freelancing_project

    steps:
      - checkout

      # Download and cache dependencies
      - restore_cache:
          keys:
          - v1-dependencies-{{ checksum "package.json" }}
          # fallback to using the latest cache if no exact match is found
          - v1-dependencies-

      - run:
          name: Install local dependencies
          command: npm install

      - run:
          name: Create database
          command: npm run db:reset:test



      - save_cache:
          paths:
            - node_modules
          key: v1-dependencies-{{ checksum "package.json" }}

      # run tests!
      - run:
          name: Running Tests
          command: npm run test

I have cross-posted this question to CircleCI Discourse.

halfer
  • 19,824
  • 17
  • 99
  • 186
Jerrod
  • 115
  • 12
  • 2
    Just a guess, but maybe PostgreSQL doesn't manage to start in time? Try waiting for the database daemon's port to become open: https://stackoverflow.com/a/27601038/200603 – Linas Valiukas Aug 16 '18 at 17:17
  • Thanks for the suggestion Linas, i've been trying different things, every time I think it's solved, it'll happen again. The code they use in your suggestion is a different language, i'll have to look into a javascript or circleci version, but where do you suggest I put it? in the yml file? create test db js file? – Jerrod Aug 21 '18 at 19:25

1 Answers1

1

For anyone is interested, I was a able to figure out a solution. Following Linas suggestion in the comments above, I found appropriate code for circleci that waits for postgres to start before moving onto the next step, I simply added a command to wait for the 5432 port to be open before creating the database like so: 

- run:
    name: Install local dependencies
    command: npm install

- run:
    name: Wait for Postgres to start
    command: dockerize -wait tcp://localhost:5432 -timeout 1m

- run:
    name: Create database
    command: npm run db:reset:test
Jerrod
  • 115
  • 12