1

I am trying to run alembic migration via the bitbucket pipeline.

Below is the code of the bitbucket pipeline.

  steps:
    - step: &health-check
        name: Health Check
        script:

          - export HTTP_ADDR=$BITBUCKET_DOCKER_HOST_INTERNAL
          - export HTTP_PORT=dummy_port_1
          - export FLASK_ENV=development
          - export GOOGLE_APPLICATION_CREDENTIALS=fury/secrets/credit-staging.json
          - export FLASK_APP=fury
          - export CONTAINER_ENV=localhost
          - export TEMP_STORAGE_PATH=/var/tmp
          - export SERVICE_NAME=fury-service
          - export POSTGRES_USER=postgres
          - export POSTGRES_PASSWORD=dummy
          - export POSTGRES_DB=fdummy
          - export POSTGRES_PORT=dummy_port
          - export POSTGRES_URL=dummmy_url_1
          - export GOOGLE_CLOUD_PROJECT=credit-staging
          - export REDIS_HOST=localhost
          - export REDIS_PORT=dummy_port_11
          - pip install --upgrade pip==20.2.1
          - pip install -r requirements.txt
          - apt-get update
          - apt-get install -y postgresql postgresql-client
          - export DOCKER_FLAG=true
          - chmod 777 http_request.sh
          - exec gunicorn -b $HTTP_ADDR:$HTTP_PORT -t 3600 main:app &
          - sleep 10
          - ./http_request.sh
          - python -c "from ci_script_libs import trigger_ci_scripts;trigger_ci_scripts.trigger(_instance_with_code_is_up=True)"


    - step: &prod_migration
        name: db migrations
        image: google/cloud-sdk:latest
        script:
          - ./authorize_user.sh
          - echo "Deployment:- ${BITBUCKET_DEPLOYMENT_ENVIRONMENT}"
          - cp ./migration_dockerfile ./Dockerfile
          - export IMAGE_NAME=gcr.io/$PROJECT/fury-service
          - echo $GCLOUD_API_KEYFILE > ./gcloud-api-key.json
          - gcloud auth activate-service-account $email --key-file=./gcloud-api-key.json

          - gcloud config list
          - gcloud auth configure-docker -q

          - gcloud builds submit --tag $IMAGE_NAME  --project $PROJECT
          - gcloud run deploy $SERVICE_NAME --image $IMAGE_NAME --region asia-south1 --project $PROJECT --cpu $cpu --memory $memory --min-instances $MIN_INSTANCES
          - gcloud run services update-traffic $SERVICE_NAME --to-latest --region asia-south1 --project $PROJECT

          - echo "Bliss..!!!"

    deploy_and_migrate_prod:
      - variables:
          - name: SERVICE_NAME
            default: fury-service
            allowed-values:
              - fury-service
              - fury-webhooks
      - step:
          <<: *health-check
          name: Health Check

      - step:
          name: Migrations & Deploy
          <<: *prod_migration
          default: fury-service
          deployment: staging

Below is the code of migrationdockerfile and migration_start.sh.

FROM python:3.7
WORKDIR /app
COPY . /app
RUN pip install --upgrade pip==20.2.1
RUN pip install -r requirements.txt
ENV FLASK_APP=main.py
CMD ./migration_start.sh
#!/usr/bin/env bash

set -e
DOCKER_FLAG="${DOCKER_FLAG:-true}"
POSTGRES_URL="${POSTGRES_URL:-127.0.0.1}"
flask db upgrade
exec gunicor

Code of revision file.

"""empty message

Revision ID: 6c0f9dcc2bb1
Revises: bdb59e00af3c
Create Date: 2023-06-27 18:06:25.182115

"""
from alembic import op
import sqlalchemy as sa


# revision identifiers, used by Alembic.
revision = '6c0f9dcc2bb1'
down_revision = 'bdb59e00af3c'
branch_labels = None
depends_on = None


def upgrade():
    # ### commands auto generated by Alembic - please adjust! ###
    op.add_column('KYC_Test', sa.Column('test_column_prod', sa.VARCHAR(length=20), autoincrement=False, nullable=True))
    # ### end Alembic commands ###


def downgrade():
    # ### commands auto generated by Alembic - please adjust! ###
    op.drop_column('KYC_Test', 'test_column_prod')
    # ### end Alembic commands ###

Code of models.py file.



class KYC_Test(BASE):
    # pylint: disable = invalid-name
    __tablename__ = 'KYC_Test'

    test_col1 = Column(String(20), primary_key=True)

    dt_created = Column(DateTime, default=datetime.datetime.now)
    dt_updated = Column(DateTime, default=datetime.datetime.now, onupdate=datetime.datetime.now)
    test_column_prod = Column(String(20))

After running the bitbucket pipleine the migration is happing but I am getting below errors.

psycopg2.errors.DuplicateColumn: column "test_column_prod" of relation "KYC_Test" already exists

More details of the above error.

"Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1820, in _execute_context
    cursor, statement, parameters, context
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute
    cursor.execute(statement, parameters)
psycopg2.errors.DuplicateColumn: column "test_column_prod" of relation "KYC_Test" already exists"

Why the above error is coming this column doesn't existed before running the migration.

Paras jain
  • 432
  • 3
  • 15

0 Answers0