I am trying to run alembic migration via the bitbucket pipeline.
Below is the code of the bitbucket pipeline.
steps:
- step: &health-check
name: Health Check
script:
- export HTTP_ADDR=$BITBUCKET_DOCKER_HOST_INTERNAL
- export HTTP_PORT=dummy_port_1
- export FLASK_ENV=development
- export GOOGLE_APPLICATION_CREDENTIALS=fury/secrets/credit-staging.json
- export FLASK_APP=fury
- export CONTAINER_ENV=localhost
- export TEMP_STORAGE_PATH=/var/tmp
- export SERVICE_NAME=fury-service
- export POSTGRES_USER=postgres
- export POSTGRES_PASSWORD=dummy
- export POSTGRES_DB=fdummy
- export POSTGRES_PORT=dummy_port
- export POSTGRES_URL=dummmy_url_1
- export GOOGLE_CLOUD_PROJECT=credit-staging
- export REDIS_HOST=localhost
- export REDIS_PORT=dummy_port_11
- pip install --upgrade pip==20.2.1
- pip install -r requirements.txt
- apt-get update
- apt-get install -y postgresql postgresql-client
- export DOCKER_FLAG=true
- chmod 777 http_request.sh
- exec gunicorn -b $HTTP_ADDR:$HTTP_PORT -t 3600 main:app &
- sleep 10
- ./http_request.sh
- python -c "from ci_script_libs import trigger_ci_scripts;trigger_ci_scripts.trigger(_instance_with_code_is_up=True)"
- step: &prod_migration
name: db migrations
image: google/cloud-sdk:latest
script:
- ./authorize_user.sh
- echo "Deployment:- ${BITBUCKET_DEPLOYMENT_ENVIRONMENT}"
- cp ./migration_dockerfile ./Dockerfile
- export IMAGE_NAME=gcr.io/$PROJECT/fury-service
- echo $GCLOUD_API_KEYFILE > ./gcloud-api-key.json
- gcloud auth activate-service-account $email --key-file=./gcloud-api-key.json
- gcloud config list
- gcloud auth configure-docker -q
- gcloud builds submit --tag $IMAGE_NAME --project $PROJECT
- gcloud run deploy $SERVICE_NAME --image $IMAGE_NAME --region asia-south1 --project $PROJECT --cpu $cpu --memory $memory --min-instances $MIN_INSTANCES
- gcloud run services update-traffic $SERVICE_NAME --to-latest --region asia-south1 --project $PROJECT
- echo "Bliss..!!!"
deploy_and_migrate_prod:
- variables:
- name: SERVICE_NAME
default: fury-service
allowed-values:
- fury-service
- fury-webhooks
- step:
<<: *health-check
name: Health Check
- step:
name: Migrations & Deploy
<<: *prod_migration
default: fury-service
deployment: staging
Below is the code of migrationdockerfile and migration_start.sh.
FROM python:3.7
WORKDIR /app
COPY . /app
RUN pip install --upgrade pip==20.2.1
RUN pip install -r requirements.txt
ENV FLASK_APP=main.py
CMD ./migration_start.sh
#!/usr/bin/env bash
set -e
DOCKER_FLAG="${DOCKER_FLAG:-true}"
POSTGRES_URL="${POSTGRES_URL:-127.0.0.1}"
flask db upgrade
exec gunicor
Code of revision file.
"""empty message
Revision ID: 6c0f9dcc2bb1
Revises: bdb59e00af3c
Create Date: 2023-06-27 18:06:25.182115
"""
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision = '6c0f9dcc2bb1'
down_revision = 'bdb59e00af3c'
branch_labels = None
depends_on = None
def upgrade():
# ### commands auto generated by Alembic - please adjust! ###
op.add_column('KYC_Test', sa.Column('test_column_prod', sa.VARCHAR(length=20), autoincrement=False, nullable=True))
# ### end Alembic commands ###
def downgrade():
# ### commands auto generated by Alembic - please adjust! ###
op.drop_column('KYC_Test', 'test_column_prod')
# ### end Alembic commands ###
Code of models.py file.
class KYC_Test(BASE):
# pylint: disable = invalid-name
__tablename__ = 'KYC_Test'
test_col1 = Column(String(20), primary_key=True)
dt_created = Column(DateTime, default=datetime.datetime.now)
dt_updated = Column(DateTime, default=datetime.datetime.now, onupdate=datetime.datetime.now)
test_column_prod = Column(String(20))
After running the bitbucket pipleine the migration is happing but I am getting below errors.
psycopg2.errors.DuplicateColumn: column "test_column_prod" of relation "KYC_Test" already exists
More details of the above error.
"Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1820, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute
cursor.execute(statement, parameters)
psycopg2.errors.DuplicateColumn: column "test_column_prod" of relation "KYC_Test" already exists"
Why the above error is coming this column doesn't existed before running the migration.