connecting to sql server using connectorx and polars. everything works correctly locally and not getting any errors. however, when using azure pipelines to run code getting the following error "result = _read_sql(RuntimeError: Server requested a connection to an alternative address":
This doesn't make sense since the exact same details worked locally but does not work in pipeline?
internal files which error is caused:
File "C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\site-packages\polars\io.py", line 1095, in read_sql tbl = cx.read_sql( File "C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\site-packages\connectorx_init_.py", line 257, in read_sql
#code
import polars as pl
import connectorx as cx
from datetime import timedelta
load_dotenv()
username=os.environ["UID"]
password=os.environ["PWD"]
host=os.environ["SERVER"]
port=os.environ["PORT"]
db=os.environ["DATABASE"]
table_name=os.environ["DB_TABLE"]
col_name=os.environ["DB_COL"]
datetime_limit= str(datetime.today() - timedelta(days=50))[:-3]
conn=(f"mssql://{username}:{password}@{host}:{port}/{db}?encrypt=true")
query=f"SELECT * from {table_name} WHERE {col_name} > '{datetime_limit}'"
print(query)
start=timer()
data=pl.read_sql(query, conn, partition_on="Partition", partition_num=10)
end = timer()
yaml file
trigger:
none
schedules:
- cron: "0 13 * * *"
displayName: Daily build
branches:
include:
- dev
always: true
pool:
vmImage: "windows-latest"
#environment var
variables:
- group: env-group
steps:
# Set the version of Python which the build agent should use
- task: UsePythonVersion@0
displayName: Use Python 3.8
inputs:
versionSpec: '3.8'
addToPath: true
# Run Pip installer for any prerequisite libraries
- bash: pip3 install -r requirements.txt
displayName: Install pip package requirements
workingDirectory: project-requirements
# Invoke a Python script via Azure CLI in to use it's credentials
- task: AzureCLI@2
inputs:
azureSubscription: 'Subscription'
scriptType: 'batch'
scriptLocation: 'inlineScript'
inlineScript: 'python etl.py'
workingDirectory: wd/scripts