1

TL;DR How to handle errors in pyodbc (similar to how to catch specific pyodbc error message) pyodbc.ProgrammingError Unclosed quotation mark after the character string ''

I'm running a task instance using airflow and it's giving me an error. The thing is, when I check the data, there's not a quotation mark. And the error message is vague- it just says Unclosed quotation mark after the character string ''. Does anyone know the best way to go about debugging this error? What I'm looking for is a way to better understand when this is occurring.

I was thinking of surrounding my python code in try catch to better catch the error (and output the data), but to be honest I don't have much experience in doing this (in python). I tried it in my code, but I don't know if it works and It doesn't tell me enough to where I can find the problem data point(s).

[2021-02-02 12:54:37,116] {{taskinstance.py:1150}} ERROR - ('42000', "[42000] [FreeTDS][SQL Server]Unclosed quotation mark after the character string ''. (105) (SQLExecDirectW)")
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/airflow/models/taskinstance.py", line 984, in _run_raw_task
    result = task_copy.execute(context=context)
  File "/usr/local/airflow/mnt/dags/DAG/digital_chat/chat_bot/operators/postgres_to_ms_sql_operator.py", line 133, in execute
    self.__insert_records(rows, target_fields)
  File "/usr/local/airflow/mnt/dags/DAG/digital_chat/chat_bot/operators/postgres_to_ms_sql_operator.py", line 103, in __insert_records
    cursor.execute(self.__generate_insert_sql_statement(record_dictionary))
pyodbc.ProgrammingError: ('42000', "[42000] [FreeTDS][SQL Server]Unclosed quotation mark after the character string ''. (105) (SQLExecDirectW)")

code

    def __generate_insert_sql_statement(self, record_dictionary: dict) -> str:
        update_statement = 'set '
        insert_columns = '('
        insert_values = '('

        for record in record_dictionary.items():
            update_statement += f"{record[0]}={record[1]}, "
            insert_columns += f"{record[0]}, "
            insert_values += f"{record[1]}, "

        update_statement = update_statement[:-2]
        insert_columns = insert_columns[:-2] + ")"
        insert_values = insert_values[:-2] + ")"

        where_clause = self.__generate_where_clause(record_dictionary)

        insert_script = f"update {self.mssql_table} {update_statement} {where_clause} "
        insert_script += 'if @@ROWCOUNT = 0 '
        insert_script += f'insert into {self.mssql_table} {insert_columns} values {insert_values}'

        return insert_script
    def __insert_records(self, rows, target_fields):
        try:      
            mssql_connection = BaseHook.get_connection(self.mssql_conn_id)

            connection = pyodbc.connect(DRIVER='FreeTDS',host=mssql_connection.host,DATABASE=mssql_connection.schema,user=mssql_connection.login,password=mssql_connection.password,port=mssql_connection.port,driver='/usr/lib64/etc')
            cursor = connection.cursor()

            for i, row in enumerate(rows, 1):
                record = []

                for cell in row:
                    record.append(self._serialize_cell(cell))

                record_dictionary = self.__get_record_dictionary(record, target_fields)

                cursor.execute(self.__generate_insert_sql_statement(record_dictionary))

                connection.commit()

            cursor.close()
            connection.close()
        except pyodbc.ProgrammingError as programmingError:
            sqlstate = programmingError.args[0]
            if sqlstate = '42000':
                print(programmingError.args[0])
cluis92
  • 664
  • 12
  • 35
  • This is not really airflow. You only get the connection from airflow but then you use pyodbc. You don't use OdbcHook. Consider removing airflow label as it's not related. – Elad Kalif Feb 02 '21 at 14:21
  • 1
    You got SQL injection issues. If you sort that out by using SQLparameters, you wouldn't have syntax errors either. – Charlieface Feb 02 '21 at 14:50

0 Answers0