0

I have a glue job using psycopg2 that was previously working, and now is throwing this error: OperationalError: SCRAM authentication requires libpq version 10 or above.

Here is the psycopg2 version and libpq version:

2.9.5 (dt dec pq3 ext)
90224

This job was previously running on glue and started failing. It appears I need to upgrade my libpq version, but I have found nothing documenting how to do this in AWS Glue. This is not a python library, but rather a c library that psycopg2 depends on.

eclipse
  • 33
  • 5

2 Answers2

0

AWS Glue controls the version of libpq, you cannot upgrade this. Even when upgrading to psycopg2===2.9.6. The version is still locked to version 9.x. After opening a support ticket with AWS, they confirmed that Glue does not allow SCRAM auth via python code today.

Reason for the error: upgrading CockroachDB to version 22.2.x, these changes included default encryption change from bcrypt, to scram-sha-256. Additionally, existing passwords were migrated to scram. After contacting CockroachDB, the recommended solution was to downgrade from scram to bcrypt.

You need to set these cluster fields to revert back: server.user_login.downgrade_scram_stored_passwords_to_bcrypt.enabled server.user_login.password_encryption

eclipse
  • 33
  • 5
0

Add job parameter, will help to resolve the connection issue. AWS Glue will get binary from pip3.

--additional-python-modules : psycopg2-binary
Dharman
  • 30,962
  • 25
  • 85
  • 135
Bhushan_AWS ETL
  • 1