0

I am trying to create a connection to AWS Redshift from the Airflow Web UI, but so far it does not work. I always get the error: ('communication error', ConnectionRefusedError(111, 'Connection refused').

My Redshift is in a subnet of a VPC, but I have allowed public access. In its security group I have furthermore added: 0.0.0.0/0 IPV4 incoming access and my own IP as incoming access allowed.

As The input parameters to the airflow connection template, I use:

As the connection-id: An arbitrary name

host: redshift-clusterxxxxxxxxxxxxxxxxxxxxxxxxxx.redshift.amazonaws.com The other parameters are also correct.

I have build a little python using redshift_connector to test whether I can connect from there but I am always getting a timeout error.

Has anyone got further ideas on how to solve this issue?

luke33
  • 13
  • 3

1 Answers1

0

I have run into this same issue, and ended up switching to a different Airflow Operator which solved the issue for me.

I had been using from airflow.providers.amazon.aws.operators.redshift_sql import RedshiftSQLOperator and I switched to from airflow.providers.amazon.aws.operators.redshift_data import RedshiftDataOperator and this then magically worked.

I was using the amazon-provider package 6.0.0 and MWAA version 2.2.2 and 2.4.3.

I was determined to try and figure out why the other Operator did not work but ran out of time. I did find this link https://repost.aws/knowledge-center/cannot-connect-redshift-cluster and was going to setup the second option as I think this fits my situation but never got around to doing that.

Ricardo Sueiras
  • 126
  • 6