0

I set up an private Airflow environment in AWS -v2.2.2-. Environment and plugins are up and running, I want to connect to Snowflake but I am getting the error below . -whl files in plugins.zip using requirements.txt-

snowflake.connector.vendored.urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='......snowflakecomputing.com', port=443): Max retries exceeded with url: /session/v1/login-request?request_id=....... (Caused by ConnectTimeoutError(<snowflake.connector.vendored.urllib3.connection.HTTPSConnection object at >, 'Connection to ........snowflakecomputing.com timed out. (connect timeout=60)'))

Same connection works in public mwaa. I am adding connection informations into admin-connections tab from the UI I know private env does not have connection to internet

I am aware i need to add some kind of outbound rule or endpoint but couldn't figure out . Checked the endpoints and couldn't see anything related to Snowflake. I will also be connecting to postgres, mysql DB's and few API's, which currently fails also

Is there a 1 click solution like adding some kind of outbound rule or should i be applying everything 1by1, and what would be that ?

If i want to connect to google-api something new, for snowflake something new etc ?

Also private mwaa environment is running on an existing vpc that has igw attachment, but the subnets that mwaa is running doesn't have any igw or nat attachment -as documentation suggests-

chook
  • 11
  • 1
  • Are you referring to Privatelink connections? That would require you to use a Snowflake account that has Privatelink enabled and configured which means account URL contains **privatelink** as well. – Sergiu Dec 05 '22 at 11:50
  • I think i do? Do i setup privatelink for every service that i am going to use ? Also not sure if it is possible but what if a servise is not supporting Privatelink ? – chook Dec 05 '22 at 12:30
  • AWS Privatelink is between VPCs not directly between services. Have a look [here](https://aws.amazon.com/privatelink/) to see how it works. – Sergiu Dec 05 '22 at 13:17
  • allright then privatelink is not useful for me, then how can i connect to API's and snowflake from my private network mwaa ? – chook Dec 05 '22 at 16:13
  • Private network MWAA probably means you need privatelink, that's how AWS communicates between VPCs without routing via the public internet. – Sergiu Dec 05 '22 at 17:25

0 Answers0