Stack Overflow
Stack Overflow

Questions tagged [airflow-connections]

10 questions
2
votes
1 answer

Airflow dag cannot find connection-id

I am managing a Google Cloud Composer environment which runs Airflow for a data engineering team. I have recently been asked to troubleshoot one of the dags they run which is failing with this error : [12:41:18,119] {credentials_utils.py:23} WARNING…
Notauser
  • 406
  • 2
  • 10
1
vote
0 answers

Airflow snowflake connection not appearing in UI

I have set up airflow 2.5.1 on docker. pip freeze in scheduler, worker and…
Kelvin Chow
  • 13
  • 4
1
vote
1 answer

KubernetesPodOperator is not able to access the Airflow Connections stored in Google Secrets Manager

I am using the Composer version 2.0.0 Airflow 2.1.4 and I have created a KubernetesPodOperator that is trying to access the Airflow connection stored in the Google Secrets Manager. But it isn't able to locate the credentials (check exceptions…
tank
  • 465
  • 8
  • 22
1
vote
1 answer

How Do I properly Encode `extra` parameters while using `airflow connections add`

Problem Statement When editing the UI I can add modify the extra field to contain {"no_host_key_check": true} But when I attempt to add this connection from the CLI with this command which follows the connections documentation format airflow…
AlexLordThorsen
  • 8,057
  • 5
  • 48
  • 103
0
votes
0 answers

Airflow providers installation for connections

I have set up a airflow 2.5.1, using docker compose in an EC2. I have already migrated all connections from my previous airflow 1.10, and installed most of the packages. But i have some problems with some connections. I have installed each in…
Kelvin Chow
  • 13
  • 4
0
votes
0 answers

Airflow cant connect to google credentials

I am running on VM SSH server on Google cloud. i have a problem airflow is failing to connect to GCP, I followed an answers which someone did do and it helps for him but for me the process is not helping me, here is the error and this are what i…
Efe
  • 1
  • 1
0
votes
0 answers

how to use run impala command from Airflow

I have a task to run a query on impala, from airflow .Not sure how to do that ? I tried Impala query using Airflow but no luck .
rahul ranjan
  • 1
  • 1
0
votes
0 answers

Airflow - How to encrypt connection that is built through Extra

I've a connection to AWS S3 on Airflow that is made with Extra config: aws_access_key_id aws_secret_access_key However, since this credentials are stored on Extra field, it's not encrypted and everyone that has access to Airflow UI can see those…
Pedro Alves
  • 1,004
  • 1
  • 21
  • 47
0
votes
1 answer

Connection used in Airflow DAG is not providing _decrypted_ password or extra - DAG authoring issue

I'm extending the Airflow:2.2.0 image and trying to use a Connection in a DAG to make a GET requests with a custom Hook. However, regardless what I try, and following any suggestions found, the DAG does not seem to get a decrypted version of the…
davidverweij
  • 328
  • 1
  • 15
-4
votes
0 answers

Unable to connect to Airflow with ODBC

I am struggling to connect to SQL using ODBC even to connect directly to SQL without using ODBC. I have tried connecting directly to SQL and connecting using ODBC. Look at the picture below I tried to go to Admin on airflow to connect SQL and…
Mighty Hlungwane
  • 1
  • 1