Stack Overflow
Stack Overflow

Questions tagged [postgres-operator]

Use this tag for questions related to PGO (The Postgres Operator).

22 questions
2
votes
1 answer

CrunchyData Postgres operator Pod always in pending state

I am trying to set up Postgres cluster using CrunchyData Postgres operator. I am facing issue where the pod backrest-shared-repo is always in Pending state. NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS …
Prog_G
  • 1,539
  • 1
  • 8
  • 22
1
vote
1 answer

Not able to use Jinja template in PostgresOperator for filename of sql script

Have been searching for this issue, but all the answers are old and not relevant to my issue. I am trying to pass a sql parameter as a Jinja Template, but not able to. I am a bit new to Airflow, here is the code, I am not sure if the Jinja…
Kamesh Kotwani
  • 104
  • 1
  • 7
1
vote
1 answer

How to create dynamic task mapping for postgres operator in airflow?

I am completely new to AirFlow and I am trying to create 8 tasks which are pretty simillar. I've read about expand() methond though I am not quite sure how to use it for PostgresOperator? So I have this task: t1 =…
Vlad Vlad
  • 530
  • 1
  • 5
  • 17
1
vote
1 answer

Error while retrieving the connections string from the K8s service that is pointing to master pod

I am trying to retrieve the connections string from the K8s service that is pointing to the master pod in Minikube. I tried to execute this : export HOST_PORT=$(minikube service acid-minimal-cluster --url | sed 's,.*/,,') But I got the below error…
user19930511
  • 299
  • 2
  • 15
1
vote
0 answers

Use PGO (The Postgres Operator) to deploy backup replica for existing server

I have an existing (non-PGO) PostgreSQL server running in production (using AWS RDS). Now, I would like to deploy a replica server for backup/archiving/analytics purposes in a separate cloud (e.g. GCP) and I've found PGO (The Postgres Operator) to…
Slava Fomin II
  • 26,865
  • 29
  • 124
  • 202
0
votes
0 answers

Airflow/PostgresOperator Impossible to use xcom + params with Jinja template

I have a DAG with a loop on operator. Like this : with DAG( dag_id="injection_of_items_size", default_args=DEFAULT_ARGS, schedule_interval="@hourly", ) as dag: for item in ["CAR", "HOUSE"]: size_count = PythonOperator( …
Chjul
  • 399
  • 2
  • 3
  • 10
0
votes
1 answer

airflow postgresoperator transaction and commits

From the documentation of airflow: https://airflow.apache.org/docs/apache-airflow-providers-postgres/stable/_api/airflow/providers/postgres/operators/postgres/index.html Says this about autocommit: autocommit (bool) – if True, each command is…
moth
  • 1,833
  • 12
  • 29
0
votes
0 answers

Airflow. Pass the result of the previous Task Flow to the next Task Flow

I want logging in the POSTGRES-logging-tables. And to load data from the MSSQL-table to the POSTGRES-table. After inserting a row into the POSTGRES-logging-tables, I get the ID - log.t_date.i_dt_log_key I need to add log.t_date.i_dt_log_key as an…
yagrus2
  • 1
  • 2
0
votes
0 answers

crunchy postgres operator backup to azure blob fails

I want to backup the database into azure blob, but failed. My configuration is as follows apiVersion: postgres-operator.crunchydata.com/v1beta1 kind: PostgresCluster metadata: name: hippo-azure spec: image:…
wwh111
  • 1
0
votes
0 answers

Is it possible to show PostgreSQL terminal logs in Airflow Task log using just PostgresOperator?

I'm using the PostgresOperator to define tasks in Airflow. Today, i discovered that the PostgresSQL terminal logs such as 'raise notice' or information about number of rows affected didn't show up in the DAG's task log. It is possible to show them…
AdamS
  • 1
0
votes
1 answer

Airflow PostgresOperator SQL parameter syntax error

I have the following task in my DAG: create_features_table = PostgresOperator( task_id="create_features_table", postgres_conn_id="featuredb", sql="src/test.sql " ) But when I test the task, I get this error: psycopg2.errors.SyntaxError:…
Carl
  • 13
  • 2
0
votes
0 answers

Airflow not reading latest data from table

I have an airflow DAG. It has three tasks. All of them are using PostgreSQL operator. Here is a brief description of the three tasks. All tasks have autocommit = True Read events from main event tracking table, insert into event tracking temp…
mulder89520
  • 1
  • 4
0
votes
1 answer

k3sup install older kubectl version v1.24.3

I'm using crunchydata/postgres-operator for my k3s based setup however I have started getting error as below due to latest version on kubectl: time="2022-10-28T20:49:40Z" level=debug msg="debug flag set to true"…
user1988
  • 811
  • 2
  • 8
  • 17
0
votes
1 answer

how to debug and fix google gke, spilo/patroni pod label update that returns error code 4, Gateway Timeout

I'm using zalando's postgres operator and been having postgres cluster downtime now. I'm using connection pooler to connection to master and replica, but replica connection pooler not able to connect to replica pod due to replica svc does not have…
shadowlegend
  • 373
  • 2
  • 7
  • 21
0
votes
1 answer

Convert xcom_pull list in the "in" clause of SQL query

I am using Apache Airflow with a DAG with 2 tasks. Task 1 pulls a list of ids using a SELECT query, and sets the result using xcom_push. Task 2 needs to xcom_pull that list and convert it to a comma-separated string and use that in an IN clause of…
Bash
  • 4,140
  • 1
  • 13
  • 8
1
2