Questions tagged [google-cloud-composer-2]
5 questions
4
votes
1 answer
Cloud Composer 2: prevent eviction of worker pods
I am currently planning to upgrade our Cloud Composer environment from Composer 1 to 2. However I am quite concerned about disruptions that could occur in Cloud Composer 2 due to the new autoscaling behavior inherited from GKE Autopilot. In…
Stephen
- 8,508
- 12
- 56
- 96
0
votes
1 answer
Cloud Composer GKE coupling broken since composer-2.3.5-airflow-2.5.3
Since upgrading to the latest composer (composer-2.3.5-airflow-2.5.3
), our all our long-running GKE pods seem to fail.
Example error:
[2023-08-01, 14:20:23 CEST] {before.py:40} INFO - Starting call to…
Jonny5
- 1,390
- 1
- 15
- 41
0
votes
1 answer
Hash airflow in the first line of dag
Is there any significance of adding '''# airflow''' as the first line in airflow dag.
Because python will treat it as a comment.
pushpendu dhara
- 97
- 7
0
votes
0 answers
Google Cloud Composer2 - how to mounts file?
I am trying to mount a json file with gcp authentication on cloud composer2 pod.
I couldn't find a way in the official documentation.
i want to know how..
nightshiftman
- 71
- 1
- 5
0
votes
2 answers
DataprocCreateClusterOperator - in Composer2(Airflow), pip install specific versions
I'm using DataprocCreateClusterOperator to create Dataproc cluster in the dag, using the following code (only relevant part of the dag shown below) :
I'm using init_actions_uris & passing script pip_install_versions.sh
This script installs…
Karan Alang
- 869
- 2
- 10
- 35