Stack Overflow
Stack Overflow

Questions tagged [mwaa]

Amazon Managed Workflows for Apache Airflow (Amazon MWAA) is a managed service for Apache Airflow that makes it easy for you to build and manage your workflows in the cloud.

you can easily combine data using any of Apache Airflow’s open source integrations.

use the same familiar Airflow platform as you do today to manage their workflows and now enjoy improved scalability, availability, and security without the operational burden of having to manage the underlying infrastructure

automatically scales capacity up to meet demand and back down to conserve resources and minimize costs

integrated with AWS security services to enable secure access

manages the provisioning and ongoing maintenance of Apache Airflow

https://docs.aws.amazon.com/mwaa/

291 questions
0
votes
1 answer

Why can't I get to the UI for AWS Airflow using webtoken?

In AWS Console, I did the following: Created an S3 bucket & key: s3://my-airflow and s3://my-airflow/dags Setup an Airflow Environment. Created and attached a Service role as described here:…
cosbor11
  • 14,709
  • 10
  • 54
  • 69
-1
votes
1 answer

Airflow Permission denied while getting Drive credentials

I am trying to run a bigquery query on Airflow with MWAA. This query uses a table that is based on a Google Sheet. When I run it, I have the following error: google.api_core.exceptions.Forbidden: 403 Access Denied: BigQuery BigQuery: Permission…
nolwww
  • 1,355
  • 1
  • 15
  • 33
-1
votes
1 answer

Migrating from Astronomer airflow setup to MWAA

Currently we are using Apache airflow platform provided by Astronomer with Kubernetes executer and we are using PostgreSQL as a backend DB. Now we want move to MWAA. I want some help if anyone has done the same. How to migrate existing DAGs and…
Rjkant
  • 1
-1
votes
2 answers

How to Call stored procedure function in airflow using PostgresOperator

I am not sure about how to call stored procedure in airflow operator- eg. Stored Procedure is- Call goal.dba.sp_ctrl_calendar() My code- from airflow.operators.postgres_operator import PostgresOperator sp_ctrl_calendar = PostgresOperator( …
JaySean
  • 125
  • 1
  • 15
-3
votes
1 answer

I want to add an extra column to csv file in s3

I have a 10+ gb csv file coming in everyday in my S3 and I want to add an extra column to that file and save it there. I am using MWAA to do this task but its failing due to large file size. tried using boto3.client.get_object. And then use…
Abhi
  • 3
  • 1
-3
votes
2 answers

Ignoring already configured resources in Terraform - AWS

There is a terraform code to configure an MWAA Environment in AWS. When it runs second time, no need to create IAM role or policy again. So it gives an error. How to ignore the creation of existing resources in TF?
Amantha
  • 17
  • 6
1 2 3
…
19
20