4

The cloud workflow doesn't come with a scheduling feature. Apart from that, what are all the differences between these two services in terms of features? In which use case should we prefer the workflow over composer or vice versa?

Robin Raju
  • 1,135
  • 10
  • 17
Balasubramanian Naagarajan
  • 338
  • 3
  • 17
  • 1
    Is that can help? https://cloud.google.com/blog/products/application-development/get-to-know-google-cloud-workflows – guillaume blaquiere Jul 07 '21 at 07:42
  • The 2 product achieve the same thing, but in 2 different manners and paradigm – guillaume blaquiere Jul 07 '21 at 07:43
  • 2
    In my opinion it comes down to whether you need the processing power of Composer/Airflow. For our project, we are largely just triggering a series of steps, and for that Workflows is easy to use and works perfectly. We are in the free tier of Workflows. Only downside to Workflows I have found is that the documentation is a work in progress. – Benny Aug 01 '21 at 19:54

2 Answers2

5

There are some key differences to consider when choosing between the two solutions :

A Composer instance needs to be in a running state to trigger DAGs and you'll also need to size your Cloud Composer instance based on your usage, You do not need to do this in Cloud Workflows as it is a Serverless service and you pay for anytime a workflow is triggered

Another key difference is that Cloud Composer is really convenient for writing and orchestrating data pipelines because of it's internal scheduler and also because of the provided Operators, You can interact with any Data services inside of GCP.
However, Cloud Workflows interacts with Cloud Functions, wich is a task that Composer cannot do really well.

Will
  • 2,057
  • 1
  • 22
  • 34
2

Both Composer and Workflows support orchestrating multiple services and can handle long running workflows. Despite there being some overlap in the capabilities of these products, each has differentiators that make them well suited to particular use cases.

Composer is most commonly used for orchestrating the transformation of data as part of ELT or data engineering. Workflows, in contrast, is focused on the orchestration of HTTP-based services built with Cloud Functions, Cloud Run, or external APIs.

Composer is designed for orchestrating batch workloads that can handle a delay of a few seconds between task executions. It wouldn’t be suitable if low latency was required in between tasks, whereas Workflows is designed for latency sensitive use cases.

While you don’t have to worry about maintaining Airflow deployments in Composer, you do need to specify how many workers you need for a given Composer environment. Workflows is completely serverless; there is no infrastructure to manage or scale.

For further information refer to this google blog article and this one.

Async_Niombor
  • 21
  • 2