Stack Overflow
Stack Overflow

Questions tagged [faust]

Faust is a stream processing library, porting the ideas from Kafka Streams to Python.

Faust is used to build high performance distributed systems and real-time data pipelines that process billions of events every day.

Faust provides both stream processing and event processing, sharing similarity with tools such as Kafka Streams, Apache Spark/Storm/Samza/Flink,

It does not use a DSL, it’s just Python! This means you can use all your favorite Python libraries when stream processing: NumPy, PyTorch, Pandas, NLTK, Django, Flask, SQLAlchemy, ++

Faust requires Python 3.6 or later for the new async/await syntax, and variable type annotations.

Repository: https://github.com/robinhood/faust

96 questions
0
votes
1 answer

ERROR : cannot open file 'worker' : No such file or directory with faust hello_world eg

I am trying to follow faust [hello_world][1] example to work on my Mac. And getting below error: $ faust -A hello_world worker -l info ERROR : cannot open file 'worker' : No such file or directory to my amaze below errors out as well. $ faust…
Debpriya Seal
  • 73
  • 1
  • 10
0
votes
1 answer

Processing a maximum number of tasks per consumer in a topic, for a given time window?

My producer generates n tasks from a single input message and publishes these on topic. The requirement is that out of all the individual consumers in the consumer group of topic, no one of them should process more than 3 of any of these n tasks…
deed02392
  • 4,799
  • 2
  • 31
  • 48
0
votes
2 answers

Access kafka headers in rpc replies using faust streamming

Is it possible to access kafka headers in the faust rpc replies? Here is example of two faust agents. One (the pow) calls another (mul) and receive a result as a value. But how to know the kafka headers in the reply topic? #!/usr/bin/env python from…
Роман Коптев
  • 1,555
  • 1
  • 13
  • 35
0
votes
2 answers

Is there any alternative of celery countdown and eta in Faust framework (or other framework)?

I want to trigger some tasks on specific time by user settings. For example, if user set 4:00 PM then I would run a task at 4:00PM It could be handle in Celery with countdown and eta. But my broker prefer is Kafka. Is there any alternative of Celery…
Quyen Nguyen
  • 51
  • 8
0
votes
1 answer

How to work with asyncpg connection pool in Faust agent?

I want Faust agent to write to PostgreSQL table. I'd like to use asyncpg connection pool but cannot find a clean way to inject it into the app initialization code.
Subideal Ox
  • 61
  • 2
-1
votes
1 answer

Is there any alternative in Python like Apache Nifi?

I need to process the data and sent to Faust(streaming) like Nifi is sending data to kafka. Is there any alternative for Nifi in Python since I can't integrate Faust with Nifi In nifi I can process and convert csv to json and send to Kafka. Since I…
user10508414
1 2 3 4 5 6
7