Stack Overflow
Stack Overflow

Questions tagged [faust]

Faust is a stream processing library, porting the ideas from Kafka Streams to Python.

Faust is used to build high performance distributed systems and real-time data pipelines that process billions of events every day.

Faust provides both stream processing and event processing, sharing similarity with tools such as Kafka Streams, Apache Spark/Storm/Samza/Flink,

It does not use a DSL, it’s just Python! This means you can use all your favorite Python libraries when stream processing: NumPy, PyTorch, Pandas, NLTK, Django, Flask, SQLAlchemy, ++

Faust requires Python 3.6 or later for the new async/await syntax, and variable type annotations.

Repository: https://github.com/robinhood/faust

96 questions
0
votes
1 answer

Checking the existence of a kafka topic in Faust app

I am new to Faust. Task in hand is to make sure a few Kafka topics exist. Idea is to check if all the required topics exist in app's liveliness check. I have gone through Faust documentation but didn't find a way to do that. Is it even possible to…
ujjwal
  • 428
  • 3
  • 16
0
votes
2 answers

Sliding window using Faust

Does anyone know how to implement a sliding window using Faust? The idea is to count the occurances of a key in a 10, 30, 60, and 300s window, but we need that on a 1s or on every update basis. I have a dodgy workaround, which seems very inefficient…
Fonty
  • 239
  • 2
  • 11
0
votes
1 answer

ModuleNotFoundError("'kafka' is not a valid name. Did you mean one of aiokafka, kafka?")

I am using Celery and Kafka to run some jobs in order to push data to Kafka. I also use Faust to connect the workers. But unfortunately, I got an error after running faust -A project.streams.app worker -l info in order to run the pipeline. I wonder…
Ghazaleh Kharadpoor
  • 146
  • 3
  • 13
0
votes
1 answer

FAUST: AttributeError: 'NoneType' object has no attribute 'topic_partition'

i'm having some trouble to do a simple task with python faust, please take a look on the problem and see if you can help me. Steps to reproduce i used this code: import faust from settings import KAFKA_SERVER app = faust.App('streams',…
jhonatan teixeira
  • 760
  • 1
  • 6
  • 8
0
votes
1 answer

Faust.web API Server hangs on exception

I am new to Python and Faust. We are using Faust.web to develop our APIs. Everything seems to be working except exceptions. Problem is, when we run API server and hit a API from Postman that results in exception in our code, we correctly get 500…
ujjwal
  • 428
  • 3
  • 16
0
votes
1 answer

How to catch faust python json serialization error?

I have simple faust agent. It consumes jsons from kafka topic, and parses them to dicts by default faust serializator: @app.agent(source_topic, sink=[destination_topic]) async def fetch(records): async for record in records: result =…
coolsv
  • 268
  • 3
  • 17
0
votes
1 answer

How to specify python faust.Record field which name starts with numbers?

I'm describing faust model by extending faust.Record class. My input json looks like: { "1": "some_string" "2": "some_string" } So I have to create class fields, named with numbers only. I know, it restricted in Python in general, but is there…
coolsv
  • 268
  • 3
  • 17
0
votes
0 answers

How i can run faust workers and manage them

I want to run faust workers from my code (run from signals). And i need manage workers (pause, kill, reload). In the future, I would like to manage workers in docker containers. That is, 3 workers in one container, 2 in another ... Or maybe there…
Teregov_Ruslan
  • 1
0
votes
3 answers

Forwarding Events with Python Faust

I am trying to forward messages to internal topics in faust. As suggested by faust in this example: https://faust.readthedocs.io/en/latest/playbooks/vskafka.html I have the following code in_topic = app.topic("in_topic", internal=False,…
Chris
  • 476
  • 4
  • 10
0
votes
1 answer

Debug mode for Faust worker in PyCharm

I'm trying to run faust worker (https://faust.readthedocs.io) in debug mode. At this monent I did this: As you can see this is not actual debug. I expect this: Can you say what I'm doing wrong?
Альберт Александров
  • 807
  • 1
  • 15
  • 27
0
votes
0 answers

Faust for Python: how to start worker to run hello world

In Spyder, i am trying the Hello World application from the official Faust website: import faust app = faust.App( 'hello-world', broker='kafka://localhost:9092', value_serializer='raw', ) greetings_topic =…
pam
  • 676
  • 1
  • 7
  • 27
0
votes
2 answers

How run hello_world example in pythons' faust lib

my kafka deployed in kubernetes, so I forwarded port 9092 kubectl port-forward -n kafka svc/kafka 9092 and try run hello_world.py but faced with exception [2020-10-01 00:31:52,331] [3503] [ERROR] Unable connect to node with id 0: [Errno -2] Name or…
Александр Шинкевич
  • 31
  • 3
0
votes
1 answer

FAUST Asynchronous kafka message processing concurrency is not working

Currently, I'm trying to read the data from kafka topic and call the rest-API asynchronously with data that I fetched from kafka topic. here rest-api gives response instantly if msg is Meher else response will takes 5…
Learnis
  • 526
  • 5
  • 25
0
votes
1 answer

Faust Python Blocking Function

I'm looking a good way to run a none async lib inside a faust agent : @app.agent(topic) async def cp_agent(cp_stream): async for batch in cp_stream.take(100, within=5): print("------ Indexing a CPE BATCH -----") [... Here Use of…
Lightoyou
  • 1
0
votes
1 answer

How to get the frequency of the input from mic

I'm trying to analyze the input mic sound and get the frequency value of the input. Is there any library I can get a frequency of the input? I'm using FAUST. Thanks
Billy
  • 1
  • 1
1 2 3 4 5
6
7