Stack Overflow
Stack Overflow

Questions tagged [backpressure]

Backpressure refers to the build-up of data at an I/O switch when buffers are full and not able to receive additional data. No additional data packets are transferred until the bottleneck of data has been eliminated or the buffer has been emptied.

Backpressure refers to the build-up of data at an I/O switch when buffers are full and not able to receive additional data. No additional data packets are transferred until the bottleneck of data has been eliminated or the buffer has been emptied.

Source - Techopedia

164 questions
0
votes
1 answer

node.js back-pressure and drain event. I encountered memory leak warning

I am currently studying node.js back-pressure myself. I intend not to use .pipe() or .pipeline() because I want to understand back-pressure and drain event. but I don't know how to write appropriate drain handler. let's see the below code. "use…
Juhyun Park
  • 19
  • 6
0
votes
1 answer

Confusion on RxJava onBackPressureBuffer and onBackPressureDrop

I want my code work like this graph, but it does not work ... My Code: private fun register(cls: Class): Flowable> { return FlowableFromObservable(mRelay).onBackpressureBuffer(4).filter( /* filter target…
ZSpirytus
  • 339
  • 2
  • 10
0
votes
1 answer

How to limit the number of unprocessed records for AWS Kinesis?

For example, RabbitMQ has a way in setting queue limits. If that limit is reached the new messages from publishers will be rejected, thus applying some kind of backpressure that starts from consumers to the producers. (since messages in queues means…
raduone
  • 191
  • 1
  • 10
0
votes
0 answers

how to implement back-pressure for a reactive network library?

There is a little long background story about the problem in my socket reactive library working process. The socket library mainly based on a reactive library named Monix(similar ReactiveX). Monix has the best practice to handle back-pressure by…
LoranceChen
  • 2,453
  • 2
  • 22
  • 48
0
votes
1 answer

Multiple Flowable observedOn single Scheduler, event not consumed in same order as emitted

I'm consuming event using from different external sources/subscription wrap to different Flowable. The source doesn't matter as I could reproduce the issue with a simple loop. I have : Different FlowableEmitter (3 is enough to reproduce)…
Rod1677
  • 23
  • 3
0
votes
1 answer

Akka-streams backpressure on broadcast with async processing

I am struggling with understanding if akka-stream enforces backpressure on Source when having a broadcast with one branch taking a lot of time (asynchronous) in the graph. I tried buffer and batch to see if there was any backpressure applied on the…
Florian Felten
  • 23
  • 1
  • 1
  • 5
0
votes
2 answers

How to write data from flink pipeline to redis efficiently

I am building a pipeline in Apache flink sql api. The pipeline does simple projection query. However, I need to write the tuples (precisely some elements in the each tuple) once before the query and another time after the query. It turned out that…
ahmadi
  • 17
  • 7
0
votes
1 answer

RX terminolgy: Async processing in RX operator when there are frequent observable notifications

The purpose is to do some async work on a scarce resource in a RX operator, Select for example. Issues arise when observable notifications came at a rate that is faster than the time it takes for the async operation to complete. Now I actually…
chrisc
  • 105
  • 6
0
votes
1 answer

How to reduce ingestion rate of Kafka Spout and enable Back pressure?

I am using storm-kafka-client 1.1.1 and storm-core 1.1.0. I have tuned the following params but not able to enable back-pressure and reduce ingestion rate of kafka-spout. Spout consuming 2000 messages per sec. Downstream Bolt takes 50 ms to process…
kaniska Mandal
  • 189
  • 1
  • 12
0
votes
2 answers

Does a FlowableOperator inherently supports backpressure?

I've implemented an FlowableOperator as described in the RxJava2 wiki (https://github.com/ReactiveX/RxJava/wiki/Writing-operators-for-2.0#operator-targeting-lift) except that I perform some testing in the onNext() operation something like…
ctranxuan
  • 785
  • 5
  • 9
0
votes
1 answer

Rx Java 2 pre-pull next item on separate thread

Scenario: I have a stream of data I am reading from the database. What I would like to do is read a chunk of data, process it and stream it using rx-java 2. But while I am processing and streaming it I would like to load the next chunk of data on a…
cchanley2003
  • 102
  • 1
  • 8
0
votes
0 answers

What is the resources consuming when Observable switched to Flowable

We are switching project from javaRx 1.x to javaRx 2.x. i cant really understand: Why i should keep Observable in my project instead of using Flowable everywhere. If i know the place there is no any back pressure do i need to use Flowable with no…
Aleksandr Belov
  • 1
  • 2
0
votes
1 answer

Create a flowable with generate function using RxJava2

I need to create a custom Flowable with backpressure implemented. I'm trying to achieve some sort of paging. That means when downstream requests 5 items I will "ask the data source" for items 0 - 5. Then when downstream needs another 5, I'll get…
bio007
  • 893
  • 11
  • 20
0
votes
0 answers

How to control akka stream backpressure slow downstream api

I have one API which calls another Downstream API. The downstream api has its limitation, the throughput of it only can handle 75 request per sec. I would like to use akka stream to control the API to not exceed 75 requests per sec for the…
Xiaohe Dong
  • 4,953
  • 6
  • 24
  • 53
0
votes
0 answers

backpressure .net reactive - fast obeservable

I am working to build a producer..consumer pattern using .NET reactive. Producer reads messages from Kafka message bus. Once the message is read, it needs to be handed over to the consumer to process the message. I was able to do that using .NET…
Ashish Bhatia
  • 161
  • 1
  • 6
1 2 3
…
10
11