Stack Overflow
Stack Overflow

Questions tagged [backpressure]

Backpressure refers to the build-up of data at an I/O switch when buffers are full and not able to receive additional data. No additional data packets are transferred until the bottleneck of data has been eliminated or the buffer has been emptied.

Backpressure refers to the build-up of data at an I/O switch when buffers are full and not able to receive additional data. No additional data packets are transferred until the bottleneck of data has been eliminated or the buffer has been emptied.

Source - Techopedia

164 questions
6
votes
2 answers

What is the best way to get backpressure for Cassandra Writes?

I have a service that consumes messages off of a queue at a rate that I control. I do some processing and then attempt to write to a Cassandra cluster via the Datastax Java client. I have setup my Cassandra cluster with maxRequestsPerConnection…
Micah Zoltu
  • 6,764
  • 5
  • 44
  • 72
6
votes
2 answers

How to buffer only latest emission from rx.Observable during backpressure

I have an rx.Observable which emits the progress of a task to onNext(). The onNext() emissions can sometimes occur so quickly that the Observer cannot keep up, resulting in backpressure. I would like to handle the backpressure by only buffering the…
ashughes
  • 7,155
  • 9
  • 48
  • 54
5
votes
2 answers

Possible to make an event handler wait until async / Promise-based code is done?

I am using the excellent Papa Parse library in nodejs mode, to stream a large (500 MB) CSV file of over 1 million rows, into a slow persistence API, that can only take one request at a time. The persistence API is based on Promises, but from Papa…
blitter
  • 347
  • 5
  • 21
5
votes
0 answers

Redis PubSub Lettuce: How is back pressure handled?

We are using RedisPubSubReactiveCommands and calling the subscribe and observeChannels methods of Lettuce. In case of a fast publisher and a slow subscriber, how is the back pressure handled? Since publishers and subscribers are independent in…
Dollyg
  • 317
  • 2
  • 7
5
votes
3 answers

Kotlin Flow onBackpressureDrop RxJava2 analog

In RxJava 2 Flowable there are different backpressure strategies, among them the most interesting are: LATEST BUFFER DROP which are respected throughout whole Rx chain. In Kotlin there is Flow, which declares that it has backpressure support…
krossovochkin
  • 12,030
  • 7
  • 31
  • 54
5
votes
2 answers

RxJS take first then throttle and wait

I want to observer the mousewheel event using RxJS-DOM so that when the first event fires, I forward that on and then drop any and all values until the delay between subsequent values passes a previously specified duration. The operator I'm…
barndog
  • 6,975
  • 8
  • 53
  • 105
5
votes
1 answer

Why do we need Publish and RefCount Rx operators in this case?

I'm trying to familiarise myself with the problem of reactive backpressure handling, specifically by reading this wiki: https://github.com/ReactiveX/RxJava/wiki/Backpressure In the buffer paragraph, we have this more involved example code: // we…
Zsombor Erdődy-Nagy
  • 16,864
  • 16
  • 76
  • 101
5
votes
2 answers

An enumerator wrapper that pre-buffers a number of items from underlying enumerator in advance

Suppose I have some IEnumerator which does a fair amount of processing inside the MoveNext() method. The code consuming from that enumerator does not just consume as fast as data is available, but occasionally waits (the specifics of which are…
Evgeniy Berezovsky
  • 18,571
  • 13
  • 82
  • 156
5
votes
1 answer

WebRTC DataChannel flow/control/back-pressure

The RTCDataChannel API does not provide any kind of flow/control or back-pressure, does this mean, that a sender could, theoretically, crash the browser of the receiver ? In my opinion the browser, (Chrome, Firefox, etc. all use SCTP under the…
Kr0e
  • 2,149
  • 2
  • 24
  • 40
4
votes
1 answer

RxJava multiple consumers of one publisher

I'm writing some kind of middleware HTTP proxy with cache. The workflow is: Client requests this proxy for resource If resurce exists in cache, proxy returns it If resource wasn't found, proxy fetching remote resource and returns to the user. Proxy…
Kirill
  • 7,580
  • 6
  • 44
  • 95
4
votes
3 answers

How to measure Websocket backpressure or network buffer from client

I am using the ws Node.js package to create a simple WebSocket client connection to a server that is sending hundreds of messages per second. Even with a simple onMessage handler that just console.logs incoming messages, the client cannot keep up.…
oliakaoil
  • 1,615
  • 2
  • 15
  • 36
4
votes
2 answers

How does subscription.request(n) in Flow API perform backpressure at any n value?

I'm playing around with the Flow API and, so far, I understand that the request() method is used for back pressure. Most articles state that this is akin to controlling the speed of consumption. However, almost every example code I see passes the…
Carven
  • 14,988
  • 29
  • 118
  • 161
4
votes
1 answer

how does Monix use back-pressure with flatMap operator?

Monix use Ack to sync emitted messages, but if I use groupBy and flatMap, the inner Observable not follow the back-pressure out of the source. See this test code please: import java.util.concurrent.TimeUnit import…
LoranceChen
  • 2,453
  • 2
  • 22
  • 48
4
votes
1 answer

Kotlin coroutines: concurrent execution throttling

Imagine we are reading messages from a message queue and on receive pushing them for processing into a threadpool. There are limited number of threads, so if all the threads are busy, we'll have natural backpressure. How this can be solved in Kotlin…
silent-box
  • 1,649
  • 3
  • 21
  • 40
4
votes
1 answer

Application-level back-pressure VS TCP native flow-control

I am investigating why some systems implement application-level back-pressure given that TCP provides native flow control. I was reading, in particular, akka-streams and (an higher level discussion) reactive streams. Is it only to abstract out the…
affo
  • 453
  • 3
  • 15
1
2
3
…
10 11