Stack Overflow
Stack Overflow

Questions tagged [backpressure]

Backpressure refers to the build-up of data at an I/O switch when buffers are full and not able to receive additional data. No additional data packets are transferred until the bottleneck of data has been eliminated or the buffer has been emptied.

Backpressure refers to the build-up of data at an I/O switch when buffers are full and not able to receive additional data. No additional data packets are transferred until the bottleneck of data has been eliminated or the buffer has been emptied.

Source - Techopedia

164 questions
1
vote
1 answer

Test backpressure application to a Flux in a Reactor application

I am using Reactor to read from a Kafka topic. The elaboration of each message requests a query into a MongoDB, that is slower than the read of messages from the Kafka topic. So, I applied backpressure handling to the stream. receiver.receive() …
riccardo.cardin
  • 7,971
  • 5
  • 57
  • 106
1
vote
1 answer

Akka stream actor-conflation-ratelimit-actor drops first few messages (sometimes)

A simple conflating combo (below) sometimes prints a debug message at staartup saying it's dropping messages because of zero demand. I would expect conflation stage to provide infinite demand, so the above should never be the case. What am I…
bobah
  • 18,364
  • 2
  • 37
  • 70
1
vote
1 answer

How to get the actual latest event on back pressure in Rxjava2? Flowable.onBackpressureLatest() not worked as expected

When a producer produces event faster than a customer consumes. I thought using Flowable with onBackpressureLatest(), I can get the latest event emitted. But it turns out there's a default buffer of size 128. What I got was the dated event…
Will Li
  • 495
  • 1
  • 3
  • 12
1
vote
1 answer

RxJava 1.x: How to simulate backpressure in unit test

Given the code: logItemPublisher .buffer( loggingProperties.getBufferTimeoutMillis(), TimeUnit.MILLISECONDS, loggingProperties.getBufferSize(), logDispatchScheduler ) …
Abhijit Sarkar
  • 21,927
  • 20
  • 110
  • 219
1
vote
0 answers

Backpressure when using interval

I have this RxJava2 flowable private val pulseFlowable = Flowable .interval(200, TimeUnit.MILLISECONDS) .subscribeOn(Schedulers.computation()) and this subscriber pulseFlowable .observeOn(Schedulers.computation()) .subscribe( …
Dabbler
  • 9,733
  • 5
  • 41
  • 64
1
vote
1 answer

correct way to process data from an rxjs oberservable subscription with back-pressure

I have an rxjs.observable (rxjs version 6.2.1) that returns urls I need to make a GET request to. var subscription = urlObservable$.subscribe( function (url) { console.log('new URL: ' + url); processURL(url) }, function (err) {…
otmezger
  • 10,410
  • 21
  • 64
  • 90
1
vote
1 answer

node.js how to handle fast producer and slow consumer with backpressure

I'm very novice in node.js and don't understand the documentation about streams. Hoping to get some tips. I'm reading a very large file line, and then for each line I'm calling an async network api. Obviously the local file is read much faster…
Avba
  • 14,822
  • 20
  • 92
  • 192
1
vote
1 answer

Avoiding data loss when slow consumers force backpressure in stream processing (spark, aws)

I'm new to distributed stream processing (Spark). I've read some tutorials/examples which cover how backpressure results in the producer(s) slowing down in response to overloaded consumers. The classic example given is ingesting and analyzing…
andrasp
  • 168
  • 1
  • 1
  • 7
1
vote
1 answer

Backpressure in a Spring DeferredResult + Akka actors application

I am thinking of using a chain of Akka workers to model a workflow inside a DeferredResult based Spring MVC web application. Essentially the controller will return a DeferredResult and the actors in the chain will work to populate a…
Kislay Verma
  • 59
  • 1
  • 10
1
vote
0 answers

Selective request-throttling using akka-http stream

I got one API which calls two another Downstream APIs. One downstream api (https://test/foo) is really important and it is very fast. Another slow downstream api (https://test/bar) has its limitation, the throughput of it only can handle 50…
Xiaohe Dong
  • 4,953
  • 6
  • 24
  • 53
1
vote
1 answer

How to pass observable's elements in correct (real) time?

Suppose I have an aobservable, that emits 2D points, denoting turn points of some route public static double[][] points = new double[][]{ {0, 0}, {0, 10}, {1, 10}, {1, 0}, {0, 0} }; public static…
Suzan Cioc
  • 29,281
  • 63
  • 213
  • 385
1
vote
1 answer

Node.js: Wrong extra bytes were wrote on Back pressure in writable stream

I made simple binary transmission with Socket.io to transfer file to server from client. I thought it worked, but I realized that size of the file was different. On writableStream.write failed, I attached drain event handler to keep wait until it…
modernator
  • 4,341
  • 12
  • 47
  • 76
1
vote
2 answers

How to skip too frequent mouse events in RxJS 5 (beta)?

I'm using RxJS 5 (beta10) in a project. Most of my events originate from a mouse or touch drag, and I would only ever need to hear the last one. i.e. How do I do a lossy stream in RxJS5? Running the demos in certain tablets clearly show a "drag"…
akauppi
  • 17,018
  • 15
  • 95
  • 120
1
vote
1 answer

How to design a reactive stream actor system with fan out capability

I am trying to implement an actor-based system with back-pressure capability. As a requirement, the master process receives streaming data in JSON format. Each JSON event however has several fields such as {ip: '123.43.12.1', country: 'US', ...…
auxdx
  • 2,313
  • 3
  • 21
  • 25
1
vote
1 answer

core.async toggle channel mix

I am experimenting with core.async's mixes. It seems that muting an input channel in a mix would be one of possible ways of implementing backpressure. I am using the code below: (def output-chan (chan)) (def input-chan (chan)) (def mixer (admix (mix…
siphiuel
  • 3,480
  • 4
  • 31
  • 34
1 2 3
…
10 11