Stack Overflow
Stack Overflow

Questions tagged [backpressure]

Backpressure refers to the build-up of data at an I/O switch when buffers are full and not able to receive additional data. No additional data packets are transferred until the bottleneck of data has been eliminated or the buffer has been emptied.

Backpressure refers to the build-up of data at an I/O switch when buffers are full and not able to receive additional data. No additional data packets are transferred until the bottleneck of data has been eliminated or the buffer has been emptied.

Source - Techopedia

164 questions
3
votes
1 answer

Java8 Stream or Reactive / Observer for Database Requests

I'm rethinking our Spring MVC application behavior, whether it's better to pull (Java8 Stream) data from the database or let the database push (Reactive / Observable) it's data and use backpressure to control the amount. Current situation: User…
Benjamin M
  • 23,599
  • 32
  • 121
  • 201
3
votes
1 answer

RxJava Backpressure and number of calls to producer

I am trying to create infinity-scroll in my Android application using backpressure in rx Java. I want it to call external service only requested number of times (after calling request(1)). But after using flatmap every subscribe loads 16 pages.…
dmnk_89
  • 499
  • 5
  • 9
3
votes
1 answer

Synchronized Feedback with Akka Streams

What I am trying to achieve is implementing something like a synchronized feedback loop with akka streams. Let's say you've got a Flow[Int].filter(_ % 5 == 0). When you broadcast a stream of Int's to this flow and zip the tuples directly behind it,…
Martin Seeler
  • 6,874
  • 3
  • 33
  • 45
3
votes
1 answer

How do you implement a stream that properly handles backpressure in node.js?

I can't for the life of me figure out how to implement a stream that properly handles backpressure. Should you never use pause and resume? I have this implementation I'm trying to get to work correctly: var StreamPeeker = exports.StreamPeeker =…
B T
  • 57,525
  • 34
  • 189
  • 207
3
votes
0 answers

Check current back pressure level in Node.js stream

Is there a way in Node.js to check the back pressure state of a Stream? For example to compare it to the high water mark. My use case is inside a unit tests, I'd like to make sure a transform stream plugin is draining the pipe and doesn't build up…
Simon Boudrias
  • 42,953
  • 16
  • 99
  • 134
3
votes
1 answer

How to use Akka BoundedMailBox to throttle a producer

I have two actors, one is producing messages and the other is consuming the messages at a fixed rate. Is it possible to have the producer throttled by the consumers BoundedMailBox? (back pressure) My producer is currently periodically scheduled…
JeffV
  • 52,985
  • 32
  • 103
  • 124
3
votes
0 answers

Node.js node-csv-parser deadlocking because back pressure isn't working

I'm trying to read URLs from a CSV file, check whether they're available, and write out bad URLs to another file. I get through about a few thousand, before slowing down drastically (from 5 checks a second to a check every 10 seconds), and then get…
Dan Kohn
  • 33,811
  • 9
  • 84
  • 100
2
votes
1 answer

WebSocket in browser with true back-pressure

I learned the hard way that the browsers' WebSocket API does not support back-pressure. Is there a way to work around this limitation without having to manually implement control flow on the application layer protocol? Would it for example be…
piegames
  • 975
  • 12
  • 31
2
votes
0 answers

How can I get number of file descriptors in use/open connections in the context of Java AWS SDK?

Problem Description Currently, I see SdkClientException: Unable to execute HTTP request: Acquire operation took longer than the configured maximum time (full error log is below) exception from Lambda SDK 2.0 (with Netty client) in a service, where…
Leonti Brechko
  • 31
  • 3
2
votes
2 answers

Reading from a Flux in chunks

Is it possible do read from a webflux flux in chunks? ( other than using delayElements ) For example after I write Flux.range(1, 10).doOnNext(System.out::println).take(5).subscribe(); is there any way to continue to read the next 5 integers? If…
Gonen I
  • 5,576
  • 1
  • 29
  • 60
2
votes
1 answer

MQTT wildcard subscriptions, backpressure and QOS

This is more of a generic question, however it might be handled differently in different clients or protocol versions or possibly even server versions. So I'm talking QOS level 2 subscription here. In which case packets are handled in order. And…
Christiaan Willemsen
  • 131
  • 6
2
votes
1 answer

What happens to nodejs server requests when the process is blocked

What happens to incoming requests when a nodejs server is blocked? There are times when the server will be blocked because it is chewing through something computationally expensive, or perhaps doing some synchronous IO (e.g. writing to a sqlite…
andykais
  • 996
  • 2
  • 10
  • 27
2
votes
1 answer

Run a batch off promises in Typescript

I have an array of Promises on my Typescript project. Most of the time I'm using Promise.all() in order to execute all of them in parallel: let promises:Array> = .... Promise.all(promises) .then(results:Array => ....) …
Benjamin Bohec
  • 118
  • 2
  • 11
2
votes
1 answer

Retry after delay on back pressure with Spring Project Reactor?

Background I'm trying to implement something similar to a simple non-blocking rate-limiter with Spring Project Reactor version 3.3.0. For example, to limit the number to 100 requests per second I use this implementation: myFlux …
Johan
  • 37,479
  • 32
  • 149
  • 237
2
votes
1 answer

Does NodeJS WebSockets (ws) module implement backpressure?

I'm implementing a WebSockets server on NodeJS using the ws module. The server should send once per minute an update to all clients. I have this already implemented, but I have some concerns about its functionality in conditions where client…
juhist
  • 4,210
  • 16
  • 33
1 2 3
…
10 11