Stack Overflow
Stack Overflow

Questions tagged [backpressure]

Backpressure refers to the build-up of data at an I/O switch when buffers are full and not able to receive additional data. No additional data packets are transferred until the bottleneck of data has been eliminated or the buffer has been emptied.

Backpressure refers to the build-up of data at an I/O switch when buffers are full and not able to receive additional data. No additional data packets are transferred until the bottleneck of data has been eliminated or the buffer has been emptied.

Source - Techopedia

164 questions
1
vote
1 answer

How to apply back pressure to kafka producer?

Back pressure can help by limiting the queue size, thereby maintaining a high throughput rate and good response times for jobs already in the queue. In RabbitMQ it can be applied setting Queue length limit. How can it be done with Kafka ? Can this…
Shubham
  • 211
  • 2
  • 13
1
vote
1 answer

How to do server-side backpressure in gRPC?

I just find that in C++, when using AsyncService, even if I don't request a new request, gRPC will still read data from the network. This caused a huge memory usage in my system. Detailed Scenario: I have a client that will send a lot of requests to…
Lixin Wei
  • 441
  • 5
  • 17
1
vote
0 answers

Is backpressure only needed when the system cannot scale?

I was wondering if using backpressure in a system generally did not provide a benefit as it just leads to higher latency / less throughput because it prevents scaling out the overwhelmed resources. Should it only be used for resources that cannot be…
Hans Wurst
  • 376
  • 1
  • 3
  • 12
1
vote
2 answers

How to measure DispatchQueue back pressure

My program has many parallel processes each on their own queue. I'd like to be able to visualize/measure the back pressure of the queues. One approach is to count every block that enters and exits, but I'm sure GCD has this information already. Is…
Michael Ozeryansky
  • 7,204
  • 4
  • 49
  • 61
1
vote
1 answer

Make sure subscribe (sink) happens only once

I'm trying to write a "closure like" combine observers. Let's say I have MyClass class MyClass { @Published var value: Int = 0 func doSomethingClosure(completion: (Int?) -> Void) { value += 1 completion(value) } …
Maor Atlas
  • 107
  • 5
1
vote
1 answer

How to apply back pressure with AWS Lambda and MSK (Kafka) event stream

Is it possible to do this? When dealing with SQS event streams, if the Lambda function does not have adequate reserved concurrency, the function will be throttled, and the unprocessed events / messages can be retried via the SQS redrive policy. I've…
InTooDeep
  • 578
  • 2
  • 7
  • 19
1
vote
0 answers

How do RSocket issue lease to multiple clients?

I could create a server lease to a single client as follows： @Slf4j public class LeaseServer { private static final String SERVER_TAG = "server"; public static void main(String[] args) throws InterruptedException { …
Ten Wang
  • 51
  • 1
  • 1
  • 6
1
vote
1 answer

Micronaut backpressure

We are looking for the best way to limit the number of concurrent connections accepted by an HTTP endpoint exposed with Micronaut 2.0.1. The endpoint accepts image files, writes them to disk and creates thumbnails. With too many requests we are…
jjrz
  • 325
  • 1
  • 2
  • 14
1
vote
1 answer

Backpressure implementation with redux-observable

I'm trying to implement backpressure logic in my react application. I find a nice post about this here and trying to add this to my app. Now i have some code: // epic.ts import { ofType } from 'redux-observable'; import { mapTo, tap, delay,…
olegshilov
  • 154
  • 7
1
vote
2 answers

Handle back-pressure in FixedThreadPool

How to deal with back-pressure in Java using thread pool? How to reject new tasks so there are no more than N submitted tasks. N - is the maximum number of allowed tasks in submission queue, which include new, running, paused (not finished)…
Yan Khonski
  • 12,225
  • 15
  • 76
  • 114
1
vote
1 answer

How do I configure backpressure in Spring WebFlux?

I'm trying to understand how to apply backpressure in Spring WebFlux. I understand the theory of backpressure, but I can't reproduce it, so I don't fully understand it. Let's take the following example: public void test() throws InterruptedException…
Fenrir
  • 11
  • 3
1
vote
2 answers

Since Publisher and Subscriber reside in same app limited by same resources, how does backpressure help in the load issue on either side?

I've been trying to build an app using the Java Flow API. While the idea of being able to perform backpressure between the publisher and subscriber in an event when their speeds are different, I'm not sure how it really helps since both the…
Carven
  • 14,988
  • 29
  • 118
  • 161
1
vote
1 answer

Backpressure between Lagom services using streaming ServiceCall not working

Lagom: 1.5.4 Consider having a ServiceCall (example): def stream: ServiceCall[NotUsed, Source[Int, NotUsed]] = ServiceCall { _ => Future.successful( Source(1.to(1000)).wireTap(msg => log.info(s"sending $msg")) ) } When another service…
antex
  • 794
  • 7
  • 8
1
vote
2 answers

Is there any automatic version of flow control in Grpc-java?

My server can handle 10000qps, but when it comes 11000qps, it's overloading. That will cause a out of memory eventually. I'm wondering how grpc-java deal with server overloading. I've searched for a long time, no configurable version found…
Brutal_JL
  • 2,839
  • 2
  • 21
  • 27
1
vote
2 answers

Back Pressure issue in rx.net

I wanted to read lines from the CSV File and use RX.Net to do some transformation and I wanted to do batch update and send the update every 250 milliseconds public static IEnumerable ReadCSV(string filePath) { var reader = new…
Will
  • 155
  • 7
1 2 3
…
10 11