Stack Overflow
Stack Overflow

Questions tagged [backpressure]

Backpressure refers to the build-up of data at an I/O switch when buffers are full and not able to receive additional data. No additional data packets are transferred until the bottleneck of data has been eliminated or the buffer has been emptied.

Backpressure refers to the build-up of data at an I/O switch when buffers are full and not able to receive additional data. No additional data packets are transferred until the bottleneck of data has been eliminated or the buffer has been emptied.

Source - Techopedia

164 questions
2
votes
0 answers

How to set up Spring Controller to handle a backpressured request

Spring MVC controller is to be used as a proxy for another service to serving possibly large files. To avoid storing the whole WebClient response from another service in memory at any given time I wanted to use reactive properties of Spring 5 +…
user3696644
2
votes
1 answer

How to create an Akka flow with backpressure and Control

I need to create a function with the following Interface: import akka.kafka.scaladsl.Consumer.Control object ItemConversionFlow { def build(config: StreamConfig): Flow[Item, OtherItem, Control] = { // Implementation goes here } My problem is…
Stoecki
  • 585
  • 1
  • 3
  • 16
2
votes
0 answers

rxjs mongo stream backpressure

Lately I've been having backpressure-problems with rx.js from a mongo-cursor-readable. The task is basically: Query DB A (Mongo), use its readable and convert it to an observable Perform various async transformations to the input stream, e.g. 2.1…
Figedi
  • 383
  • 1
  • 2
  • 11
2
votes
1 answer

how to run the EC2 snitch configuration in cassandra

I am trying to accomplish a single cassandra node start up in Ec2 snitch configuration. The yaml properties I have changed for this configuration is: seeds : 'ip-address of machine' listen: 'ip_address of machine' rpc_address:'ip_address of…
Chakri Stark
  • 166
  • 1
  • 1
  • 12
2
votes
1 answer

Back-pressure for Akka Sharding with persistent actors

Is there a Akka stream back-pressure model or something for using akka shardings with persistent actor? Because I have a problem, I have a akka sharding cluster with persistent actors. (using cassandra as journal plugin). Sometimes a lot of…
hich9n
  • 1,578
  • 2
  • 15
  • 32
2
votes
1 answer

RxJava 2: Stats about messages dropped because of BackPressure?

I am using RxJava 2 Flowable with DROP BackPressure strategy. Is there any way to collect information / stats about how many messages are actually dropped because of the BackPressure? Solution Flowable#onBackpressureDrop(consumer ->…
Thomas
  • 1,053
  • 2
  • 11
  • 20
2
votes
2 answers

RxJava: How to implement a slow consumer with buffer and a fast consumer to the same source?

While playing with various backpressure scenarios, I implemented a case where one subscriber is slow with a buffer, while another one consumes whatever is thrown to it. That was using Scala and Akka Streams. You can see the code here if you want to…
Abhijit Sarkar
  • 21,927
  • 20
  • 110
  • 219
2
votes
1 answer

Handling backpressure without dropping items or serializing in RxJava

In short, is there any solution to resolve backpressure in RxJava without resorting to dropping items, serializing operations or unbounded buffering? Consider the following task as an example of when this could be useful. Reading data from disk…
Edward Peek
  • 163
  • 1
  • 5
2
votes
1 answer

Message Bus versus Quasar/HTTP for internal Microservice Calls

I am looking to optimize a microservice architecture that currently uses HTTP/REST for internal node-to-node communication. One option is implementing backpressure capability into the services, (eg) by integrating something like Quasar into the…
Robert Christian
  • 18,218
  • 20
  • 74
  • 89
2
votes
0 answers

What is the Scala way to process a file line by line in parallel, with back pressure?

The following code will read a file line by line, create a task for each line and then queue it up to the executor. If the executor's queue is full, the reading from the file stops until there is space again. I looked at a few suggestions in SO, but…
BasilTomato
  • 1,071
  • 1
  • 8
  • 14
2
votes
0 answers

flow control in a storm topology using netty transport layer

I have created a small unreliable topology consisting of one spout that reads lines from a file containing among others lat/long coordinateds and one of the downstream bolts calls an external reverse geocoding service to determine the country.…
Vaggelis Katsaros
  • 21
  • 1
1
vote
1 answer

Rate Control and Back Pressure in gRPC in Cpp

How can I implement rate control and back pressure in gRPC in cpp? I went through the documentation and found that there's something called a flow control limit that can help in signifying if the window size is full but I cannot find how to set…
Naman Shah
  • 35
  • 4
1
vote
1 answer

Eclipse Ditto does not send all things events over target connection

We are integrating Eclipse Ditto into a digital twin platform, but we have encountered a problem while testing and we don't really know how to fix it. As a context, the goal is to receive in 593 twins (Ditto Thing) the result of a simulation. The…
Julia Robles
  • 37
  • 3
1
vote
0 answers

Project Reactor sink unexpected behavior with backpressure buffer

I have a multicast sink with backpressure buffer with the default size. To this sink I emit events asynchronously every 25ms from a dedicated thread. There are 3 subscribers A, B and C where B pretends to do some work through thread sleep. A and C…
Tomask
  • 2,344
  • 3
  • 27
  • 37
1
vote
1 answer

Reactor backpressure - request multiple in hookOnNext

I've been testing some of the Reactor backpressure stuff, and one common structure seems to be: package com.example.backpressure; import org.junit.jupiter.api.Test; import org.reactivestreams.Subscription; import…
Ziggy000
  • 183
  • 1
  • 8
1 2 3
…
10 11