Stack Overflow
Stack Overflow

Questions tagged [backpressure]

Backpressure refers to the build-up of data at an I/O switch when buffers are full and not able to receive additional data. No additional data packets are transferred until the bottleneck of data has been eliminated or the buffer has been emptied.

Backpressure refers to the build-up of data at an I/O switch when buffers are full and not able to receive additional data. No additional data packets are transferred until the bottleneck of data has been eliminated or the buffer has been emptied.

Source - Techopedia

164 questions
0
votes
1 answer

Implementing a cold source with Mutiny

I'd like to know, what is a proper way to implement my own cold source (publisher) using the Mutiny library. Let's say there is huge file parser that should return lines as Multi items according to the Subscriber's consumption rate. New…
Ultranium
  • 332
  • 2
  • 19
0
votes
1 answer

akka-stream pipeline backpressures despite inserted buffer

I have an akka-stream pipeline that fans out events (via BroadcastHub) that are pushed into the stream via a SourceQueueWithComplete. Despite all downstream consumers having a .buffer() inserted (of which I'd expect that it ensures that upstream…
MartinHH
  • 982
  • 4
  • 7
0
votes
1 answer

POST InputStream with RestTemplate and handling backpresure

We have 3 tier System (UI,Server1,Server2) We have to transfer files from our client (Angular) to some server (java server1, Spring) and this server has to pass the file data to the second server UI => Server 1 ==> Server 2 We want to stream the…
Maayan Hope
  • 1,482
  • 17
  • 32
0
votes
1 answer

Does Node stream.pipeline also provides back-pressuring prevention?

The answer to this question is perhaps one word. The documentation here is kind of confusing: in the beginning it introduces the stream.pipeline() method as even a more preferable version of .pipe method, but later on it focuses solely on the .pipe…
Payam V
  • 130
  • 7
0
votes
0 answers

Flux Reactive Microservice - one microservice signal other microservice about backpressure

Two Springboot microservices need to interact with each other to achieve mass emailing feature. Microservice 1: CollectorService collects information about clients and Microservice 2: NotificationService to send documents via SMTP service.…
Vijay Borelli
  • 1
  • 2
0
votes
0 answers

Java stream with queue

I'm searching if it's possible to use Java stream with queue/backpressure. Something like: iterator -> (queue?) -> parallelstream -> map -> collector Mapping is slower than iterator (even if multithreaded), so iterator should be read a slower pace…
Marx
  • 804
  • 10
  • 23
0
votes
1 answer

Webflux drops when overflow strategy is buffer

When I call subscribe() calls outside the reactive chain. Does it not follow the spring webflux's overflow strategy? fooRepository.findOne() .doOnNext { barRepository.save(bar).subscribe() } .flatMap { ... } I have this strange behavior…
Kyoo Sik Lee
  • 363
  • 4
  • 19
0
votes
1 answer

What is Netty closing channel policy if client does read data and application keeps writing?

My application uses Netty 4.x with TCP sockets to distribute the same data to multiple clients. I want my application to tolerate some clients stalls in data receiving, but if client is too slow I want my application to abandon (close) the…
digital_infinity
  • 534
  • 6
  • 20
0
votes
1 answer

How to download incoming file and prevent Backpressure while sending files through WebRTC data channels using streams?

I'm building a file sharing application with WebRTC and Node.js. It is a command line application so there will be no HTML invloved. I'm reading the file as a stream and sending it, then at reciever's side I'll download the file. Here's how I'll be…
Akash Asthana
  • 67
  • 7
0
votes
1 answer

What is the standard way to handle exception of OOM in Apache Flink

I'm learning how to process streaming data with Flink. I've succeeded in coding an example, which is to receive and deserialize streaming data from a data source, to transform it and print the output. Now I'm thinking how to process the exception of…
Yves
  • 11,597
  • 17
  • 83
  • 180
0
votes
1 answer

When does AkkaHttp backpressure kick in?

.. when the http response entity is not consumed, or the client tcp buffer becomes full, or when the rate of client taking from its tcp buffer is lower then the rate of server pushing data to it? I am looking for a way for to achieve the…
NicuMarasoiu
  • 776
  • 9
  • 25
0
votes
0 answers

nodejs - Intermittent TCP client for a TCP server

I want to build a TCP server (say, tcpserver.js) created with net.createServer(). I have a TCP client (say, eventSource.js) that writes to this server at a high frequency e.g. 100s per sec. I have a TCP client eventSink.js that will connect to this…
cogitoergosum
  • 2,309
  • 4
  • 38
  • 62
0
votes
2 answers

Understanding the capacity param in RxJava onBackpressureBuffer

Here is a small sample app that I wrote: package ru.maksim.sample.app import android.os.Bundle import android.util.Log import androidx.appcompat.app.AppCompatActivity import io.reactivex.rxjava3.core.BackpressureOverflowStrategy import…
Maksim Dmitriev
  • 5,985
  • 12
  • 73
  • 138
0
votes
0 answers

Flink Cassandra sink is causing backpressure

I'm running a Flink job which takes msgs from Kafka and eventually writes them into a Cassandra sink. I'm ingesting around 500 msgs/s, which are flat mapped into ~60,000 Cassandra inserts. The job parallelism is 5 (reading from 5 Kafka…
yaarix
  • 490
  • 7
  • 18
0
votes
1 answer

How do programs apply backpressure over a network?

Consider the example of a download stream that can be throttled (eg. torrent client, dropbox sync, etc). How does a program apply backpressure to the network? My thoughts are that, from a software perspective you can choose to read from a socket at…
Kevin Cianfarini
  • 649
  • 1
  • 5
  • 11
1 2 3
…
10 11