RxScala buffer elements

Question

I have application where I need parse strings and I use for it RxScala. My code:

import java.util.concurrent.TimeUnit
import rx.lang.scala.Subject
import rx.lang.scala.schedulers.NewThreadScheduler
import rx.lang.scala.subjects.{SerializedSubject, PublishSubject}
import scala.concurrent.duration.Duration

object RxScala extends App {
  val subject: Subject[String] = SerializedSubject(PublishSubject())

  val processLines = (lines: Seq[String]) => {
    // long action
  }

  subject
    .subscribeOn(NewThreadScheduler())
    .tumblingBuffer(Duration(2, TimeUnit.SECONDS), 100)
    .subscribe(processLines)

  for(i <- 1 to 100000) {
    subject.onNext("Line " + i)
  }
}

I have problem because I add lines faster then I can process them.

I want create buffer e.g. 200 lines and if buffer is full ignore new records to moment when buffer isn't full, e.g.

Add 100 records (A)
Add 100 records (B)
Program start processLines (A) // buffer have (B) elements
Add 100 records (C) // buffer have (B, C) elements and it is full
Add 100 records (D) // elements are ignored
ProcessLines is finished 
Program start processLines (B) // buffer have (C) elements
Add 100 records (E) // buffer have (C, E) elements

Does RxScala have method to do this?

Have you tried out `onBackpressureDrop`? – Samuel Gruetter Mar 31 '15 at 09:15 — Samuel Gruetter, Mar 31 '15 at 09:15

score 0 · Answer 1 · answered Jul 16 '15 at 12:40

subject.
    tumblingBuffer(Duration(2, TimeUnit.SECONDS), 100).
    onBackpressureDrop.
    observeOn(NewThreadScheduler()).
    subscribe(processLines)

http://reactivex.io/documentation/operators/backpressure.html

onBackpressureDrop drops emissions from the source Observable unless there is a pending request from a downstream Subscriber, in which case it will emit enough items to fulfill the request

RxScala buffer elements

1 Answers1