0

my flink job pretends to fail at sink operator, which sends data to mqtt.

I have tried .setBufferTimeout(0) at the one before sink operator and increase parallelism, as you can see in the pic. But it doesn't work. Can anyone help? Thanks.

enter image description here

hardillb
  • 54,545
  • 11
  • 67
  • 105
DaydreamQ
  • 1
  • 1

1 Answers1

0
  1. According to flink document

A buffer timeout of 0 should be avoided, because it can cause severe performance degradation.

  1. What sink connector do you use?

  2. What's the batch size of your sink operator?

All these factors will have a relationship with the write performance of your sink.

Juiceyang
  • 21
  • 2
  • 1. Yes, I noticed that, but I hope that once the data processing is finished, it can be immediately sent downstream, with less latency; 2. I use mqtt as sink connector; 3. No batch size. As I mentioned, once a event is processed, it would be sent immediately. – DaydreamQ Aug 08 '23 at 08:05