0

I want a sink operator to be the downstream for multiple upstream process function operators, however, I want the sink operator to be conscious of which upstream operator the data is coming from so it can add a special tag.

I don't want to include an extra field to indicate operator name in the upstream operators since that would incur a lot of extra repeated IO which doesn't seem optimal. Is there a way where the downstream sink can more directly access to check which upstream operator a certain event was flowing from?

Baiqing
  • 1,223
  • 2
  • 9
  • 21

1 Answers1

1

There's no way I know of, without adding some tagging field.

But please, please, please measure then optimize. Do the simplest thing that might work, then spend time on the straightforward optimizations, and finally start worrying about helicopter stunts to squeeze the last bits of performance out of Flink.

kkrugler
  • 8,145
  • 6
  • 24
  • 18