Why doesn't DoFnTester support outputs in finishBundle() any more?

Question

We have some functions that process the elements from a bundle in batches. They need to output the last batch in the finishBundle() method.

Something like described in the answer to this previous question.

In the latest Dataflow 2.0.0-beta1 release, which is based on Beam 0.4.0, the DoFnTester throws UnsupportedOperationException from its finishBundle() method if the function under test invokes the output method on the context there.

Is there any reason for this behavior change? Is output from DoFn.finishBundle() not supported anymore during runtime?

score 0 · Accepted Answer · answered Jan 19 '17 at 18:50

0

This is a regression in DoFnTester. I've filed a bug against Beam here

answered Jan 19 '17 at 18:50

danielm

3,000
10
15

Thank you for the help, Daniel. – M Song Jan 19 '17 at 19:05
In fact, it has never worked properly except for globally windowed test data. – Kenn Knowles Jan 19 '17 at 22:00

Why doesn't DoFnTester support outputs in finishBundle() any more?

1 Answers1