0

Is there any simple way that I can redirect the output of GroupBy into multiple output files based on Group keys?

Bin.apply(GroupByKey.<String, KV<Long,Iterable<TableRow>>>create())
.apply(ParDo.named("Print Bins").of( ... ) 
.apply(TextIO.Write.to(*Output file based on key*))

If Sink is the solution, would you please share a sample code w/ me?

Thanks!

jkff
  • 17,623
  • 5
  • 53
  • 85
AmirCS
  • 321
  • 1
  • 2
  • 14

1 Answers1

0

Beam 2.2 will include an API to do just that - TextIO.write().to(DynamicDestinations), see source. For now, if you'd like to use this API, you can use the 2.2.0-SNAPSHOT version. Note that this API is experimental and might change in Beam 2.3 or onwards.

jkff
  • 17,623
  • 5
  • 53
  • 85