0

My question revolves around kicking off a API call to get the file prefix for TextIO output.

Here is what I have now (and works):

results.get(validRecords).setCoder(StringUtf8Coder.of()).apply(Window.<String>into(Sessions.withGapDuration(Duration.standardMinutes(1)))).apply("WriteValidRecords",TextIO.write().to(cloudStorageValidfile).withSuffix(".csv").withWindowedWrites().withNumShards(1));

This is what I changed it too:

results.get(validRecords).setCoder(StringUtf8Coder.of()).apply(Window.<String>into(Sessions.withGapDuration(Duration.standardSeconds(60)))).apply("WriteRecords", TextIO.write().to(API-CALL).withWindowedWrites().withSuffix("csv").withNumShards(1));

It seems with the API call, it only gets called once on instantiation, then continually uses the same prefix. Is there a way to force a call each time?

Brian Tompsett - 汤莱恩
  • 5,753
  • 72
  • 57
  • 129
MTS57017
  • 35
  • 1
  • 8

1 Answers1

0

I figured this out, if you want to make an API call, that is fine upon instantiation. After that, you cannot, as it gets deployed in the state in which it compiled and launched. So if the API call is fine there, but it will get the value only once and swap the call for the value that was retrieved and then launch. There are other areas where you can do an API call with each call (Serializable Function should work), but here it will not.

MTS57017
  • 35
  • 1
  • 8