-1

I have this code from Google Cloud Platform Dataflow Templates.

I wish to add more functionalities to it, namely, I wish to add support for JavaScript UDF. When I try to compile the file, using this:

mvn compile exec:java \
-Dexec.mainClass=com.google.cloud.teleport.templates.${PIPELINE_NAME} \
-Dexec.cleanupDaemonThreads=false \
-Dexec.args=" \
--project=${PROJECT_ID} \
--stagingLocation=gs://${PROJECT_ID}/dataflow/${PIPELINE_FOLDER}/staging \
--tempLocation=gs://${PROJECT_ID}/dataflow/${PIPELINE_FOLDER}/temp \
--runner=DataflowRunner \
--windowDuration=2m \
--numShards=1 \
--topic=projects/${PROJECT_ID}/topics/windowed-files \
--outputDirectory=gs://${PROJECT_ID}/temp/ \
--outputFilenamePrefix=windowed-file \
--outputFilenameSuffix=.txt"

When compiling the file, I get the following error:

An exception occured while executing the Java class. Class interface com.google.cloud.teleport.templates.PubsubToText$Options missing a property named 'topic'. -> [Help 1] org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:java (default-cli) on project google-cloud-teleport-java: An exception occured while executing the Java class. Class interface com.google.cloud.teleport.templates.PubsubToText$Options missing a property named 'topic'.

Even though, I've passed the --topic flag with appropriate values plugged in.

GhostCat
  • 137,827
  • 25
  • 176
  • 248
tHappy
  • 91
  • 8
  • Other users marked your question for low quality and need for improvement. I re-worded/formatted your input to make it easier to read/understand. Please review my changes to ensure they reflect your intentions. But I think your question is still not answerable. **You** should [edit] your question now, to add missing details (see [mcve] ). Feel free to drop me a comment in case you have further questions or feedback for me. – GhostCat Oct 16 '18 at 07:03
  • Linking to source code is not appreciated. Please create a **minimal** example (not necessarily your full project!) and add that code to the question. – GhostCat Oct 16 '18 at 07:04

2 Answers2

2

The example at the top is wrong. You have to pass --inputTopic instead of --topic. You can see this in the code where the ValueProvider is defined:

@Description("The Cloud Pub/Sub topic to read from.")
@Required
ValueProvider<String> getInputTopic();
void setInputTopic(ValueProvider<String> value);

You can also run the template from the Console UI and the job details will show that the option is indeed inputTopic:

enter image description here

Guillem Xercavins
  • 6,938
  • 1
  • 16
  • 35
  • Even when I pass it the `--inputTopic` flag or any other variation (`--InputTopic`, `inputtopic`), it still shows the same error. Strangely, each time it shows that the same property is missing, i.e, if I pass `--inputTopic`, `[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:java (default-cli) on project google-cloud-teleport-java: An exception occured while executing the Java class. Class interface com.google.cloud.teleport.templates.PubsubToText$Options missing a property named 'inputTopic'. -> [Help 1]` when passing `--InputTopic` it shows missing property InputTopic – tHappy Oct 16 '18 at 10:43
  • Well that's not strange at all, it just means that whatever option you are passing is not present in the template. Are you running an older version or did you modify the code? If it's the latter, it could be that you modified the option or that the extended options are being ignored – Guillem Xercavins Oct 16 '18 at 10:57
1

The invocation example in the javadoc should now reflect the correct input parameter (--inputTopic) here.

Sameer Abhyankar
  • 261
  • 1
  • 2