While reading the messages from the pubsub with flink code I am not able to print them on the console. Where do I find the read messages. Please find the code base as well as the output on the command line interface
public class ReadFromPubsub {
public static void main(String args[]) throws Exception
{
System.out.println("Flink Pubsub Code Read 1");
StreamExecutionEnvironment streamExecEnv = StreamExecutionEnvironment.getExecutionEnvironment();
streamExecEnv.setStateBackend(new RocksDBStateBackend("file:///tmp/checkpoints"));
System.out.println("Flink Pubsub Code Read 2");
DeserializationSchema<String> deserializer = new SimpleStringSchema();
System.out.println("Flink Pubsub Code Read 3");
SourceFunction<String> pubsubSource = PubSubSource.newBuilder()
.withDeserializationSchema(deserializer)
.withProjectName("vz-it-np-gudv-dev-vzntdo-0")
.withSubscriptionName("subscription1")
.build();
System.out.println("Flink Pubsub Code Read 4"+pubsubSource);
streamExecEnv.addSource(pubsubSource).print();
//DataStreamSource<String> ds = streamExecEnv.addSource(pubsubSource);
//ds.print();
//streamExecEnv.addSource(pubsubSource).broadcast();
streamExecEnv.enableCheckpointing(10);
System.out.println("Flink Pubsub Code Read 5");
streamExecEnv.execute();
}
}
While executing the jar file on the dataproc cluster
flink run PubsubRead.jar
I am getting the below content in the command line
2023-02-17 08:48:40,396 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli - Found Yarn properties file under /tmp/.yarn-properties-root.
2023-02-17 08:48:40,732 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli - YARN properties set default parallelism to 1
2023-02-17 08:48:40,732 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli - YARN properties set default parallelism to 1
YARN properties set default parallelism to 1
2023-02-17 08:48:40,815 INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at vz-it-np-gudv-dev-vzntdo-dp-lr-flink-m/63.21.112.14:8032
2023-02-17 08:48:40,889 INFO org.apache.hadoop.yarn.client.AHSProxy - Connecting to Application History server at vz-it-np-gudv-dev-vzntdo-dp-lr-flink-m/63.21.112.14:10200
2023-02-17 08:48:40,894 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli - No path for the flink jar passed. Using the location of class org.apache.flink.yarn.YarnClusterDescriptor to locate the jar
2023-02-17 08:48:40,894 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli - No path for the flink jar passed. Using the location of class org.apache.flink.yarn.YarnClusterDescriptor to locate the jar
2023-02-17 08:48:40,973 INFO org.apache.flink.yarn.AbstractYarnClusterDescriptor - Found application JobManager host name 'vz-it-np-gudv-dev-vzntdo-dp-lr-flink-w-3.us-east4-c.c.vz-it-np-gudv-dev-vzntdo-0.internal' and port '44617' from supplied application id 'application_1676354352639_0001'
Starting execution of program
Flink Pubsub Code Read 1
Flink Pubsub Code Read 2
Flink Pubsub Code Read 3
Flink Pubsub Code Read 4org.apache.flink.streaming.connectors.gcp.pubsub.PubSubSource@4da855dd
Flink Pubsub Code Read 5
I am able to see the last print statement in the code where as I am not able see the read messages. Is the connection to the pubsub is establishing or not.
