Exception in Flink TableAPI

Question

I am trying to run below simple Flink job to count words using TableAPI. Used DataStream API to read a stream of data and used StreamTableEnvironment API to create a Table environment. I am getting below exception. Can someone please help me what is wrong in code? I am using Flink 1.8 version.

Exception:

**Exception in thread "main" org.apache.flink.table.api.TableException: Only the first field can reference an atomic type.**
    at org.apache.flink.table.api.TableEnvironment$$anonfun$5.apply(TableEnvironment.scala:1117)
    at org.apache.flink.table.api.TableEnvironment$$anonfun$5.apply(TableEnvironment.scala:1112)
    at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
    at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
    at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
    at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
    at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
    at scala.collection.mutable.ArrayOps$ofRef.flatMap(ArrayOps.scala:186)
    at org.apache.flink.table.api.TableEnvironment.getFieldInfo(TableEnvironment.scala:1112)
    at org.apache.flink.table.api.StreamTableEnvironment.registerDataStreamInternal(StreamTableEnvironment.scala:546)
    at org.apache.flink.table.api.java.StreamTableEnvironment.fromDataStream(StreamTableEnvironment.scala:91)
    at Udemy_Course.TableAPIExample.CountWordExample.main(CountWordExample.java:30)

Code

public class CountWordExample {

    public static void main(String[] args) throws Exception {
        StreamExecutionEnvironment environment=StreamExecutionEnvironment.getExecutionEnvironment();
        StreamTableEnvironment streamTableEnvironment=StreamTableEnvironment.create(environment);


        DataStream<WC> streamOfWords =
                environment.fromElements(
                        new WC("Hello",1L),
                        new WC("Howdy",1L),
                        new WC("Hello",1L),
                        new WC("Hello",1L));

        Table t1 = streamTableEnvironment.fromDataStream(streamOfWords, "word, count");

        Table result = streamTableEnvironment.sqlQuery("select word, count(word) as wordcount 
        from " + t1 + " group by word");


        streamTableEnvironment.toRetractStream(result, CountWordExample.class ).print();

        environment.execute();
    }

    public static class WC {

        private String word;
        private Long count;

        public WC() {}

        public WC(String word,Long count) {
            this.word=word;
            this.count=count;
        }
    }
}

David Anderson · Answer 1 · 2020-09-02T09:06:19.077

All that's needed to get this running is to change the private word and count fields in WC so that they are public, and to use Row.class rather than CountWordExample.class in toRetractStream.

Those fields either have to be public, or have publicly accessible getters and setters in order for the SQL engine to work with them.

public class CountWordExample {

    public static void main(String[] args) throws Exception {
        StreamExecutionEnvironment environment=StreamExecutionEnvironment.getExecutionEnvironment();
        StreamTableEnvironment streamTableEnvironment=StreamTableEnvironment.create(environment);
        
        DataStream<WC> streamOfWords =
                environment.fromElements(
                        new WC("Hello",1L),
                        new WC("Howdy",1L),
                        new WC("Hello",1L),
                        new WC("Hello",1L));

        Table t1 = streamTableEnvironment.fromDataStream(streamOfWords, "word, count");

        Table result = streamTableEnvironment.sqlQuery("select word, count(word) as wordcount from " + t1 + " group by word");

        streamTableEnvironment.toRetractStream(result, Row.class).print();

        environment.execute();
    }

    public static class WC {

        public String word;
        public Long count;

        public WC() {}

        public WC(String word,Long count) {
            this.word=word;
            this.count=count;
        }
    }
}

Thank you, it worked. When I executed the above program, I got the output as below. (true,Hello,1) 3> (true,Howdy,1) 4> (false,Hello,1) 4> (true,Hello,2) 4> (false,Hello,2) 4> (true,Hello,3) In each output, I am seeing true or false. I didn't understand why it is coming. Also, as you suggested to use Row.class rather than CountWordExample.class. Can you please explain why? Thanks! — Shailendra, Sep 02 '20 at 15:59

Exception in Flink TableAPI

1 Answers1