1

Sorry for the simple question, but I am struggling to understand how to find out whether result of a given query has watermark or not.

For example, I define my source in Datastream api and then convert it to Table API leveraging SOURCE_WATERMARK() feature

using input.printSchema() I clearly see the Watermark Strategy

(
  `id` BIGINT NOT NULL,
  `amount` DECIMAL(38, 18),
  `rowtime` TIMESTAMP_LTZ(3) *ROWTIME* METADATA,
  WATERMARK FOR `rowtime`: TIMESTAMP_LTZ(3) AS SOURCE_WATERMARK(),
  CONSTRAINT `PK_id` PRIMARY KEY (`id`) NOT ENFORCED
)

then, I get the schema on a simple select

input2 = input.select('id, 'rowtime)
input2.printSchema()

and the result is

(
  `id` BIGINT NOT NULL,
  `rowtime` TIMESTAMP_LTZ(3) *ROWTIME*
)
  1. Does this mean input2 table dont have watermark?

  2. When I do a regular join (such as self-join), the result table schema look similar, with just rowtime. I noticed watermark in the flink UI as well but I came across this other note by David Anderson here that

The result of a regular join cannot have a well-defined watermark strategy

Would appreciate any help to understand these concepts better.

tootso
  • 11
  • 2

0 Answers0