Flink SQL - How to parse a TIMESTAMP with custom pattern?

Question

From documentation it looks like Flink's SQL can only parse timestamps in a certain format, namely:

TIMESTAMP string: Parses a timestamp string in the form "yy-mm-dd hh:mm:ss.fff" to a SQL timestamp.

Is there any way to pass in a custom DateTimeFormatter to parse a different kind of timestamp format?

score 3 · Accepted Answer · answered May 06 '18 at 09:26

3

You can implement any parsing logic using a user-defined scalar function (UDF).

This would look in Scala as follows.

class TsParser extends ScalarFunction {
  def eval(s: String): Timestamp = {
    // your logic
  }
}

Once defined the function has to be registered at the TableEnvironment:

tableEnv.registerFunction("tsParser", new TsParser())

Now you can use the function tsParser just like any built-in function.

See the documentation for details.

answered May 06 '18 at 09:26

Fabian Hueske

Thank you! I also realized that Flink does support many different timestamp formats but those are not listed in the documentation, but is printed out in the error message. Was able to change the source date format to one of the supported format in my case. – Rakesh Chalasani May 07 '18 at 13:28

1 Answers1