0

I'm trying to do a simple join using Beam SQL but I'm getting an exception while compilation:

Exception in thread "main" java.lang.ClassCastException: org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.rex.RexCall cannot be cast to org.apache.beam.repackaged.beam_sdks_java_extensions_sql.org.apache.calcite.rex.RexInputRef

The join is something like:

select T1.x from table1 T1 join table2 T2
on 
(case when T1.a = 'ABC' then 'ABC' else T1.b end = T2.c)

This condition works fine when executed in BigQuery (tried as a sanity check). Not sure why it's breaking in Beam SQL. I even tried using Beam SQL UDF but it did not help either. I assume it's because of Apache Calcite and the format it follows but I do not know how to handle it.

Can someone please help with this.

rish0097
  • 1,024
  • 2
  • 18
  • 39
  • I don't believe something like this is supported in Beam SQL at the moment. Only field equality joins are expected to work. `CASE` should work in `SELECT` though, so you might be able to model this as `JOIN` between a table and a select, similar to this: https://github.com/apache/beam/blob/master/sdks/java/testing/nexmark/src/main/java/org/apache/beam/sdk/nexmark/queries/sql/SqlQuery7.java#L60 – Anton Feb 12 '19 at 19:29
  • There is a JIRA related to this exception: https://jira.apache.org/jira/browse/BEAM-6112. Contribution will definitely very welcomed to it. – Rui Wang Feb 13 '19 at 03:01

1 Answers1

0

Interesting. Can you try the IF conditioning? I hope this will work:

select T1.x from table T1 join table T2 on IF(T1.a = 'ABC', 'ABC',T1.b) = T2.c
khan
  • 7,005
  • 15
  • 48
  • 70