0

It was ok to use >= operator in Cloudera Hive sql, After migration to Databricks I am not able to use this operator in Partition Parentheses

ALTER TABLE tmprry.tmp_table DROP IF EXISTS PARTITION (log_date >= $LOG_DATE_EXECUTE_TWO_DAYS_AGO AND log_date <= $LOGDATE ) PURGE;

On the notebook it gives me error:

com.databricks.backend.common.rpc.DatabricksExceptions$SQLExecutionException: org.apache.spark.sql.catalyst.parser.ParseException: mismatched input '>=' expecting {')', ','}(line 2, pos 20)

== SQL == ALTER TABLE tmprry.tmp_table DROP IF EXISTS PARTITION (log_date >= 20220130 ) PURGE ---------------------------------------------------------------------------------------------------------^^^

bozok
  • 13
  • 4

1 Answers1

0

This supposedly happens because the partition column is a string and we are using comparison operators.

You can find the answer here

Karthikeyan Rasipalay Durairaj
  • 1,920
  • 13
  • 35
  • thank you for your answer, however partition column is INT : """SHOW CREATE TABLE `tmprry`.`tmp_table ` (.... `country_id` INT,`log_date` INT, `hour` INT) USING delta PARTITIONED BY (log_date) """ Might i have missed something? – bozok Feb 01 '22 at 22:59