0

I am trying to import data using cloudfiles in databricks. 2 different ways give me errors. With

CREATE OR REFRESH STREAMING LIVE TABLE orders_raw
COMMENT "The raw books orders, ingested from orders-raw"
AS SELECT * FROM cloud_files("${datasets_path}/orders-raw", "parquet",
                             map("schema", "order_id STRING, order_timestamp LONG customer_id STRING, quantity LONG")
                             )

I get:

[PARSE_SYNTAX_ERROR] Syntax error at or near 'STRING'.(line 1, pos 9)

== SQL ==
order_id STRING, order_timestamp LONG customer_id STRING, quantity LONG
---------^^^

And with :

CREATE OR REFRESH STREAMING LIVE TABLE orders_raw
COMMENT "The raw books orders, ingested from orders-raw"
AS SELECT * FROM cloud_files("${datasets_path}/orders-json-raw", "json",
                             map("cloudFiles.inferColumnTypes", "true")
                             )

The error is:

Caused by: java.lang.IllegalArgumentException: cloudFiles.schemaLocation Could not find required option: schemaLocation. Please provide a schema location using cloudFiles.schemaLocation for storing inferred schema and supporting schema evolution.

any help? thanks

Alex Ott
  • 80,552
  • 8
  • 87
  • 132
gaut
  • 5,771
  • 1
  • 14
  • 45

0 Answers0