0

I am copying an existing table into a different environment using its DDL. Some of the columns are type 'NULL.' Right now the values for said column are NULL in the sample I am given, but they probably shouldnt be according to their names (will probably be dates, IDs, bool). This is using databricks delta tables. It will be an automatic ingestion so I wont be able to update columns later.

Is this going to create issues if there is non null data in the future being ingested into this table? If so, how can I avoid this ingestion issue?

Example:

show create table oldEnv.tableName


create table oldEnv.tableName(
col1 decimal(5,0),
col2 decimal(5,0),
col3 timestamp,
col4 string,
...
colX_flag,
colY_update_date,
colZ_ID
) using delta location db.location/path

Can I just keep them as null and let databricks handle it when values come up?

Alex Ott
  • 80,552
  • 8
  • 87
  • 132
Raie
  • 61
  • 6
  • I've never heard of NULL data type. NULL is the state a field can have but AFAIK it is not a data type. Fields can be NULL unless field property is set to not allow (data required). – June7 Jan 13 '23 at 20:16
  • 1
    There is [NULL type](https://docs.databricks.com/sql/language-manual/data-types/null-type.html) in Databricks. It can contain only `NULL` values, it will not be possible to insert a non-null value. But what stops you from just explicitly declaring column types? (`create table ... colX_flag BOOLEAN, colY_update_date DATE, colZ_ID INT ...`) – Kombajn zbożowy Jan 13 '23 at 21:43
  • Its a table not originally made by me-I dont know what the use of these columns will be. Right now they are just filled with nulls and the column type is declared as NULL. I am ingesting the whole table into a different workspace/environment but eventually after I promote it, it will update with new values from wherever its getting these from each day. – Raie Jan 13 '23 at 21:48

0 Answers0