I have a large table with 830 columns and need to copy certain rows from one table to another, including all columns, and adding an additional string column in the destination. The destination table has identical columns to the source table but with one extra string column. The source is also partitioned by a date string of the form '2022-07-20' (YYYY-MM-DD).
The source table previously didn't have array structures so I was able to copy row(s) by doing:
INSERT INTO destination_table SELECT *, 'some string' FROM source_table WHERE <some criteria>
A new schema has been deployed so that the source table now has four array structures in it (in various places, not on the end). Impala doesn't copy these when SELECT *
is used. This SO answer suggests I can just specify the arrays in the FROM clause. I tried that but it unpacks the array fields and outputs them at the end of all the other columns (i.e. not in their correct place in the schema, for the purposes of the insert).
Is there a way to achieve this easily without specifying all the 830 columns twice in my query (once in the column permutation/list and then again in the SELECT clause) ?
I cannot change the source table but I can configure, drop or change the destination table if required.
Thanks