1

I'm working on a client platform. It is a DataLake linked to AWS S3 and AWS ATHENA

I've uploaded a Dataset to an S3 Bucket using AWS GLUE. The job ran successfully and a table was created under ATHENA. When I try to "Preview" the content of the table, I get the following error :

HIVE_INVALID_METADATA: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe: columns has 43 elements while columns.types has 34 elements

P.S: The file i'm uploading contains 34 columns

Ben Yedder Saif
  • 11
  • 2
  • 1
    Hi, can you give us more information? For example table metadata and sample of data you have tried to upload. – Francesco Serra Nov 28 '19 at 14:28
  • Can you post the sample data and DDL for the table generated. You can get the DDL by running show create table . – Prabhakar Reddy Nov 29 '19 at 03:54
  • 1
    These are usually issues with commas (,) appearing in textual fields, quotes in others etc. – Guy Nov 30 '19 at 11:08
  • 1
    I had the same issue and it was due to commas in my COLUMN/HEADER names. (not in the actual rows of data). What's weird is that the table in the Athena UI showed my expected columns, not split ones – ThomasRones Oct 04 '22 at 17:01

1 Answers1

0

Thanks to @ThomasRones comment:

This indicates that there are more columns in the header row than that there are columns in the data rows (actually the determined types for these data rows).

Probable cause for this error is that in the header row one or more names contains one or more comma's.

Martijn Burger
  • 7,315
  • 8
  • 54
  • 94