2

Does MonetDB support tables with ~74,000 columns?

Number of Columns: 74,000 columns. Yes I am asking about a table with seventy-four-thousand columns.

Data types: All data types for columns in these tables would be a mix of 50% of columns Int, 40% of columns Decimal(10,2), and 10% of columns Varchar(255).

Number of Rows / Records: Each table would contain 300,000 to 750,000,000 rows

Context of Project:

  • This is not for a data warehouse use case.

    • This is a data science project with high dimensions and no duplicate data so there's no way to build a different data model, e.g. Star Schema, etc. is not an option for this project.

    • My use case is reading from and inserting into these tables. No updates. No deletes. I will analyze data in these tables using R natively in MonetDB so I can keep all data in MonetDB rather than transfer to R, analyze and insert results from R back into MonetDB, etc...

Webby
  • 337
  • 1
  • 3
  • 10

1 Answers1

2

Yes, 74K columns should be fine.

But: If you have 750M rows in such a table (55 trillion fields), for four-byte-integers, the table will occupy ~ 200 TB of disk space (!). Its unlikely you have that much...

Hannes Mühleisen
  • 2,542
  • 11
  • 13