How do I write polars dataframe to external database?

Question

I have big polars dataframe that I want to write into external database (sqlite for example) How can I do it?

In pandas, you have to_sql() function, but I couldn't find any equivalent in polars

Please provide enough code so others can better understand or reproduce the problem. — Community, Feb 24 '23 at 18:29
For **`polars>=0.16.10`** you can use `df.write_database()` [method](https://github.com/pola-rs/polars/releases/tag/py-0.16.10). — glebcom, Mar 04 '23 at 08:05

score 2 · Accepted Answer · answered Feb 24 '23 at 17:28

Polars doesn't provide a utility for inserting data to a database.

Your options, that I can think of, are:

to_pandas().to_sql(...)
DIY

a. Generate insert statements from df.rows() like this

b. Apparently sqlite doesn't do bulk inserts but for postgres you can use a combination of io, write_csv, and copy_from which will be much faster than using insert statements.
Save parquet files and either use them directly or use something like duckdb to access them as though they were a traditional db

score 1 · Answer 2 · answered May 31 '23 at 20:42

1

You can use the DataFrame.write_database method.

answered May 31 '23 at 20:42

ritchie46

2 Answers2