1

Hi I am running a Streamlit app with DuckDB persisted on a file. The app is is growing quite large, If I split it into a multi-page app, will it be problematic with database files locks and such, given DuckDB stores the data on a single file? Are Streamlit multi-page app pages sharing the same process and thread? If so, does that mean it won't likely cause issues, or will it be wise to avoid such a setup and store the data in another way?

Tofusoul
  • 123
  • 6

1 Answers1

0

What I have discovered so far is that DuckDB is not really meant for the use cases where concurrent users read and write. The complexity to manage a single write process using locks comes to our hands and it is better to avoid this setup.

If your goal is to power an analytics page, you can create multiple read-only connections from multiple instances of the app. There should be no issues. But to populate data in duckdb you will have to create a single instance of a corn job, which opens DuckDB database file in read/write mode, populates duckdb, and then closes the connection. I am following a similar approach.

These FAQs could be helpful

Nirdosh Gautam
  • 236
  • 2
  • 5