I am fetching a data from an different APIs. I process the data and make a json file of size ~1GB. I truncate the document and insert json file in a MongoDB document. Due to nature of my app, I have to do this every hour. The insertion takes about 30 seconds.
While Mongo DB is inserting, it goes into a locked state so clients cannot read the data from the document.
I want to send the old data to the clients while MongoDB is inserting the data.
Any input to improve my architecture will be appreciated.
