I need to design a system from scratch that keeps track of the trend on a dataset. This is achievable easily. The complexity increases because, on the same dataset, we need to perform dynamic filtering based on different attributes.
Do you have a suggestion or an example I can look at? I need an end-to-end solution for the backend and am least bothered about the front-end pieces.
Thank you for your time.
We have implemented static count tracking, where a job runs to calculate the total count of the dataset and save it as a total count against each day. This is static and does not allow on-demand filtering because it is a fixed count persisted in DB.
