I am reading book 'Designing Data-Intensive Applications' by Martin Kleppmann. At one point author has explained different approaches Twitter takes to publish tweets(fanout) to users. As per one of the approach, when someone posts a tweet, Twitter adds this tweet to a home timeline cache for all of the followers of the poster, as shown in image below.
Does it mean Twitter maintains a live cache for all of its users who are followers of someone? Won't it be too many caches to maintain?
Too many cache entries and not too many caches, precisely equal to total number of users.
So that would require TBs of cache approximately and with usage of commodity hardware can be served well on read path but upon write it will have high write amplification for celebrity.
But from maintenance perspective it won’t be an issue but from cost perspective it will.
Every single user has a cache not a single cache but multiple caches. Each cache is actually just a different data structure based on the purpose of the cache. Each user has its own User Timeline, Home Timeline and also a Search Timeline. For each timeline, Twitter might be using multiple caches. So the situation is more crazier than you think
Your example talks about only one timeline, the "Home Timeline" of each user. In the image, it shows that when the tweet is posted, it is added to the cache (It must be saved to the DB as well). The image shows those tweets are sent to the users, up to 31 million different followers. I think 31 million is a very high number for fanout but assuming it is true, what happens to people who have more than 31 million followers? Sending tweets to each follower's cache will take too much time.
In this case, the user might have a different cache for those people who has more than 31 million followers. (Twitter might implement different strategies for different ranges. 31-50,50-75, I am not sure). In this case, if our big guy posts a tweet, it just saves it to the DB and to our guy's "User timeline" (not the "Home timeline" cache), this tweet is not sent to all the followers. When one of the followers checks his "Home Timeline", the app looks at his Home timeline cache and also checks his cache for Big Guys cache. then it goes to check these Big Guys User Timeline, if there is a new tweet, it gets it, combines all the data and shows the final result to the user.
