0

Hello I was load testing my social media platform with LocustIO that uses django 2.1 and SQlite3, and these are my results. Load Testing Results

I need to make sure it can run for 10000+ visitors and have looked into caching options like redis. It is a real time app so it involves heavy database usage and power. What road should I go down to make it quicker and scalable?

Ty C
  • 69
  • 9
  • Welcome to SO! SQlite3 is not really meant for a production environment, at least not for busy sites. And yes, caching is important and Memcached etc. is your best friend. BTW 10K visitors a day is not much while 10K concurrent visitors is a slightly different story. Which one is it? – WayBehind Nov 14 '18 at 01:44
  • @WayBehind thanks for the response. I am planning for 10K concurrent visitors. I recently switched my database to Postgres. I’ve looked up possible solutions such as web sockets and pusher. It’s an app where tokens are being bought and spent for context. – Ty C Nov 14 '18 at 01:47
  • Just to give you an idea. On AWS I run Django app on EalactiBeanstalk with two T2.small servers behind a Loadbalancer, and just one cache.t2.small Memcached server along t2.medium MySQL RDS (Multi AZ) and its all running at about at 10% capacity while having about 2M requests per month through CloudFlare. – WayBehind Nov 14 '18 at 01:56
  • @WayBehind Do you use a classic load balancer through AWS? Also what do you use CloudFare for if you are using EalactiBeanstock and AWS? – Ty C Nov 14 '18 at 06:12
  • Yes, classic LoadBalancer. As per CloudFlare, there is a lot of benefits to use their DNS etc but one of the main reasons is that I dont have to deal with SSL certificates anymore as those are included. – WayBehind Nov 14 '18 at 09:09
  • @WayBehind Thanks for the advice. I was wondering if you would have any advice on how to setup a real time ranking system. I am currently using database and post-save signals to run the ranking but this is brutal on the database and scalability. – Ty C Nov 14 '18 at 23:21
  • While I dont have much experience with real time ranking systems, I do have some ranking results that have millions of "votes" and some of the queries are quite slow so I cant even imagine that would run in a real time as the queries may take 10+ seconds and running concurrent query before the first was even executed does not make any sense. So depending on the your query speeds I would say that caching the results for at least 10-60 seconds would still count as a real time, at least for my needs. Perhaps there is a better solution. – WayBehind Nov 15 '18 at 00:30

2 Answers2

1

So the first thing you'll want to do is figure out where your time is being taken up. Is most of the time taken up with database queries? other network operations? page rendering?

Django-debug-toolbar is a pretty good first step at answering that all-important first question.

Then once you have some idea of where the problem is, you can focus your brain power on solving that problem.

Chris Curvey
  • 9,738
  • 10
  • 48
  • 70
0

Sqlite 3 cant handle much concurrency.So migrate to more robust sql. Also Check out silk.

Garry Kevin
  • 303
  • 2
  • 9