2

I am developing a website using Django1.7 on python3.4 along with MySql as database engine. For the next 15-20 days I am planning to test it. The site is something like linkedin in terms of functionality and complexity and I am expecting to get around 20-30 thousand users in the next 6 months. I have learnt about MySql during the development of my website only. I am using Django-debug-toolbar and have tried to reduce the query time and reduce the number of joins. I have a few questions:

  1. what tools can be used to create http request to automatically fill the database and also query various pages.

  2. Is django-debug-toolbar enough to do the profiling and optimization considering increasing number of requests from multiple user.

  3. Whether should I work on reducing the number of database hits or the size of querysets django would be caching and how the use of RAM is going to affect the performance of the website.

Considering that I have no prior experience of database administration and handling a website, how should i determine whether the website is performing up to the mark. Please share the bestpractices as i am quite unfamiliar.

Rohit
  • 475
  • 1
  • 7
  • 16

1 Answers1

4

The single biggest factor in SQL performance is the number of rows in the tables you use. You should figure out how to load 50 thousand fake users and 1 million fake nodes into your test database.

Then guess which of your pageviews will be most common. Find a free load testing tool on the net (there are quite a few) and use it to hit that page hard on your server.

Figure out which queries are slow. Add appropriate indexes, or if you must, redesign your data base, and get those queries to be fast.

Then guess at your second most popular pageview and repeat. Keep going until you run out of time.

Keep in mind that this is guesswork. As your service ramps up in users, you need to keep an eye on the pageviews your real users prefer, and keep an eye on those slow queries.

This will, if you add users at the rate you plan, take a sizeable fraction of your time during your first year in operation.

Read a web site called http://use-the-index-luke.com/

O. Jones
  • 103,626
  • 17
  • 118
  • 172
  • Thanks sir for your great advice and the methodical approach you have suggested. I have started reading about indexing. However, i would like to get a little more specific advises as to what particular tools I should use, whether apache2 installed on ubuntu server with 1 gb RAM be able to handle the load or not, whether should I try to minimize the database hits or reduce the querysets being loaded in the memory. For that purpose, I am leaving the question open for now. – Rohit May 29 '15 at 18:36
  • With respect, questions like the ones you ask (how much RAM? Apache or some other server? result set caching? etc) are things you can only learn from experience *with your web app.* Load testing, as I described it, is merely a way to accelerate your experience. – O. Jones May 30 '15 at 11:27