I'm doing load testing with locust to see if my server can handle 1500 users.
What I'm using: Django, Nginx, Gunicorn, Postgresql My droplet: 24vCPUs, 128GB RAM, 25GB SSD
at about 1100 users I start getting the following errors in locust:
GET / HTTPError('502 Server Error: Bad Gateway for url: myurl.here ')
GET /aboutpage/ HTTPError('502 Server Error: Bad Gateway for url: myurl.here ')
In my nginx error.log I get the following errors:
2020/01/26 23:14:17 [error] 30465#30465: *167765 connect() to unix:/var/www/file/to/sock/gunicorn failed (11: Resource temporarily unavailable) while connecting to upstream, client: 8x.8x.1xx.3x, server: mysite.here, request: "GET // HTTP/1.1", upstream: "http://unix:/var/www/file/to/sock/gunicorn://", host: "mysite.here"
for some reason it tells me that my Resource is temporarily unavailable.
Here is my nginx.conf:
user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;
events {
worker_connections 2048;
# multi_accept on;
}
http {
##
# Basic Settings
##
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
# server_tokens off;
# server_names_hash_bucket_size 64;
# server_name_in_redirect off;
include /etc/nginx/mime.types;
default_type application/octet-stream;
##
# SSL Settings
##
ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
ssl_prefer_server_ciphers on;
##
# Logging Settings
##
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
##
# Gzip Settings
##
gzip on;
# gzip_vary on;
# gzip_proxied any;
# gzip_comp_level 6;
# gzip_buffers 16 8k;
# gzip_http_version 1.1;
# gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;
# Virtual Host Configs
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
}
And here is my server block configuration:
upstream mysite-production {
server unix:/var/www/path/to/sock/gunicorn;
}
server {
listen [::]:80;
listen 80;
server_name mysite.here;
# set client body size to 100M #
client_max_body_size 100M;
location / {
include proxy_params;
proxy_pass http://unix:/var/www/path/to/sock/gunicorn;
auth_basic "Restricted Content";
auth_basic_user_file /etc/nginx/.htpasswd;
}
location /static/ {
root /var/www/site/production/;
expires 30d;
add_header Vary Accept-Encoding;
access_log off;
gzip on;
gzip_comp_level 6;
gzip_vary on;
gzip_types text/plain text/css application/json application/x-javascript application/javascript text/xml application/xml application/rss+xml text/javascript image/svg+xml application/vnd.ms-fontobject application/x-font-ttf font/opentype;
}
location /media/ {
root /var/www/site/production/;
expires 30d;
add_header Vary Accept-Encoding;
access_log off;
}
}
Here is my gunicorn service file:
[Unit]
Description=mysite production daemon
After=network.target
[Service]
User=www-data
Group=www-data
WorkingDirectory=/var/www/path/to/site/src
ExecStart=/var/www/path/to/venv/bin/gunicorn --workers=49 --bind unix:/var/www/path/to/sock/gunicorn --log-level DEBUG --log-file '/var/www/path/to/log/gunicorn.log' mysite.wsgi:application
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/bin/kill -s TERM $MAINPID
[Install]
WantedBy=multi-user.target
What could be causing this problem? We are predicting about 1500-2000 concurrent users that will be on the site at the same time.
Thank you for taking the time! I'm looking forward to your answers!
