I'm going totally crazy here. I'm developing a website using the Python Flask framework and since this afternoon my Linux box is behaving like a mad man. Let me explain.
I've got my Flask website on an Ubuntu 14.04 server. I normally run it using ./run.py file using a virtualenv. Today I installed some packages and finally decided I wanted a clean env again. So I deleted my venv folder and created it again, like a did a million times before:
$ virtualenv venv
$ . venv/bin/activate
(venv) $ pip install -r requirements.txt
# EVERYTHING INSTALLS WITHOUT ERRORS
(venv) $ ./run.py
/home/kramer65/mysite/venv/local/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:100: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
* Running on http://0.0.0.0:5000/
* Restarting with inotify reloader
* Debugger is active!
* Debugger pin code: 219-856-383
/home/kramer65/mysite/venv/local/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:100: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
This is all pretty much as expected. I then proceed to my browser and load my website (also like I did a million times before). The browser loads a very long time, but nothing gets loaded. In the browser console I don't even see any connection opening at all.
I then check the terminal and see nothing happening either. So I stop the dev server using a CTRL+C and get the following output (as expected):
^CException in thread Thread-6:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
self.run()
File "/home/kramer65/mysite/venv/local/lib/python2.7/site-packages/watchdog/observers/inotify_buffer.py", line 59, in run
inotify_events = self._inotify.read_events()
File "/home/kramer65/mysite/venv/local/lib/python2.7/site-packages/watchdog/observers/inotify_c.py", line 296, in read_events
event_buffer = os.read(self._inotify_fd, event_buffer_size)
KeyboardInterrupt
So I check out any network connections which are open:
$ netstat -tulpn
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 127.0.0.1:6379 0.0.0.0:* LISTEN -
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN -
tcp 0 0 0.0.0.0:5000 0.0.0.0:* LISTEN 6127/python
tcp 0 0 127.0.0.1:27017 0.0.0.0:* LISTEN -
tcp 0 0 127.0.0.1:3306 0.0.0.0:* LISTEN -
tcp6 0 0 :::22 :::* LISTEN -
udp 0 0 0.0.0.0:68 0.0.0.0:* -
udp 0 0 0.0.0.0:24675 0.0.0.0:* -
udp6 0 0 :::22996 :::* -
So I decide to kill the python process on port 5000 using kill 6127. What then happens just stunns me. The command gets "accepted" (I get no error), but then I see the browser suddenly loads the website and the terminal starts outputting logs:
(venv) $ kill 6127
(venv) $ 83.161.103.21 - - [2016-04-13 13:59:44] "GET / HTTP/1.1" 200 7334 0.045281
83.161.103.21 - - [2016-04-13 13:59:44] "GET / HTTP/1.1" 200 7334 0.062855
83.161.103.21 - - [2016-04-13 13:59:44] "GET / HTTP/1.1" 200 7334 0.080915
83.161.103.21 - - [2016-04-13 13:59:44] "GET / HTTP/1.1" 200 7334 0.099523
83.161.103.21 - - [2016-04-13 13:59:44] "GET / HTTP/1.1" 200 7334 0.118382
83.161.103.21 - - [2016-04-13 13:59:45] "GET /bower/bootstrap/dist/css/bootstrap.min.css?version=3.3.5 HTTP/1.1" 304 745 0.006047
83.161.103.21 - - [2016-04-13 13:59:45] "GET /bower/font-awesome/css/font-awesome.min.css?version=1448904482.0 HTTP/1.1" 304 745 0.003684
etc. etc.
And from this I don't understand anything. I get the output even though I thought I stopped the dev-server using CTRL+C and ran two commands in between (netstat -tulpn and kill 6127).
I can then hit enter a couple times, which gets me my start line again, and then run arbitrary commands:
(venv) $
(venv) $
(venv) $ pwd
/home/kramer65/mysite
(venv) $
and when I reload the website in the browser I simply get the output again in the same terminal window. My commands, any output of my commands, and the Flask dev-server logs just all get outputted in the same terminal window.
I tried rebooting the server, completely removing all website code and removing and installing the virtualenv again, and I even created a new user and installed everything in there, but I get this same result over and over again.
The thing is that I don't know WHY I get this result. Is it because of Flask, because of the virtualenv, because of some weird bug with the network, because of bash or even something else?!
And most importantly, I have no idea how I can solve this. Does anybody have any idea? All tips are welcome, because I'm pulling out my hair here..
[EDIT]
I ran $ ps -eaf | grep python, with the following results:
$ ps -eaf | grep python
immotest 1489 1444 0 14:33 pts/1 00:00:00 venv/bin/python ./run.py
immotest 1497 1489 1 14:33 pts/1 00:00:00 /home/kramer65/mysite/venv/bin/python ./run.py
immohie+ 1510 1374 0 14:34 pts/0 00:00:00 grep --color=auto python
