Xvfb has just begun failing without code changes

Question

I have server that runs gunicorn and python programs. Another server sends a url and the server in question scrapes the url and sends some information back.

I have selenium running a firefox tab in the background using Xvfb.

This is how I have been restarting my server for awhile now:

sudo Xvfb :10 -ac
export DISPLAY=:10
sudo pkill gunicorn
gunicorn -b 0.0.0.0:8080 wsgi:app

Suddenly, any scrape that used Xvfb no longer works and errors out, but the same code still works on my local computer, so I know it has to do with it being on the server and using Xvfb.

Here is the new content that it started printing after it stopped working:

7 XSELINUXs still allocated at reset
SCREEN: 0 objects of 256 bytes = 0 total bytes 0 private allocs
DEVICE: 0 objects of 96 bytes = 0 total bytes 0 private allocs
CLIENT: 0 objects of 136 bytes = 0 total bytes 0 private allocs
WINDOW: 0 objects of 32 bytes = 0 total bytes 0 private allocs
PIXMAP: 2 objects of 16 bytes = 32 total bytes 0 private allocs
GC: 4 objects of 16 bytes = 64 total bytes 0 private allocs
CURSOR: 1 objects of 8 bytes = 8 total bytes 0 private allocs
TOTAL: 7 objects, 104 bytes, 0 allocs
2 PIXMAPs still allocated at reset
PIXMAP: 2 objects of 16 bytes = 32 total bytes 0 private allocs
GC: 4 objects of 16 bytes = 64 total bytes 0 private allocs
CURSOR: 1 objects of 8 bytes = 8 total bytes 0 private allocs
TOTAL: 7 objects, 104 bytes, 0 allocs
4 GCs still allocated at reset
GC: 4 objects of 16 bytes = 64 total bytes 0 private allocs
CURSOR: 1 objects of 8 bytes = 8 total bytes 0 private allocs
TOTAL: 5 objects, 72 bytes, 0 allocs
1 CURSORs still allocated at reset
CURSOR: 1 objects of 8 bytes = 8 total bytes 0 private allocs
TOTAL: 1 objects, 8 bytes, 0 allocs
1 CURSOR_BITSs still allocated at reset
TOTAL: 0 objects, 0 bytes, 0 allocs

I have tried killing all Xvfb and Firefox jobs and killing all background jobs and restarting, but the same error still occurs.

EDIT 1

Also, one recent change is that when I type:

sudo Xvfb :10 -ac

It has started outputting:

Initializing built-in extension Generic Event Extension
Initializing built-in extension SHAPE
Initializing built-in extension MIT-SHM
Initializing built-in extension XInputExtension
Initializing built-in extension XTEST
Initializing built-in extension BIG-REQUESTS
Initializing built-in extension SYNC
Initializing built-in extension XKEYBOARD
Initializing built-in extension XC-MISC
Initializing built-in extension SECURITY
Initializing built-in extension XINERAMA
Initializing built-in extension XFIXES
Initializing built-in extension RENDER
Initializing built-in extension RANDR
Initializing built-in extension COMPOSITE
Initializing built-in extension DAMAGE
Initializing built-in extension MIT-SCREEN-SAVER
Initializing built-in extension DOUBLE-BUFFER
Initializing built-in extension RECORD
Initializing built-in extension DPMS
Initializing built-in extension Present
Initializing built-in extension DRI3
Initializing built-in extension X-Resource
Initializing built-in extension XVideo
Initializing built-in extension XVideo-MotionCompensation
Initializing built-in extension SELinux
Initializing built-in extension GLX

EDIT 2

Here are the new error codes after I deleted some error blocking try: except: statements.

  ...
  File "./app/scraper/main.py", line 340, in uScrape
    driver = webdriver.Firefox()
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/firefox/webdriver.py", line 77, in __init__
    self.binary, timeout),
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/firefox/extension_connection.py", line 49, in __init__
    self.binary.launch_browser(self.profile)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/firefox/firefox_binary.py", line 68, in launch_browser
    self._wait_until_connectable()
  File "/usr/local/lib/python2.7/dist-
packages/selenium/webdriver/firefox/firefox_binary.py", line 103, in _wait_until_connectable
    raise WebDriverException("Can't load the profile. Profile "
WebDriverException: Message: Can't load the profile. Profile Dir: %s If you specified a log_file in the FirefoxBinary constructor, check it for details.

EDTI 3

$pip show selenium

Name: selenium
Version: 2.53.1
Location: /usr/local/lib/python2.7/dist-packages

Is there some reason you want gunicorn to run inside your virtual X session? Also, I doubt it will help, but the way I've been doing selenium is via xvfb-run python .py ... this kicks off a virtual framebuffer session and runs my selenium and then stops the Xvfb session... also, what errors do you get when running your selenium (or just running firefox?) — Foon, Apr 18 '16 at 21:44
@Foon See edit 2, and ill take a look at case by case xvfb running — Rorschach, Apr 18 '16 at 21:52

score 0 · Answer 1 · answered Apr 18 '16 at 22:13

0

After doing these 3 commands this issue was fixed

sudo apt-get upgrade firefox
pip install --upgrade selenium
sudo apt-get update

answered Apr 18 '16 at 22:13

Rorschach

3,684
7
33
77

Xvfb has just begun failing without code changes

1 Answers1