selenium webdriver.Firefox not starting for non-sudo user

Question

I have a python method that sets up a browser in headless-mode on a linux server for website scraping with selenium. The display gets setup perfectly fine regardless of which user executes the python script but if the sudo user doesn't execute the script it will hang at the webdriver.Firefox() setup line indefinitely.

Here is the full method:

def browserSetup(self, browser=None):
    try:
        # now Firefox will run in a virtual display. you will not see the browser.  
        self.display = Display(visible=0, size=(800, 600))
        self.display.start()

        if self.verbose:
            print "Virtual display started for browser instantiation."

        #change user agent
        profile = webdriver.FirefoxProfile()
        profile.set_preference("general.useragent.override", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) AppleWebKit/536.30.1 (KHTML, like Gecko) Version/6.0.5 Safari/536.30.1")
        profile.set_preference("webdriver.log.file", "webdriver.log")

        # Create a new instance of the Firefox driver
        browser = webdriver.Firefox(profile)

        if self.verbose:
            print "Browser window object established @ %s." % browser

        return browser
    except Exception, e:
        raise e

So, to repeat my issue: if the script is not executed as sudo then the script will hang indefinitely at the webdriver.Firefox creation line. Why would this be happening?

UPDATE: The problem is this line here:

 browser = webdriver.Firefox() #with or without the profile variable - the results are the same

UPDATE AGAIN Several people in the comments below have suggested I try running Firefox from the command line manually to see if there are any issues; here are the results:

#initialize the virtual display
$ sudo Xvfb :10 -extension RANDR
[dix] Could not init font path element /usr/share/fonts/X11/cyrillic, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/100dpi/:unscaled, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/75dpi/:unscaled, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/Type1, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/100dpi, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/75dpi, removing from list!
[dix] Could not init font path element /var/lib/defoma/x-ttcidfont-conf.d/dirs/TrueType, removing from list!

#now start firefox in another ssh window (since the Xvfb process is consuming my prompt)
$ export DISPLAY=:10
$ firefox

(firefox:6347): GConf-WARNING **: Client failed to connect to the D-BUS daemon:
//bin/dbus-launch terminated abnormally without any error message

The last error message displays hundreds of times...

What are the permissions on firefox. How about `sudo chmod 777 location/to/firefox` and then trying it. Just shooting in the dark :) — Amey, Feb 27 '14 at 06:19
doing a `dpkg -L firefox` primarily shows all the firefox files in `/usr/lib/firefox` so I did a quick `sudo chmod -R 777 /usr/lib/firefox` and I am still having my issue. So, no that did not fix my problem — sadmicrowave, Feb 27 '14 at 13:01
Does this run as your normal user? Does the user where it's not working have permissions to start X? — oz123, Mar 03 '14 at 19:18
@Oz123 it only runs as `sudo`, even my normal user cannot start it. — sadmicrowave, Mar 03 '14 at 19:23
Any chance you're not running this within a virtualenv? Total shot in the dark. — 2rs2ts, Mar 03 '14 at 19:23
Add `profile.set_preference("webdriver.log.file", "/some/file")` and see if that file (change the path to where your user has write permissions) contains any useful info. — , Mar 03 '14 at 19:25
@André running firefox manually after starting `Xvfb` gives me an error: `Xlib: extension "RANDR" missing on display ":10".` — sadmicrowave, Mar 03 '14 at 19:28
@André I just tried `sudo Xvfb :10 -extension RANDR` to no avail — sadmicrowave, Mar 03 '14 at 19:33
@sadmicrowave did you try setting the log path in the profile as I said in a previous comment ? Maybe that log will get populated with some interesting info. — , Mar 03 '14 at 19:36
@André I added it (see my OP for the line) and after execution there is no `webdriver.log` file in the location specified. — sadmicrowave, Mar 03 '14 at 19:43
for what it's worth I've also tried switching to chrome using the `webdriver.Chrom("/path/to/chromedriver")` syntax, and it still stopped in the same place — sadmicrowave, Mar 03 '14 at 19:50
did you check perms on usr/lib? If it's dir access, you can either run as a user that has access to that dir or move the drivers inside your program's folder structure. sudo chmod 777 /usr/lib — Brantley Blanchard, Mar 10 '14 at 04:19
@sadmicrowave did you find a solution for this? Hope so because I am also running in to this issue. — Paul, Jun 20 '14 at 21:06
It wasn't a very clean solution but I first used chrome and chromedriver instead of firefox; I think I had to add sudo access to the chromedriver file. But ultimately I found a way to exclude Selenium and the virtual display & browser entirely by extending urllib(2). No need to include another library if you don't need to ;) — sadmicrowave, Jun 23 '14 at 13:04

score 2 · Answer 1 · answered Mar 03 '14 at 19:36

2

I can only guess, but here's what I suspect.

sudo clears your environment variables when you start Firefox via it. That includes the DISPLAY variable as well.

Two ways to disable this behavior: - Disable env_reset in your sudoers configuration. - Use sudo -i, that will preserve the value of DISPLAY.

answered Mar 03 '14 at 19:36

ldx

3,984
23
28

do you mean `sudo -E` for preserving environment variables? – sadmicrowave Mar 03 '14 at 19:44
That's also an option to preserve them. – ldx Mar 03 '14 at 19:52
This does not make a difference for my issue. Executing my python script with the -E option does not help. Not only that, my OP stated that I'm trying to run this script **without** using `sudo` – sadmicrowave Mar 03 '14 at 20:20
Oh I see, the OP was a bit misleading before the update. – ldx Mar 03 '14 at 21:00
Try checking out where firefox hangs. After you started xvfb and firefox, from another xterm: `DISPLAY=:1 import -window root screen.png` and check screen.png. – ldx Mar 03 '14 at 21:01

score 1 · Accepted Answer · answered Mar 08 '14 at 21:51

First you need to start the Xvfb (virtual frame-buffer X server) in the background.

For example, $ sudo Xvfb :1 & or $ sudo Xvfb :1 -screen 0 1280x1024x8.

Then, whenever you want to run your script, do it with the DISPLAY environment variable set accordingly.

For example, $ DISPLAY=:1 python your_script.py.

score 0 · Answer 3 · answered Mar 10 '14 at 04:22

You may not want to open usr/lib up for good but try testing it with open perms on the directory.

sudo chmod 755 /usr/lib

If that's the issue, you can always move the drivers inside your application. Unless you're sharing them with many people, they're small enough to have several copies.

score 0 · Answer 4 · answered Dec 17 '19 at 13:34

For anyone else like me that until today could be having this problem: For me was due to Firefox needs to be able to create a profile, which is created at the user home folder. The user i created didn't had its own user home folder so when i created it the problem was solved.

selenium webdriver.Firefox not starting for non-sudo user

4 Answers4