Issue with using Tor with Selenium WebDriver (Java)

Question

So I am looking to do some simple data collection on a few .onion sites. I am going about this by using selenium webdriver to call Tor as part of the Firefox webdriver. However, I can't seem to figure out how to get firefox to successfully go to .onion sites. Here is the code.

public static void main(String[] args) throws InterruptedException, IOException {
    File torProfileDir = new File("C:\\Users\\Chambers\\Desktop\\Tor Browser\\Browser\\TorBrowser\\Data\\Browser\\profile.default");                
    FirefoxBinary binary = new FirefoxBinary(new File("C:\\Users\\Chambers\\Desktop\\Tor Browser\\Browser\\firefox.exe"));
    FirefoxProfile torProfile = new FirefoxProfile(torProfileDir);
    torProfile.setPreference("webdriver.load.strategy", "unstable");
    try {
        binary.startProfile(torProfile, torProfileDir, "");
    } catch (IOException e) {
        e.printStackTrace();
    }
    FirefoxProfile profile = new FirefoxProfile();
    profile.setPreference("network.proxy.type", 1);
    profile.setPreference("network.proxy.socks", "127.0.0.1");
    profile.setPreference("network.proxy.socks_port", 9150);
    FirefoxDriver driver = new FirefoxDriver(profile);
    int firstCheck = "0";

    while (firstCheck == 0) {
        driver.navigate().to("onion site here");
        ......

The problem I am having is that I end up with a firefox browser that cannot connect to .onion sites. If I change FirefoxDriver driver = new FirefoxDriver(profile); to FirefoxDriver driver = new FirefoxDriver(binary, profile); then I am left with a blank Tor window that I can't seem to control with the webdriver.

Anyone have any ideas on how to fix this? any help would be appreciated!

score 1 · Answer 1 · answered Feb 23 '15 at 20:55

Figured it out. Needed to add a lot of permissions to the new profile. Here is the fixed code for those that are interested. It allows you to browse the dark web with Firefox as if you were using Tor. The commands for controlling the webdriver don't change.

File torProfileDir = new File("C:\\Users\\Chambers\\Desktop\\Tor Browser\\Browser\\TorBrowser\\Data\\Browser\\profile.default");                
    FirefoxBinary binary = new FirefoxBinary(new File("C:\\Users\\Chambers\\Desktop\\Tor Browser\\Browser\\firefox.exe"));
    FirefoxProfile torProfile = new FirefoxProfile(torProfileDir);
    torProfile.setPreference("webdriver.load.strategy", "unstable");
    try {
        binary.startProfile(torProfile, torProfileDir, "");
    } catch (IOException e) {
        e.printStackTrace();
    }
    FirefoxProfile profile = new FirefoxProfile();
    profile.setPreference("network.proxy.type", 1);
    profile.setPreference("network.proxy.socks", "127.0.0.1");
    profile.setPreference("network.proxy.socks_port", 9150);
    profile.setPreference("network.proxy.socks_version", 5);
    profile.setPreference("places.history.enabled", false);
    profile.setPreference("privacy.clearOnShutdown.offlineApps", true);
    profile.setPreference("privacy.clearOnShutdown.passwords", true);
    profile.setPreference("privacy.clearOnShutdown.siteSettings", true);
    profile.setPreference("privacy.sanitize.sanitizeOnShutdown", true);
    profile.setPreference("signon.rememberSignons", false);
    profile.setPreference("network.cookie.lifetimePolicy", 2);
    profile.setPreference("network.dns.disablePrefetch", true);
    profile.setPreference("network.http.sendRefererHeader", 0);
    profile.setPreference("network.proxy.socks_remote_dns", true);
    FirefoxDriver driver = new FirefoxDriver(profile);
    String firstCheck = "";
    while (firstCheck == 0) {
        driver.get("http://kbhpodhnfxl3clb4.onion/");
        .........................

Also, add in `profile.setPreference("permissions.default.image", 2);` to make loading sites a lot faster. regular .onion sites take a while as it is, so every little bit counts right? — Chambers Volk, Feb 23 '15 at 23:22

Issue with using Tor with Selenium WebDriver (Java)

1 Answers1