How do I get the response headers in Selenium?

Question

How can I get request/response logs in selenium? I have an ajax call that returns login information, and whenever I try to capture it via:

 selenium.captureNetworkTraffic("json");

it returns only client-side items (like images .pn), but not the actual JSON response I'm interested in.

can any one help me,i thought selenium should obviously have clicks made inside and response from it?? didnt any one tried this?? — sasikumar, Aug 18 '10 at 10:45

score 19 · Accepted Answer · edited May 23 '17 at 12:34

Unfortunately, it doesn't look like you can do that with selenium alone in an automated fashion, as this is against their design principles for selenium. I don't agree that it is (since it supports getting cookie information -- what user actually goes through and parses through their cookies?).

Their 'schtick' is that selenium helps you automate what a user does; since a user doesn't care about the response headers, why would this framework? For me, this is disappointing because I need to ensure the content type being returned is what I believe it should be (to test an API). It appears another Stack Overflow question has come to a similar conclusion.

score 8 · Answer 2 · answered Oct 18 '20 at 17:34

In case someone is still looking for an answer, there is a way to get the response headers using Selenium 4 and Chrome developer tools API

DevTools devTools = driver.getDevTools();
devTools.createSession();
devTools.send(Network.enable(Optional.empty(), Optional.empty(), Optional.empty()));
devTools.addListener(Network.responseReceived(),
        responseReceived -> {
            Headers headers = responseReceived.getResponse().getHeaders();
            if (!headers.isEmpty()){
                System.out.println("Headers:");
                headers.forEach((key,value) -> {
                    System.out.println("    " + key + "="+ value);
                });
            }
        });

This was tested using Selenium 4 alpha 6

Not _just_ `ChromeDriver`... Technically any of the drivers implementing `HasDevTools` (which appears to at least include firefox and edge). — Lucas, Jul 21 '22 at 13:44

score 2 · Answer 3 · answered Jun 10 '15 at 22:18

2

I realize that this question is old as hell, but I've developed a way to do this for firefox + selenium by modifying webdriver.xpi and creating driver.http_logger.variousLogMethods(). Please reply to this post if there's interest in this, and I can post the code to github.

answered Jun 10 '15 at 22:18

sam-6174

3,104
1
33
34

1

I am interested. Please post the github link. – Krishna Pandey Mar 09 '16 at 05:41
@user2426679 I'm interested as well, could you please show me on Github how to execute this? – David542 Oct 05 '16 at 19:56
@David542 -- I finally got around to uploading this little project. I did this a while back, so the documentation is sparse. https://bitbucket.org/stbrownlow/seleniumwithfirefoxhttplogging/src – sam-6174 May 13 '17 at 06:18
Interesting. Has anyone been able to get this to work on newer versions of FF? – FlyingZebra1 May 03 '19 at 21:10
@FlyingZebra1 it won't work without rewriting the patch within `webdriver.xpi`. See the readme in the BitBucket for locating the edits I made to the .xpi selenium extension, as a starting point for porting to a newer version of Firefox. – sam-6174 May 07 '19 at 15:58

score 2 · Answer 4 · answered Sep 16 '19 at 10:22

2

If you are looking for other way round solution it could be also done using proxy server which will output log file including headers for each request/response which go through the proxy.

I have used Titanium Web Proxy and registered proxy for driver options like this:

ChromeOptions chromeOptions = new ChromeOptions
{
    Proxy = new Proxy
    {
        Kind = ProxyKind.Manual,
        HttpProxy = "localhost:8888"
    }
};

answered Sep 16 '19 at 10:22

Gucu112

877
10
12

1

I upvoted, I did not check whether it actually works. But I think the idea is worth a vote nonetheless. – Mike de Klerk Aug 12 '20 at 07:50

score 1 · Answer 5 · edited Jun 19 '17 at 00:08

You can do it by injecting a XMLHttpRequest and waiting for the result:

Object response = ((JavascriptExecutor) driver).executeAsyncScript(
       "var callback = arguments[arguments.length - 1];" +
       "var xhr = new XMLHttpRequest();" +
       "xhr.open('GET', '/resource/data.json', true);" +
       "xhr.onreadystatechange = function() {" +
       "  if (xhr.readyState == 4) {" +
       "    callback(xhr.getResponseHeader());" +
       "  }" +
       "};" +
       "xhr.send();");
   JsonObject json = new JsonParser().parse((String) response);
   assertEquals("cheese", json.get("food").getAsString());

You can modify the xhr callback to get what you want.

score 0 · Answer 6 · answered Aug 29 '14 at 13:40

I found a trick if you use jquery you can register a global listener :

First you need to declare a global variable in your main script :

var globalVariableForSelenium = "";

In your selenium you register the listener :

if (driver instanceof JavascriptExecutor) {
                ((JavascriptExecutor) driver)
                .executeScript("$(document).ajaxSuccess(function(event, xhr, settings) {  globalVariableForSelenium = settings.headers    });");
            }

Then you execute your action

Finaly you get the value of the global variable :

if (driver instanceof JavascriptExecutor) {
            Object res = ((JavascriptExecutor) driver)
            .executeScript("return  serializeGlobalvariable();");

}

serializeGlobalvariable() is a function that will serialize globalVariableForSelenium which contains an array of header.

ps: my need was to access to the ajax response so I set the global variable with xhr.responseJSON.mypath

How do I get the response headers in Selenium?

6 Answers6