Getting raw HTTP response headers

Question

Is there any way to get raw response http header?

The getHeaderField() method doesn't work for me, because server spits multiple 'Set-Cookie' and some of them get lost.

Answer 1

The getHeaderField() method doesn't work for me

You're asking this in the context of java.net.URLConnection, is it? No, obtaining the raw HTTP response headers is not possible with URLconnection. You'll need to fall back to low-level Socket programming. Here's an SSCCE, just copy'n'paste'n'run it.

package com.stackoverflow.q2307291;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.OutputStreamWriter;
import java.io.PrintWriter;
import java.net.Socket;

public class Test {

    public static void main(String[] args) throws IOException {
        String hostname = "stackoverflow.com";
        int port = 80;

        Socket socket = null;
        PrintWriter writer = null;
        BufferedReader reader = null;

        try {
            socket = new Socket(hostname, port);
            writer = new PrintWriter(new OutputStreamWriter(socket.getOutputStream()));
            writer.println("GET / HTTP/1.1");
            writer.println("Host: " + hostname);
            writer.println("Accept: */*");
            writer.println("User-Agent: Java"); // Be honest.
            writer.println(""); // Important, else the server will expect that there's more into the request.
            writer.flush();

            reader = new BufferedReader(new InputStreamReader(socket.getInputStream()));
            for (String line; (line = reader.readLine()) != null;) {
                if (line.isEmpty()) break; // Stop when headers are completed. We're not interested in all the HTML.
                System.out.println(line);
            }
        } finally {
            if (reader != null) try { reader.close(); } catch (IOException logOrIgnore) {} 
            if (writer != null) { writer.close(); }
            if (socket != null) try { socket.close(); } catch (IOException logOrIgnore) {} 
        }
    }

}

To avoid SO being overloaded by everyone trying this snippet, here's how the output will look like:

HTTP/1.1 200 OK
Cache-Control: private
Content-Type: text/html; charset=utf-8
Expires: Sun, 21 Feb 2010 20:39:08 GMT
Server: Microsoft-IIS/7.5
Date: Sun, 21 Feb 2010 20:39:07 GMT
Connection: close
Content-Length: 208969

To learn more about sending HTTP requests the low-level way, read the HTTP specification.

However, you probably want to make use of getHeaderFields() method instead to retrieve a header with multiple values. The getHeaderField() namely only returns the last value, as per the linked API doc.

List<String> cookies = connection.getHeaderFields().get("Set-Cookie");

Answer 2

Not exactly 'raw' but concise:

for (Map.Entry<String, List<String>> k : myHttpURLConnection.getHeaderFields().entrySet()) {
    System.out.println(k.toString());
}

IF you worry that some of the headers are getting lost use:

for (Map.Entry<String, List<String>> k : myHttpURLConnection.getHeaderFields().entrySet()) {
    for (String v : k.getValue()){
         System.out.println(k.getKey() + ":" + v);
    }
}

PS: Better late than never. :)

Answer 3

The easy way is to use the getHeaderFields() method of URLConnection. Here is some code that does something equivalent.

static String[] getHeaders(HttpURLConnection con, String header) {
  List<String> values = new ArrayList<String>();
  int idx = (con.getHeaderFieldKey(0) == null) ? 1 : 0;
  while (true) {
    String key = con.getHeaderFieldKey(idx);
    if (key == null)
      break;
    if (header.equalsIgnoreCase(key))
      values.add(con.getHeaderField(idx));
    ++idx;
  }
  return values.toArray(new String[values.size()]);
}

Answer 4

Late to the party, but here's the simplest solution. Just implement CookieStore. (or use the default implementation and let it take care of adding the cookies to your subsequent calls.)

http://docs.oracle.com/javase/7/docs/api/java/net/CookieStore.html

Set your cookie store as the default cookie manager

CookieManager cookieManager = new CookieManager(new MyCookieStore(), CookiePolicy.ACCEPT_ALL);
CookieHandler.setDefault(cookieManager);

And every new cookie will appear to you in add() in your CookieStore. I had the same problem with params being overwritten by having the same name "Set-Cookie" in a single request, and now I get both the cookie and the sessionId.