Jsoup HTTP error fetching URL. Status=403 when doing a post petition

Question

I'm been searching for this issue, supposedly this problem is fixed using an user agent, but this is not the case.
what i'm trying to do is to fetch the cookies from a petition, this the code

note: i'm try to do the petition to https webpage

/*obtiene cookies de la peticion*/
        Connection.Response res = Jsoup.connect(liga).header("Content-Type","text/html;charset=UTF-8")
                .cookie("TALanguage", "ALL")
                .data("mode", "filterReviews")
                .data("filterRating", "")
                .data("filterSegment", "")
                .data("filterSeasons", "")
                .data("filterLang", "ALL")
                .referrer(liga)         
                .header("X-Requested-With", "XMLHttpRequest")
                .header("X-Puid",xpuid)
                .data("returnTo",returnTo)
                .userAgent("Mozilla/5.0 (Windows; U; WindowsNT 5.1; en-US; rv1.8.1.6) Gecko/20070725 Firefox/2.0.0.6")                           
                .method(Method.POST)
                .execute();

        doc = res.parse();


        Map<String, String> cookies = res.cookies();

the program fails at the line .execute(); with this error in the log:

org.jsoup.HttpStatusException: HTTP error fetching URL. Status=403, URL=https://somepage.html

    at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:459)
    at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:434)
    at org.jsoup.helper.HttpConnection.execute(HttpConnection.java:181)
    at mx.oeste.crawler.htmlunit.obtenerComentarios(htmlunit.java:82)
    at mx.oeste.crawler.htmlunit.main(htmlunit.java:40)

You are trying to connect https, try it with http. I think the problem is you need to configure certificate first to call https. — tesnik03, Apr 28 '16 at 18:01
there is way to get the certificate from the site with jsoup? — Progs, Apr 28 '16 at 18:38
It's hard to tell what the problem is without knowing the URL. — TDG, Apr 28 '16 at 19:29
Possible duplicate of [Sending POST request with username and password and save session cookie](http://stackoverflow.com/questions/7206133/sending-post-request-with-username-and-password-and-save-session-cookie) — Stephan, Apr 28 '16 at 19:36
@Stephan not a dupe, this the first petition i make to get those cookies — Progs, Apr 28 '16 at 20:37

score 2 · Accepted Answer · answered Apr 28 '16 at 19:36

Try to set the content type header to "application/x-www-form-urlencoded" like below:

Connection.Response res = Jsoup.connect(liga)
                               .header("Content-Type","application/x-www-form-urlencoded")
                               .cookie("TALanguage", "ALL")
                               .data("mode", "filterReviews")
                               .data("filterRating", "")
                               .data("filterSegment", "")
                               .data("filterSeasons", "")
                               .data("filterLang", "ALL")
                               .referrer(liga)         
                               .header("X-Requested-With", "XMLHttpRequest")
                               .header("X-Puid",xpuid)
                               .data("returnTo",returnTo)
                               .userAgent("Mozilla/5.0 (Windows; U; WindowsNT 5.1; en-US; rv1.8.1.6) Gecko/20070725 Firefox/2.0.0.6")                           
                               .method(Method.POST)
                               .execute();

If it doesn't work, try to spy your favourite browser while fetching the cookies from the petition manually. You can use the developer tools for spying your browser.

actually i fetch cookies from another query i make before doing the post one — Progs, Apr 28 '16 at 20:25
@B.J.A.A. Did you spy on your browser to see how it performs the POST query? — Stephan, Apr 28 '16 at 20:37
@Stephan Hello. Could you take a look at my [post](https://stackoverflow.com/questions/45822632/why-do-i-get-http-403-from-my-app-but-i-can-browse-the-url-in-my-browser) I have the same error but this solution did not fix it — Bloomberg58, Aug 22 '17 at 19:09

Jsoup HTTP error fetching URL. Status=403 when doing a post petition

1 Answers1