HTTP authentication in Groovy?

Question

def http = new HTTPBuilder(url)
http.auth.basic username, password

def data = new URL(url)

def response = data.getText()

def relativePaths = new XmlParser().parseText(response)

relativePaths.each {page ->

    def link = page.content.@href.first()

    if(link.contains(".txt")){
        println link
    }

}

Hi,

when trying to run this, I get a 401. It seems like here:

def response = data.getText()

it is actually not authenticated anymore (or at all) ? In Ruby I would write it like that:

(open(url, :http_basic_authentication => [username, password])
data = Nokogiri::XML(open(url, :http_basic_authentication => [username, password])))

How would that look like in Groovy ? Am I using httpbuilder wrong ? Thanks a lot in advance !

Update 1:

After applying the changes:

def http = new HTTPBuilder(url)
http.auth.basic username, password

http.get(path: url, contentType: ContentType.TEXT) { resp, reader ->

    def relativePaths = new XmlParser().parse(reader)

    relativePaths.each { page ->

        def link = page.content.@href.first()

        if (link.contains(".txt")) {
            println link
        }
    }
}

I get an 406:

Response code: 406; found handler: org.codehaus.groovy.runtime.MethodClosure@5be46f9d Caught: groovyx.net.http.HttpResponseException: Not Acceptable groovyx.net.http.HttpResponseException: Not Acceptable

Update 2:

Still not working. I changed the contentType to XML and the path to the actual path, but getting an error message:

GET <URL>/<PATH>:(id:ContinuousIntegration_BuildTestDistribute)/artifacts/children/
Response code: 200; found handler: ArtifactDownloader$_run_closure1@5be46f9d
Parsing response as: application/xml
Could not find charset in response; using UTF-8
Parsed data to instance of: class groovy.util.slurpersupport.NodeChild
Caught: groovy.lang.MissingMethodException: No signature of method: groovy.util.XmlParser.parse() is applicable for argument types: (groovy.util.slurpersupport.NodeChild) values: []
Possible solutions: parse(java.io.File), parse(java.io.InputStream), parse(java.io.Reader), parse(java.lang.String), parse(org.xml.sax.InputSource), use([Ljava.lang.Object;)
groovy.lang.MissingMethodException: No signature of method: groovy.util.XmlParser.parse() is applicable for argument types: (groovy.util.slurpersupport.NodeChild) values: []
Possible solutions: parse(java.io.File), parse(java.io.InputStream), parse(java.io.Reader), parse(java.lang.String), parse(org.xml.sax.InputSource), use([Ljava.lang.Object;)
    at ArtifactDownloader$_run_closure1.doCall(ArtifactDownloader.groovy:14)
    at groovyx.net.http.HTTPBuilder$1.handleResponse(HTTPBuilder.java:503)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:218)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:160)
    at groovyx.net.http.HTTPBuilder.doRequest(HTTPBuilder.java:515)
    at groovyx.net.http.HTTPBuilder.get(HTTPBuilder.java:285)
    at groovyx.net.http.HTTPBuilder$get.call(Unknown Source)
    at ArtifactDownloader.run(ArtifactDownloader.groovy:12)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)

Just for comparison, the Ruby script. Maybe httpbuilder is just not what I want ? I actually wanted to use it for authentication only.

(open(url, :http_basic_authentication => [username, password])
data = Nokogiri::XML(open(url, :http_basic_authentication => [username, password])))

relativePaths = data.xpath("//content/@href")

relativePaths.each { |path| 
    if path.text.include? ".apk"
        puts "<URL>:<PORT>"+path.text
    end
}

Emmanuel Rosa · Answer 1 · 2016-03-04T15:23:55.373

8

Are you ready for a face palm?

You created an HTTPBuilder and... never used it. You got it all set up...

def http = new HTTPBuilder(url)
http.auth.basic(username, password)

... but then used a new URL to retrieve the data:

def data = new URL(url)
def response = data.getText()

Solution

An HTTPBuilder is used like this:

import groovyx.net.http.HTTPBuilder
import groovyx.net.http.ContentType

http.get(path : path, contentType : ContentType.TEXT) { resp, reader ->
    def relativePaths = new XmlParser().parse(reader)    
    ...
}

Note: I'm not sure whether the get() method runs in the background or not.

Updated solution

Setting the content type to XML instructs HTTPBuilder to parse the response and provide the parsed document as the second argument. The document is represented as a Node.

http.get(path : path, contentType : ContentType.XML) { response, node ->
    def relativePaths = node    
    ...
}

edited Mar 04 '16 at 15:23

answered Mar 03 '16 at 21:50

Emmanuel Rosa

9,697
2
14
20

Thanks ! Face palm, indeed. Now I'm getting a 406. Maybe I'm just tired ;-) – daniel.lozynski Mar 03 '16 at 22:34
The content type may be wrong. It's probably supposed to be ContentType.XML. It's my turn to face palm! – Emmanuel Rosa Mar 03 '16 at 22:50
Oh, the url given to `http.get()` is actually supposed to be a path relative to the url given to `HTTPBuilder`. For example, the url might be *http://www.google.com* and the path */search*. – Emmanuel Rosa Mar 03 '16 at 22:55
Thanks! I changed those things, but it still doesn't seem to work. I posted the Ruby code above, just realized I used xpath there. – daniel.lozynski Mar 04 '16 at 06:33
The problem is no longer with HTTPBuilder. It's authenticating and providing the response. The stack trace provided a clue: *Parsed data to instance of: class groovy.util.slurpersupport.NodeChild Caught: groovy.lang.MissingMethodException: No signature of method: groovy.util.XmlParser.parse() is applicable for argument types: (groovy.util.slurpersupport.NodeChild)*. It turns out that HTTPBuilder takes the content type into account when providing the response. In this case, it's parsing the XML document on your behalf :) So *stream* is really a *Node*. I updated my answer to demonstrate this. – Emmanuel Rosa Mar 04 '16 at 15:27

HTTP authentication in Groovy?

1 Answers1

Solution

Updated solution