Nexus3 only downloads 50 in list

Question

So I have a code that downloads the version nr of each nuget package but it all stops after 50 in list.

I use jenkins with groovy code and get out a list of versions.

import groovy.json.JsonSlurperClassic 
import groovy.json.JsonBuilder
import wslite.rest.*


 def data = new URL("http://nexus.xx.xx.se:8081/service/rest/v1/search?repository=xx-sx-nuget&name=XXXFrontend").getText()  
 println data


/**
 * 'jsonString' is the input json you have shown
 * parse it and store it in collection
 */
 Map convertedJSONMap = new JsonSlurperClassic().parseText(data)


//If you have the nodes then fetch the first one only
if(convertedJSONMap."items"){

     println "Version : " + convertedJSONMap."items"[0]."version"
 }   

def list = convertedJSONMap.items.version
Collections.sort(list)


list

So the problem is that it only get 50 of the versions. How can I get more than 50? I have read about a continuetoken but I dont understand how to use that?

UPDATE

I have added this but still dont work

while(convertedJSONMap."continuesToken" != null){
def token =  convertedJSONMap."continuationToken"
def data2 = new URL("http://nexus.xxx.xxx.se:8081/service/rest/v1/search?repository=xxx-xx-nuget&name=xxxxxx&continuationToken=" +token).getText() 
 convertedJSONMap = JsonSlurperClassic().parseText(data2)

}

See here: https://help.sonatype.com/display/NXRM3/Pagination — rseddon, Jun 19 '19 at 20:20
Check the nexus request.log to see if your script is actually sending the continuation token correctly. — rseddon, Jun 25 '19 at 20:09
How many componenets do you have in your repository? Now, after your updated script you are getting two result sets which at most will contains up to a 100 components. To make sure you get all of them, you have to keep calling the endpoint until it returns an empty result. Keep in mind that after each request you receive a new continuation token that you should use. — Dawid Sawa, Jun 29 '19 at 11:23

score 0 · Answer 1 · answered Jul 08 '19 at 10:56

This is how I solved it for me. It is just a snippet of the code that I use

def json = sendRequest(url)

addResultToMap(map2, json, release) //I do something here with the received result

def continuationToken = json.continuationToken
if (continuationToken != null) {
    while (continuationToken != null) {
        json = sendRequest(url + "&continuationToken=" + continuationToken)
        addResultToMap(map2, json, release) //I do something here with the received result as above
        continuationToken = json.continuationToken
    }
}

And my sendRequest method looks like this

def sendRequest(def url, String method = "GET") {
    String userPass = "${nexus.username}:${nexus.password}"
    String basicAuth = "Basic " + "${printBase64Binary(userPass.getBytes())}"

    def connection = new URL( url ).openConnection() as HttpURLConnection
    connection.setRequestProperty('Accept', 'application/json' )
    connection.setRequestProperty('Authorization', basicAuth)
    connection.setRequestMethod(method)

    try {
        if ( connection.responseCode <= 299 ) {
            if (connection.responseCode == 200) {
                return connection.inputStream.withCloseable { inStream -> new JsonSlurper().parse( inStream as InputStream ) }
            }
        } else {
            displayAndLogError(connection.responseCode + ": " + connection.inputStream.text, loglevel.DEBUG)
        }
    } catch(Exception exc) {
        displayAndLogError(exc.getMessage())
    }
}

score 0 · Answer 2 · answered Aug 19 '19 at 15:38

Here is an alternative :

import groovy.json.JsonSlurper

try {
    N_PAGES_MAX = 10
    List<String> versions = new ArrayList<String>()
    continuationToken = "123"
    artifactsUrl = "http://nexus.zzz.local/service/rest/v1/components?repository=releases-super-project"
    currentPage = 1

    while (true) {

        artifactsObjectRaw = ["curl", "-s", "-H", "accept: application/json", "-k", "--url", "${artifactsUrl}"].execute().text
        artifactsJsonObject = (new JsonSlurper()).parseText(artifactsObjectRaw)
        continuationToken = artifactsJsonObject.continuationToken

        if (continuationToken!=null && continuationToken!='123') {
            artifactsUrl = artifactsUrl + "&continuationToken=$continuationToken"
        }

        def items = artifactsJsonObject.items
        for(item in items){
            versions.add(item.name)
        }

        currentPage += 1
        if (continuationToken==null || currentPage>N_PAGES_MAX) break
    }

    return versions.sort().reverse()
}

catch (Exception e) {
    print "There was a problem fetching the versions"
}

Nexus3 only downloads 50 in list

2 Answers2