Exception when parsing Json response from Swift api

Question

I have some files uploaded in openstacks' object store. This snippet fetches container and object list and prints.

private void listContainers() {
      ContainerApi containerApi = swiftApi.getContainerApiForRegion("region");
      Set<Container> containers = containerApi.list().toSet();

      for (Container container : containers) {
          ObjectApi objectApi = swiftApi.getObjectApiForRegionAndContainer("region", container.getName());

          ObjectList objects = objectApi.list();  // crashes here
          for (SwiftObject object: objects) {
             System.out.println("\t\t"+ object);
          }

          System.out.println("\t" + container);
      }
   }

console output:

DEBUG o.j.rest.internal.InvokeHttpMethod - >> invoking container:list
DEBUG o.j.h.i.JavaUrlHttpCommandExecutorService - Sending request -363056976: GET http://127.0.0.1/swift/v1/?format=json HTTP/1.1
DEBUG jclouds.headers - >> GET http://127.0.0.1/swift/v1/?format=json HTTP/1.1
DEBUG jclouds.headers - >> Accept: application/json
DEBUG jclouds.headers - >> X-Auth-Token: MIIQ...
DEBUG o.j.h.i.JavaUrlHttpCommandExecutorService - Receiving response -363056976: HTTP/1.1 200 OK
DEBUG jclouds.headers - << HTTP/1.1 200 OK
DEBUG jclouds.headers - << Transfer-Encoding: chunked
DEBUG jclouds.headers - << Date: Tue, 25 Nov 2014 07:39:44 GMT
DEBUG jclouds.headers - << Keep-Alive: timeout=5, max=98
DEBUG jclouds.headers - << Connection: Keep-Alive
DEBUG jclouds.headers - << Server: Apache/2.2.22 (Ubuntu)
DEBUG jclouds.headers - << Content-Type: application/json; charset=utf-8
DEBUG jclouds.wire - << "[{"name":"jclouds-example","count":1,"bytes":12},{"name":"test_name","count":3,"bytes":22008217}]"
DEBUG o.j.rest.internal.InvokeHttpMethod - >> invoking object:list
DEBUG o.j.h.i.JavaUrlHttpCommandExecutorService - Sending request -1472717862: GET http://127.0.0.1/swift/v1/jclouds-example/?format=json HTTP/1.1
DEBUG jclouds.headers - >> GET http://127.0.0.1/swift/v1/jclouds-example/?format=json HTTP/1.1
DEBUG jclouds.headers - >> Accept: application/json
DEBUG jclouds.headers - >> X-Auth-Token: MIIQ...
DEBUG o.j.h.i.JavaUrlHttpCommandExecutorService - Receiving response -1472717862: HTTP/1.1 200 OK
DEBUG jclouds.headers - << HTTP/1.1 200 OK
DEBUG jclouds.headers - << Transfer-Encoding: chunked
DEBUG jclouds.headers - << Date: Tue, 25 Nov 2014 07:39:44 GMT
DEBUG jclouds.headers - << Keep-Alive: timeout=5, max=97
DEBUG jclouds.headers - << Connection: Keep-Alive
DEBUG jclouds.headers - << Server: Apache/2.2.22 (Ubuntu)
DEBUG jclouds.headers - << Content-Type: application/json; charset=utf-8
DEBUG jclouds.wire - << "[{"name":"jclouds-example.txt","hash":"ed076287532e86365e841e92bfc50d8c","bytes":12,"content_type":"application\/unknown","last_modified":"2014-11-25T07:39:44.000Z"}]"
java.lang.NumberFormatException: null
    at java.lang.Long.parseLong(Long.java:404)
    at java.lang.Long.parseLong(Long.java:483)
    at org.jclouds.openstack.swift.v1.functions.ParseContainerFromHeaders.apply(ParseContainerFromHeaders.java:39)
    at org.jclouds.openstack.swift.v1.functions.ParseObjectListFromResponse.apply(ParseObjectListFromResponse.java:66)
    at org.jclouds.openstack.swift.v1.functions.ParseObjectListFromResponse.apply(ParseObjectListFromResponse.java:41)
    at org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:90)
    at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:73)
    at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:44)
    at org.jclouds.reflect.FunctionalReflection$FunctionalInvocationHandler.handleInvocation(FunctionalReflection.java:117)
    at com.google.common.reflect.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:87)
    at com.sun.proxy.$Proxy66.list(Unknown Source)
    at test.jcloud.JCloudsSwift.listContainers(JCloudsSwift.java:99)
    at test.jcloud.JCloudsSwift.main(JCloudsSwift.java:39)

If I specify file name and fetch its' metadata, everything works fine:

...
for (Container container : containers) {
    ObjectApi objectApi = swiftApi.getObjectApiForRegionAndContainer("region", container.getName());
    SwiftObject obj = objectApi.get("jclouds-example.txt");
    System.out.println("--  "+obj.getMetadata());
    ObjectList objects = objectApi.list();
...

console output:

DEBUG o.j.rest.internal.InvokeHttpMethod - >> invoking object:get
DEBUG o.j.h.i.JavaUrlHttpCommandExecutorService - Sending request -614124758: GET http://127.0.0.1/swift/v1/jclouds-example/jclouds-example.txt HTTP/1.1
DEBUG jclouds.headers - >> GET http://127.0.0.1/swift/v1/jclouds-example/jclouds-example.txt HTTP/1.1
DEBUG jclouds.headers - >> Accept: application/json
DEBUG jclouds.headers - >> X-Auth-Token: MIIQ...
DEBUG o.j.h.i.JavaUrlHttpCommandExecutorService - Receiving response -614124758: HTTP/1.1 200 OK
DEBUG jclouds.headers - << HTTP/1.1 200 OK
DEBUG jclouds.headers - << etag: ed076287532e86365e841e92bfc50d8c
DEBUG jclouds.headers - << Date: Tue, 25 Nov 2014 07:48:49 GMT
DEBUG jclouds.headers - << Last-Modified: Tue, 25 Nov 2014 07:48:49 GMT
DEBUG jclouds.headers - << Keep-Alive: timeout=5, max=97
DEBUG jclouds.headers - << X-Object-Meta-key4: blu
DEBUG jclouds.headers - << X-Object-Meta-key3: value3
DEBUG jclouds.headers - << Connection: Keep-Alive
DEBUG jclouds.headers - << Accept-Ranges: bytes
DEBUG jclouds.headers - << Server: Apache/2.2.22 (Ubuntu)
DEBUG jclouds.headers - << Content-Type: application/unknown
DEBUG jclouds.headers - << Content-Length: 12
DEBUG jclouds.wire - << "Hello World!"
--  {key4=blu, key3=value3}

The code is based on this example and getting object info on this example

Problem

I get NumberFormatException when jclouds converts json data containing object list to java object.

Question

How can I get the file list for each container and get their metadata?

score 1 · Answer 1 · answered Dec 03 '14 at 20:55

1

I just tried this on OpenStack Juno with jclouds 1.8.1 and it's working for me. What versions of OpenStack and jclouds are you using?

I also noticed the object list response is missing the X-Container-Object-Count and X-Container-Bytes-Used headers. That's odd, they should be there according to this api reference. Is your Apache server maybe manipulating or filtering headers?

If you want to try jclouds 1.8.1, you'll want to read these release notes first.

answered Dec 03 '14 at 20:55

Everett Toews

10,337
10
44
45

I'm using jclouds 1.8.1 and openstack 2014.1 – gkiko Dec 04 '14 at 06:59
Swift really should be retuning the X-Container-Object-Count and X-Container-Bytes-Used headers, they've been around since long before 2014.1 (see [server.py](https://github.com/openstack/swift/blob/fbc58b082cf93eb142c038f24e75ec01e6005894/swift/container/server.py#L62-70)). Is your Apache server maybe manipulating or filtering headers? – Everett Toews Dec 04 '14 at 20:36
Also, how did you get the request to end with ?format=json `GET http://127.0.0.1/swift/v1/jclouds-example/?format=json`? There's no way to do that when calling ObjectApi.list() in 1.8.1 (see [ObjectApi.java](https://github.com/jclouds/jclouds/blob/master/apis/openstack-swift/src/main/java/org/jclouds/openstack/swift/v1/features/ObjectApi.java#L79-84)). – Everett Toews Dec 04 '14 at 20:39
I downloaded the source of jclouds 1.8.1 and it differs from the github copy. For example `ObjectApi` method `ObjectList list();` has tag __@QueryParams(keys = "format", values = "json")__ – gkiko Dec 05 '14 at 10:43
Can you please send a link to *exactly* what you downloaded? – Everett Toews Dec 05 '14 at 18:06
I use maven example from jclouds [website](https://jclouds.apache.org/start/install/). The sources were downloaded via maven – gkiko Dec 06 '14 at 05:49
If that's the case, the canonical sources are coming from [sources.jar](https://repository.apache.org/service/local/artifact/maven/redirect?r=releases&g=org.apache.jclouds.api&a=openstack-swift&v=1.8.1&e=jar&c=sources) on repository.apache.org (also in [sources.jar](http://search.maven.org/remotecontent?filepath=org/apache/jclouds/api/openstack-swift/1.8.1/openstack-swift-1.8.1-sources.jar) on Maven Central). Both agree with GitHub [ObjectApi.java](https://github.com/jclouds/jclouds/blob/jclouds-1.8.1/apis/openstack-swift/src/main/java/org/jclouds/openstack/swift/v1/features/ObjectApi.java). – Everett Toews Dec 08 '14 at 03:00
I'd love to know about that discrepancy too, as I think that's relevant for an issue I'm having where JClouds will not add that param to the querystring and therefore Swift will return the answer in plain/text format, so I'll get an error : http://stackoverflow.com/questions/35439056/jclouds-and-openstack-illegalstateexception-expected-begin-array-but-was-stri – krause Feb 18 '16 at 16:39

Exception when parsing Json response from Swift api

1 Answers1