Creating JClouds ComputeService to AWS many times throws an "Error in custom provider" exception

Question

I'm developing a system that makes use from jclouds to instantiate a bunch of nodes on Amazon cloud.

When I need to know the nodes I've already created, I use the computeService.listNodes(). To each invocation of listNodes, I create an object computeService, as following:

Properties overrides = new Properties();
overrides.setProperty(AWSEC2Constants.PROPERTY_EC2_AMI_QUERY, "image-id=" + image);

ComputeServiceContext context = new ComputeServiceContextFactory()
.createContext(PROVIDER,
  Configuration.get("AMAZON_ACCESS_KEY_ID"),
  Configuration.get("AMAZON_SECRET_KEY"),
  ImmutableSet.<Module> of(), overrides);

computeService = context.getComputeService();

Usually it works, but when I got multiple concurrent invocations to computeService.listNodes() the execution fails.

In such executions, I got exceptions in some threads in the createContext line. The full stack trace is attached.

What puzzles me is that I've written a little program with a loop to instantiate multiple computeServices, and I got no trouble.

Would I making a wrong use form JClouds API? Does not JClouds supports concurrent invocations? Or would it be some kind of limit to AWS invocations? (the previous paragraph suggests the answer to these questions is "no", but I cannot think in other options now).

Important information: my code runs from REST invocations to a standalone local server set up with apache cxf.

This is the class responsible to set up the server: https://github.com/choreos/choreos_middleware/blob/master/ServiceDeployer/src/main/java/org/ow2/choreos/npm/rest/NPMServer.java.

And here is the class that receives invocations: https://github.com/choreos/choreos_middleware/blob/master/ServiceDeployer/src/main/java/org/ow2/choreos/npm/rest/NodesResource.java

Some relevant parts of the stack trace:

1) Error in custom provider, java.lang.reflect.UndeclaredThrowableException
  while locating org.jclouds.ec2.config.EC2RestClientModule$RegionIdToZoneId
    Caused by: java.lang.reflect.UndeclaredThrowableException
    at $Proxy98.describeAvailabilityZonesInRegion(Unknown Source)
    at     org.jclouds.ec2.config.EC2RestClientModule$RegionIdToZoneId.get(EC2RestClientModule.java:152)
Caused by: java.util.concurrent.TimeoutException
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:258)
at java.util.concurrent.FutureTask.get(FutureTask.java:119)
at     org.jclouds.concurrent.config.ExecutorServiceModule$AddToStringFuture.get(ExecutorServiceModule.java:234)

2) Error in custom provider, org.jclouds.http.HttpResponseException: ec2.ap-northeast-1.amazonaws.com connecting to POST https://ec2.ap-northeast-1.amazonaws.com/ HTTP/1.1
  while locating org.jclouds.ec2.config.EC2RestClientModule$RegionIdToZoneId
Caused by: org.jclouds.http.HttpResponseException: ec2.ap-northeast-1.amazonaws.com connecting to POST https://ec2.ap-northeast-1.amazonaws.com/ HTTP/1.1
at     org.jclouds.http.internal.BaseHttpCommandExecutorService$HttpResponseCallable.call(BaseHttpCommandExecutorService.java:183)
Caused by: java.net.UnknownHostException: ec2.ap-northeast-1.amazonaws.com
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:175)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:384)

OBS: the "at org.ow2.choreos.npm.cloudprovider.AWSCloudProvider.getClient(AWSCloudProvider.java:53)" line in stack trace is the "createContext" line.

Thank you, Leonardo Leite

score 1 · Answer 1 · answered Nov 16 '12 at 20:52

nodes query in jclouds is not cached, so multiple concurrent requests may lead to network timeouts. You might consider using a guava cache if you are doing many listNodes simultaneously.

Regardless, I'd bump to jclouds 1.5.3 which also has a cleaner syntax for creating connections.

ComputeServiceContext context = ContextBuilder.newBuilder(PROVIDER)
                                              .credentials(Configuration.get("AMAZON_ACCESS_KEY_ID"), Configuration.get("AMAZON_SECRET_KEY"))
                                              .overrides(properties)
                                              .buildView(ComputeServiceContext.class);

Finally, you can isolate the regions you work with by supplying the context property "jclouds.regions" with the comma-separated list of regions you are interested in. For example "us-west-1", will prevent network calls to remote regions.

Creating JClouds ComputeService to AWS many times throws an "Error in custom provider" exception

1 Answers1