Im getting a not so rare error, which happens to many, the thing is that im not using threads in my app. Its just a loop, one task after the other, so i dont understand why im getting out of threads.
Basically for each line of the text file i make a GET request to an api, so i obtain some json data...
Here is the error log:
106
105
104
103
102
ene 06, 2016 4:08:39 PM org.jboss.netty.channel.DefaultChannelFuture
ADVERTENCIA: An exception was thrown by ChannelFutureListener.
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:714)
at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950)
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1368)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.start(AbstractNioWorker.java:160)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.executeInIoThread(AbstractNioWorker.java:306)
at org.jboss.netty.channel.socket.nio.NioWorker.executeInIoThread(NioWorker.java:38)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.executeInIoThread(AbstractNioWorker.java:290)
at org.jboss.netty.channel.socket.nio.NioWorker.executeInIoThread(NioWorker.java:38)
at org.jboss.netty.channel.socket.nio.AbstractNioChannelSink.execute(AbstractNioChannelSink.java:34)
at org.jboss.netty.channel.Channels.fireExceptionCaughtLater(Channels.java:504)
at org.jboss.netty.channel.AbstractChannelSink.exceptionCaught(AbstractChannelSink.java:47)
at org.jboss.netty.channel.Channels.close(Channels.java:821)
at org.jboss.netty.channel.AbstractChannel.close(AbstractChannel.java:194)
at org.jboss.netty.channel.ChannelFutureListener$2.operationComplete(ChannelFutureListener.java:52)
at org.jboss.netty.channel.DefaultChannelFuture.notifyListener(DefaultChannelFuture.java:399)
at org.jboss.netty.channel.DefaultChannelFuture.notifyListeners(DefaultChannelFuture.java:385)
at org.jboss.netty.channel.DefaultChannelFuture.setFailure(DefaultChannelFuture.java:352)
at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:404)
at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:361)
at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:277)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
ERROR en getML():/orders/1045232034
com.mercadolibre.sdk.MeliException: java.util.concurrent.ExecutionException: java.net.ConnectException: unable to create new native thread to https://api.mercadolibre.com/orders/1045232034?access_token=APP_USR-1978251743308541-010613-c60e34a2a34d0fd5b8c936ccfb3e3732__K_I__-113746522
at com.nubimedia.app.ml.Melis.get(Melis.java:264)
at com.nubimedia.app.ml.Melis.getML(Melis.java:168)
at com.nubimedia.app.ModelML.encontrarMailsLog(ModelML.java:217)
at com.nubimedia.app.Pruebas.main(Pruebas.java:11)
Caused by: java.util.concurrent.ExecutionException: java.net.ConnectException: unable to create new native thread to https://api.mercadolibre.com/orders/1045232034?access_token=APP_USR-1978251743308541-010613-c60e34a2a34d0fd5b8c936ccfb3e3732__K_I__-113746522
at com.ning.http.client.providers.netty.NettyResponseFuture.abort(NettyResponseFuture.java:297)
at com.ning.http.client.providers.netty.NettyConnectListener.operationComplete(NettyConnectListener.java:104)
at org.jboss.netty.channel.DefaultChannelFuture.notifyListener(DefaultChannelFuture.java:399)
at org.jboss.netty.channel.DefaultChannelFuture.notifyListeners(DefaultChannelFuture.java:390)
at org.jboss.netty.channel.DefaultChannelFuture.setFailure(DefaultChannelFuture.java:352)
at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:404)
at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:361)
at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:277)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.ConnectException: unable to create new native thread to https://api.mercadolibre.com/orders/1045232034?access_token=APP_USR-1978251743308541-010613-c60e34a2a34d0fd5b8c936ccfb3e3732__K_I__-113746522
at com.ning.http.client.providers.netty.NettyConnectListener.operationComplete(NettyConnectListener.java:100)
... 10 more
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:714)
at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950)
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1368)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.start(AbstractNioWorker.java:160)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.register(AbstractNioWorker.java:131)
at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:401)
at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:361)
at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:277)
... 3 more
Hubo un error
101
Exception in thread "main" java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:714)
at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950)
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1368)
at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:134)
at com.ning.http.client.providers.jdk.JDKAsyncHttpProvider.execute(JDKAsyncHttpProvider.java:159)
at com.ning.http.client.providers.jdk.JDKAsyncHttpProvider.execute(JDKAsyncHttpProvider.java:121)
at com.ning.http.client.AsyncHttpClient.executeRequest(AsyncHttpClient.java:512)
at com.ning.http.client.AsyncHttpClient$BoundRequestBuilder.execute(AsyncHttpClient.java:234)
at com.nubimedia.app.ml.Melis.get(Melis.java:262)
at com.nubimedia.app.ml.Melis.getML(Melis.java:168)
at com.nubimedia.app.ModelML.encontrarMailsLog(ModelML.java:217)
at com.nubimedia.app.Pruebas.main(Pruebas.java:11)
And here is the code:
public static void main(String[] args) throws IOException, MeliException {
ModelML.getInstance().encontrarMailsLog(731);
}
public void encontrarMailsLog(int max) throws IOException{
BufferedReader br = new BufferedReader(new FileReader("/Users/fabrizioguespe/Downloads/catalina.out"));
Set<String> resources=new TreeSet<String>();
try {
StringBuilder sb = new StringBuilder();
String line = br.readLine();
while (line != null) {
if(line.equals("Nueva Request!")){
String cliente=br.readLine();
String resource=br.readLine();
resources.add(cliente+"-"+resource);
}
sb.append(line);
sb.append(System.lineSeparator());
line = br.readLine();
}
} finally {
br.close();
}
Log.log(resources.size()+" Para procesar");
int i=resources.size();
for(String s: resources){
i--;
Log.log(i+"");
if(max>0)
if(i>max)continue;
String cliente=s.split("-")[0];
String resource=s.split("-")[1];
//Log.log("Cliente: "+cliente+" "+resource);
ModelML.getInstance().procesarRequest(Melis.getML(resource, null,cliente,false),resource);
}
}
UPDATED: As suggested, and makes sense. I may be doing async request, using the SDK provided by the API
Here is the class
import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.net.URLEncoder;
import java.util.Date;
import java.util.List;
import java.util.concurrent.ExecutionException;
import org.hibernate.Query;
import org.hibernate.Session;
import com.google.gson.JsonObject;
import com.google.gson.JsonParser;
import com.google.gson.JsonSyntaxException;
import com.mercadolibre.sdk.AuthorizationFailure;
import com.mercadolibre.sdk.MeliException;
import com.ning.http.client.AsyncHttpClient;
import com.ning.http.client.AsyncHttpClient.BoundRequestBuilder;
import com.ning.http.client.AsyncHttpClientConfig;
import com.ning.http.client.FluentStringsMap;
import com.ning.http.client.Response;
import com.nubimedia.app.ModelML;
import com.nubimedia.app.util.HibernateUtil;
import com.nubimedia.app.util.Log;
import com.sun.jersey.api.client.Client;
import com.sun.jersey.api.client.ClientResponse;
import com.sun.jersey.api.client.WebResource;
public class Melis {
public static String apiUrl = "https://api.mercadolibre.com";
private String accessToken;
private String refreshToken;
private Long clientId;
private String clientSecret;
//public static Map<String,UserML> clientes=new HashMap<String,UserML>();
public static UserML getCliente(String id) {//Solo para inicializar la aplicacion
Session s=HibernateUtil.getSessionFactory().openSession();
s.beginTransaction();
Query query = s.createQuery("from UserML where id = :id");
query.setString("id", id);
UserML q = (UserML) query.uniqueResult();
s.flush();
s.getTransaction().commit();
s.close();
return q;
}
public static List<UserML> getClientes() {
Session s=HibernateUtil.getSessionFactory().openSession();
s.beginTransaction();
Query query = s.createQuery("from UserML");
@SuppressWarnings("unchecked")
List<UserML> q = (List<UserML>) query.list();
s.flush();
s.getTransaction().commit();
s.close();
return q;
}
public static String preguntarML(String item,String pregunta,String idcliente){
UserML cliente=getCliente(idcliente);
if(cliente==null)return null;
Melis m = new Melis(Long.parseLong(ModelML.ml_appid),ModelML.ml_secret,cliente.getToken(),cliente.getToken());
String params="{\"text\":\""+pregunta.trim()+"\",\"item_id\":\""+item+"\"}";
String json=Melis.postML("/questions/"+item+"?access_token="+m.getAccessToken(), null, cliente, params);
return json;
}
public static String postML(String resource,FluentStringsMap params,UserML cliente,String json){
try {
Melis m = new Melis(Long.parseLong(ModelML.ml_appid),ModelML.ml_secret,cliente.getToken(),cliente.getRefresh_token());
if(params==null){
params = new FluentStringsMap();
params.add("access_token", m.getAccessToken());
}
Response respuesta = m.post(resource, params,json);
String jsons=respuesta.getResponseBody();
//jsons=new String(json.getBytes("ISO-8859-1"),"UTF-8");
return jsons;
} catch (Exception e) {
Log.log("ERROR en getML():"+resource);
e.printStackTrace();
}
return null;
}
public static String getUserId(String json){
try{
JsonParser parser = new JsonParser();
JsonObject venta= (JsonObject)parser.parse(json);
if(venta.isJsonNull())return null;
return venta.get("id").getAsString();
}catch(Exception e){
e.printStackTrace();
Log.log("ERROR al parsear json VENTA mercadolibre: ");
}
return null;
}
public static String setToken(String token,String refresh_token,String idcliente) {
UserML a=getCliente(idcliente);
if(a==null)return null;
a.setToken(token);
a.setRefresh_token(refresh_token);
a.setLast_modif(new Date());
Session s=HibernateUtil.getSessionFactory().openSession();
s.beginTransaction();
s.saveOrUpdate(a);
s.flush();
s.getTransaction().commit();
s.close();
return null;
}
public static String getML2(String resource){
Client client = Client.create();
WebResource webResource = client.resource(apiUrl+resource);
ClientResponse response = webResource.accept("application/json").get(ClientResponse.class);
if (response.getStatus() != 200)
throw new RuntimeException("Failed : HTTP error code : " + response.getStatus());
return response.getEntity(String.class);
}
public static String remokeAccess(String idcliente){
try {
UserML cliente=getCliente(idcliente);
if(cliente==null)return null;
String resource="/users/"+cliente+"/applications/"+ModelML.ml_appid+"?access_token="+cliente.getToken();
Melis m = new Melis(Long.parseLong(ModelML.ml_appid),ModelML.ml_secret,cliente.getToken());
FluentStringsMap params = new FluentStringsMap();
params.add("access_token", m.getAccessToken());
Response respuesta = m.delete(resource, params);
String json=respuesta.getResponseBody();
json=new String(json.getBytes("ISO-8859-1"),"UTF-8");
return json;
} catch (Exception e) {
Log.log("ERROR en removkeAccess():"+idcliente);
e.printStackTrace();
}
return null;
}
public static String getML(String resource,FluentStringsMap params,String idcliente,boolean access_final){
try {
UserML cliente=getCliente(idcliente);
if(cliente==null)return null;
Melis m = new Melis(Long.parseLong(ModelML.ml_appid),ModelML.ml_secret,cliente.getToken(),cliente.getRefresh_token());
if(params==null){
params = new FluentStringsMap();
params.add("access_token", m.getAccessToken());
}
if(access_final)resource+="?access_token="+cliente.getToken();
Response respuesta = m.get(resource, params);
String json=respuesta.getResponseBody();
json=new String(json.getBytes("ISO-8859-1"),"UTF-8");
return json;
} catch (Exception e) {
Log.log("ERROR en getML():"+resource);
e.printStackTrace();
}
return null;
}
private AsyncHttpClient http;
{
AsyncHttpClientConfig cf = new AsyncHttpClientConfig.Builder()
.setUserAgent("MELIJAVASDK0.0.1").build();
http = new AsyncHttpClient(cf);
}
public Melis(Long clientId, String clientSecret ) {
this.clientId = clientId;
this.clientSecret = clientSecret;
}
public Melis(Long clientId, String clientSecret, String accessToken) {
this.accessToken = accessToken;
this.clientId = clientId;
this.clientSecret = clientSecret;
}
public Melis(Long clientId, String clientSecret, String accessToken, String refreshToken) {
this.accessToken = accessToken;
this.clientId = clientId;
this.clientSecret = clientSecret;
this.refreshToken = refreshToken;
}
public String getAccessToken() {
return this.accessToken;
}
public Response get(String path) throws MeliException {
return get(path, new FluentStringsMap());
}
private BoundRequestBuilder prepareGet(String path, FluentStringsMap params) {
return http.prepareGet(apiUrl + path)
.addHeader("Accept", "application/json")
.setQueryParameters(params);
}
private BoundRequestBuilder prepareDelete(String path,
FluentStringsMap params) {
return http.prepareDelete(apiUrl + path)
.addHeader("Accept", "application/json")
.setQueryParameters(params);
}
private BoundRequestBuilder preparePost(String path,
FluentStringsMap params, String body) {
return http.preparePost(apiUrl + path)
.addHeader("Accept", "application/json")
.setQueryParameters(params)
.setHeader("ContentType", "application/json").setBody(body)
.setBodyEncoding("UTF8");
}
private BoundRequestBuilder preparePut(String path,
FluentStringsMap params, String body) {
return http.preparePut(apiUrl + path)
.addHeader("Accept", "application/json")
.setQueryParameters(params)
.setHeader("ContentType", "application/json").setBody(body)
.setBodyEncoding("UTF8");
}
private BoundRequestBuilder preparePost(String path, FluentStringsMap params) {
return http.preparePost(apiUrl + path)
.addHeader("Accept", "application/json")
.setQueryParameters(params);
}
public Response get(String path, FluentStringsMap params) throws MeliException {
BoundRequestBuilder r = prepareGet(path, params);
Response response;
try {
response = r.execute().get();
} catch (Exception e) {
throw new MeliException(e);
}
if (params.containsKey("access_token") && this.hasRefreshToken() && response.getStatusCode() == 401) {
try {
refreshAccessToken();
} catch (AuthorizationFailure e1) {
Log.log("ERROR al refrescar token");
return response;
}
params.replace("access_token", this.accessToken);
r = prepareGet(path, params);
try {
response = r.execute().get();
} catch (Exception e) {
throw new MeliException(e);
}
}
return response;
}
public void refreshAccessToken() throws AuthorizationFailure {
FluentStringsMap params = new FluentStringsMap();
params.add("grant_type", "refresh_token");
params.add("client_id", String.valueOf(this.clientId));
params.add("client_secret", this.clientSecret);
params.add("refresh_token", this.refreshToken);
BoundRequestBuilder req = preparePost("/oauth/token", params);
parseToken(req);
}
public String getAuthUrl(String callback) {
try {
return "https://auth.mercadolibre.com.ar/authorization?response_type=code&client_id="
+ this.clientId
+ "&redirect_uri="
+ URLEncoder.encode(callback, "UTF8");
} catch (UnsupportedEncodingException e) {
return "https://auth.mercadolibre.com.ar/authorization?response_type=code&client_id="
+ this.clientId + "&redirect_uri=" + callback;
}
}
public String authorize(String code, String redirectUri) throws AuthorizationFailure {
FluentStringsMap params = new FluentStringsMap();
params.add("grant_type", "authorization_code");
params.add("client_id", String.valueOf(this.clientId));
params.add("client_secret", this.clientSecret);
params.add("code", code);
params.add("redirect_uri", redirectUri);
BoundRequestBuilder r = preparePost("/oauth/token", params);
return parseToken(r);
}
private String parseToken(BoundRequestBuilder r) throws AuthorizationFailure {
Response response = null;
String responseBody = "";
try {
response = r.execute().get();
responseBody = response.getResponseBody();
} catch (InterruptedException e) {
throw new AuthorizationFailure(e);
} catch (ExecutionException e) {
throw new AuthorizationFailure(e);
} catch (IOException e) {
throw new AuthorizationFailure(e);
}
JsonParser p = new JsonParser();
JsonObject object;
try {
object = p.parse(responseBody).getAsJsonObject();
} catch (JsonSyntaxException e) {
throw new AuthorizationFailure(responseBody);
}
if (response.getStatusCode() == 200) {
this.accessToken = object.get("access_token").getAsString();
this.refreshToken = object.get("refresh_token").getAsString();
String user_ID = object.get("user_id").getAsString();
Melis.setToken(this.accessToken, this.refreshToken,user_ID);//PERSISTO
return object.toString();
} else return object.get("message").getAsString();
}
private boolean hasRefreshToken() {
return this.refreshToken != null && !this.refreshToken.isEmpty();
}
public Response post(String path, FluentStringsMap params, String body)
throws MeliException {
BoundRequestBuilder r = preparePost(path, params, body);
Response response;
try {
response = r.execute().get();
} catch (Exception e) {
throw new MeliException(e);
}
if (params.containsKey("access_token") && this.hasRefreshToken() && response.getStatusCode() == 401) {
try {
refreshAccessToken();
} catch (AuthorizationFailure e1) {
return response;
}
params.replace("access_token", this.accessToken);
r = preparePost(path, params, body);
try {
response = r.execute().get();
} catch (Exception e) {
throw new MeliException(e);
}
}
return response;
}
public Response put(String path, FluentStringsMap params, String body)
throws MeliException {
BoundRequestBuilder r = preparePut(path, params, body);
Response response;
try {
response = r.execute().get();
} catch (Exception e) {
throw new MeliException(e);
}
if (params.containsKey("access_token") && this.hasRefreshToken()
&& response.getStatusCode() == 401) {
try {
refreshAccessToken();
} catch (AuthorizationFailure e1) {
return response;
}
params.replace("access_token", this.accessToken);
r = preparePut(path, params, body);
try {
response = r.execute().get();
} catch (Exception e) {
throw new MeliException(e);
}
}
return response;
}
public Response delete(String path, FluentStringsMap params)
throws MeliException {
BoundRequestBuilder r = prepareDelete(path, params);
Response response;
try {
response = r.execute().get();
} catch (Exception e) {
throw new MeliException(e);
}
if (params.containsKey("access_token") && this.hasRefreshToken()
&& response.getStatusCode() == 401) {
try {
refreshAccessToken();
} catch (AuthorizationFailure e1) {
return response;
}
params.replace("access_token", this.accessToken);
r = prepareDelete(path, params);
try {
response = r.execute().get();
} catch (Exception e) {
throw new MeliException(e);
}
}
return response;
}
public BoundRequestBuilder head(String path) {
return null;
}
public BoundRequestBuilder options(String path) {
return null;
}
}
