How to use Jsoup with Volley?

Question

I have a working example with Jsoup and AsyncTask, and that works fine. I am just not satisfied with the performance. It takes 3-6 seconds to load a simple list page with text and images.

I want to boost the performance somehow... so I have stumbled on volley.

Can anyone explain hot to use volley with jsoup?

I use this to get the doc object that holds the particular URL:

 public Document GetDocument(String site) {      
        Document doc = Jsoup.connect(site).timeout(600000)
        .data("query", "Java")
        .userAgent("Mozilla")
        .get();

        return doc;
 }

I guess I would only analyze the data with jsoup and connect/download with volley? Whene I use Jsoup.connect(site).timeout(600000) I should do that with volley ?

Can anyone write/link a simple example using volley and jsoup?

Answer 1

Can anyone write/link a simple example using volley and jsoup?

Under the hood, Jsoup make use of HttpUrlConnection. This class has known unresolved issues, bugs and performance issues on the Android Platform.

Instead, load the data with Volley first then parse it with Jsoup.

Sample Code:

private static RequestQueue myRequestQueue = null;

public Document GetDocument(String site) throws Exception {   
   final Document[] doc = new Document[1];
   final CountDownLatch cdl = new CountDownLatch(1);
 
   StringRequest documentRequest = new StringRequest( //
        Request.Method.GET, //
        site, //
        new Response.Listener<String>() {
           @Override
           public void onResponse(String response) {
               doc[0] = Jsoup.parse(response);
               cdl.countDown();
           }
        }, //
        new Response.ErrorListener() {
           @Override
           public void onErrorResponse(VolleyError error) {
               // Error handling
               System.out.println("Houston we have a problem ... !");
               error.printStackTrace();
           }
        } //
   );

   if (myRequestQueue == null) {
       myRequestQueue = Volley.newRequestQueue(this);
   }

   // Add the request to the queue...
   myRequestQueue.add(documentRequest);

   // ... and wait for the document.
   // NOTE: Be aware of user experience here. We don't want to freeze the app...
   cdl.await();

   return doc[0];
}

References

• An Introduction to Volley
• Transmitting Network Data Using Volley

Answer 2

With Stephan`s answer I have made some little modifications to this code and it looks like this. I have added UTF 8 support so it can read other languages and specified the retry policy.

private static RequestQueue myRequestQueue = null;

public Document GetDocument(String site) {
    final Document[] doc = new Document[1];
    final CountDownLatch cdl = new CountDownLatch(1);
    try {

        StringRequest documentRequest = new StringRequest( //
                Request.Method.GET, //
                site, //
                new Response.Listener<String>() {
                    @Override
                    public void onResponse(String response) {
                        String newStr = null;
                        try {
                            newStr = URLDecoder.decode(URLEncoder.encode(response, "iso8859-1"), "UTF-8");
                        } catch (UnsupportedEncodingException e) {
                            e.printStackTrace();
                        }
                        doc[0] = Jsoup.parse(newStr);
                        cdl.countDown();
                    }
                }, //
                new Response.ErrorListener() {
                    @Override
                    public void onErrorResponse(VolleyError error) {
                        // Error handling
                        System.out.println("Houston we have a problem ... !");
                        error.printStackTrace();
                    }
                } //
        );

        if (myRequestQueue == null) {
            myRequestQueue = Volley.newRequestQueue(MainActivity._Instance);

            documentRequest.setRetryPolicy(new DefaultRetryPolicy(5000,
                    DefaultRetryPolicy.DEFAULT_MAX_RETRIES,
                    DefaultRetryPolicy.DEFAULT_BACKOFF_MULT));
        }

        // Add the request to the queue...
        myRequestQueue.add(documentRequest);

        // ... and wait for the document.
        // NOTA: Be aware of user experience here. We don't want to freeze the app...
        cdl.await();
    }
    catch (Exception e) {
        Log.d("TMS", "Error parsing page " + site);
        e.printStackTrace();
        return null;
    }
    return doc[0];
}

Answer 3

Add Volley to your project.

implementation 'com.android.volley:volley:1.2.1'

Then use the code below,

 runOnUiThread(() -> {
 // Stuff that updates the UI
 new Thread(() -> {
     
     final StringBuilder builder1 = new StringBuilder();
     
     String url = "your.url.com";                                   //RequestQueue initialized
     RequestQueue mRequestQueue = Volley.newRequestQueue(this);     //String Request initialized
     StringRequest mStringRequest = new StringRequest(Request.Method.GET, url, response -> {
     
     //Toast.makeText(getApplicationContext(),"Response :" + response.toString(), Toast.LENGTH_LONG).show();
     
     Document doc = Jsoup.parse(response);
     
     Elements trs = doc.select("table.content tr");
     
     for (Element tr : trs) 
         {
             Elements tds = tr.getElementsByTag("td");
             Element td = tds.get(0);
             builder1.append(td.text()).append(",");
         }
         
         String str1 = builder1.toString();
         
         runOnUiThread(() -> {
             String[] items1 = str1.split(",");
             int i = 1;
             while (i < items1.length) 
             {
                 //
                 i++;
             }
             });
             }, error -> Log.i(TAG,"Error :" + error.toString()));
            mRequestQueue.add(mStringRequest);
     }).start();
});