0

I have a list of URLs, which I call from a for loop in a separate thread. The size of the list can be more than a 1000. When i run the code below. The data is downloaded for the first 30 to 40 URLS but later it the application gets stuck. Sometimes java.net.unknownhostexception unable to resolve host occurs but for different URLS at different times.

  for (Company aComapny:companyList) {
        URL url = null;
        HttpURLConnection con=null;
        BufferedReader in=null;
        try {
            url = new URL(DataExtractionConstant.URL+aComapny.getCompanyCode());
            Log.i(Constant.TAG, "URL:"+url.toString());
            con = (HttpURLConnection) url.openConnection();
            con.setRequestMethod("GET");
            con.setRequestProperty("connection", "close");
            int responseCode = con.getResponseCode();
            if (responseCode == HttpURLConnection.HTTP_OK) {
               in = new BufferedReader(new InputStreamReader(
                        con.getInputStream()));
                String inputLine;
                while ((inputLine = in.readLine()) != null) {
                    if (inputLine.contains(DataExtractionConstant.EXTRACT_CURRENT_VALUE_START)) {
                        String test = inputLine.substring(inputLine.indexOf(DataExtractionConstant.EXTRACT_CURRENT_VALUE_START) + DataExtractionConstant.EXTRACT_CURRENT_VALUE_START.length(),
                                inputLine.indexOf(DataExtractionConstant.EXTRACT_CURRENT_VALUE_END));
                        Log.i(Constant.TAG, "Extracted value~~~~" + test);
                    }

                }
                in.close();
            } else {
                Log.i(Constant.TAG, "GET request not worked");
            }
        } catch (Exception e) {
            e.printStackTrace();
            Log.i(Constant.TAG, "Exception");
        }
        finally {
            Log.i(Constant.TAG, "Finally called");
            try {
                in.close();
            } catch (IOException e) {
                Log.i(Constant.TAG, "Finally IOException");
            }
            con.disconnect();

        }
    }

This is the logcat message.

    10-16 22:09:52.087 30292-30446/myPackage I/URL_DATA: URL:http://MYURL
10-16 22:09:52.089 983-2241/? D/BandwidthController: [LG DATA] No such appUid: 10162
10-16 22:09:52.089 983-2241/? D/DnsProxyListener: App 10162 tries DNS query. Accept family:0 protocol:0
10-16 22:09:52.196 983-2241/? D/BandwidthController: [LG DATA] No such appUid: 10162
10-16 22:09:52.196 983-2241/? D/DnsProxyListener: App 10162 tries DNS query. Accept family:0 protocol:0
10-16 22:09:53.041 1944-2686/? D/ConnectivityService: updateCapabilities cap:[ Transports: WIFI Capabilities: INTERNET&NOT_RESTRICTED&TRUSTED&NOT_VPN LinkUpBandwidth>=1048576Kbps LinkDnBandwidth>=1048576Kbps SignalStrength: -70]
10-16 22:09:53.042 1944-2686/? D/ConnectivityService: notifyType CAP_CHANGED for NetworkAgentInfo [WIFI () - 139]
10-16 22:09:53.045 2749-3377/? I/NetworkController: onReceive: intent=Intent { act=android.net.wifi.RSSI_CHANGED flg=0x4000010 (has extras) }
10-16 22:09:53.045 2749-3377/? I/WifiSignalController: mWifiConnected = true, mWifiLevel = 2
10-16 22:09:53.045 2749-3377/? I/NetworkController: Change in state from: connected=true,enabled=true,level=2,inetCondition=1,iconGroup=IconGroup(Default Wi-Fi Icons),activityIn=true,activityOut=true,rssi=-66,,ssid="Ciff",ktGigaWiFi=false,attWifiCall=false
                                                        to: connected=true,enabled=true,level=2,inetCondition=1,iconGroup=IconGroup(Default Wi-Fi Icons),activityIn=true,activityOut=true,rssi=-70,,ssid="Ciff",ktGigaWiFi=false,attWifiCall=false
10-16 22:09:54.280 2749-2749/? I/NetworkController: Change in state from: connected=true,enabled=true,level=2,inetCondition=1,iconGroup=IconGroup(Default Wi-Fi Icons),activityIn=true,activityOut=true,rssi=-70,,ssid="Ciff",ktGigaWiFi=false,attWifiCall=false
                                                        to: connected=true,enabled=true,level=2,inetCondition=1,iconGroup=IconGroup(Default Wi-Fi Icons),activityIn=false,activityOut=false,rssi=-70,,ssid="Ciff",ktGigaWiFi=false,attWifiCall=false
10-16 22:09:56.291 2749-2749/? I/NetworkController: Change in state from: connected=true,enabled=true,level=2,inetCondition=1,iconGroup=IconGroup(Default Wi-Fi Icons),activityIn=false,activityOut=false,rssi=-70,,ssid="Ciff",ktGigaWiFi=false,attWifiCall=false
                                                        to: connected=true,enabled=true,level=2,inetCondition=1,iconGroup=IconGroup(Default Wi-Fi Icons),activityIn=true,activityOut=false,rssi=-70,,ssid="Ciff",ktGigaWiFi=false,attWifiCall=false
10-16 22:09:57.298 2749-2749/? I/NetworkController: Change in state from: connected=true,enabled=true,level=2,inetCondition=1,iconGroup=IconGroup(Default Wi-Fi Icons),activityIn=true,activityOut=false,rssi=-70,,ssid="Ciff",ktGigaWiFi=false,attWifiCall=false
                                                        to: connected=true,enabled=true,level=2,inetCondition=1,iconGroup=IconGroup(Default Wi-Fi Icons),activityIn=true,activityOut=true,rssi=-70,,ssid="Ciff",ktGigaWiFi=false,attWifiCall=false
10-16 22:09:59.306 2749-2749/? I/NetworkController: Change in state from: connected=true,enabled=true,level=2,inetCondition=1,iconGroup=IconGroup(Default Wi-Fi Icons),activityIn=true,activityOut=true,rssi=-70,,ssid="Ciff",ktGigaWiFi=false,attWifiCall=false
                                                        to: connected=true,enabled=true,level=2,inetCondition=1,iconGroup=IconGroup(Default Wi-Fi Icons),activityIn=false,activityOut=false,rssi=-70,,ssid="Ciff",ktGigaWiFi=false,attWifiCall=false
10-16 22:10:00.001 1944-2644/? D/PowerManagerServiceAO: acquireWakeLockInternal: lock=134641876, flags=0x40000001, tag="*alarm*", ws=WorkSource{1000}, historyTag=*alarm*:android.intent.action.TIME_TICK, uid=1000, pid=1944
10-16 22:10:00.071 1944-1944/? D/PowerManagerServiceAO: releaseWakeLockInternal: lock=134641876 [*alarm*], flags=0x0
10-16 22:10:00.078 2749-2749/? I/TimeTickManager: setTimeTickHandler, called onTimeChanged() nextTick: 59955

So my question is what is the best way to hit multiple URLS in a loop?

D-D
  • 954
  • 2
  • 12
  • 27
  • Looks like the WIFI (re-connected), which is something you'll need to take into account. It will reset the current downloads in progress and you'll need to add the tasks to download those urls back to the queue. You can also watch for certain network/connectivity events to detect connection loss and pause your downloads until the connection is restored. – jessehouwing Oct 16 '16 at 17:01
  • I suggest you take a look a `Volley` library. Each request can be added to a queue. – K Neeraj Lal Oct 16 '16 at 17:05
  • u got any solution??? – Pranita Sep 21 '17 at 11:36

0 Answers0