1

I previously was trying this using a tree structure. The JSON response is about 3.5MB, so it was understandable that trying to parse it this way would cause older devices to run out of memory.

My question is: Why, after I have switched to a streaming JsonReader, am I still getting the OutOfMemoryException? I even save the response to a .json file on the sd card and read from there so that the String doesn't need to be stored in memory.

  List<CurrencyPair> currencyPairs = new ArrayList<CurrencyPair>();
  URL url = new URL("http://pubapi.cryptsy.com/api.php?method=orderdatav2");

  File sdCard = Environment.getExternalStorageDirectory();
  File directory = new File(sdCard.getAbsolutePath() + "/zTrader");
  directory.mkdirs();
  File file = new File(directory, "cryptsy_response.json");

  BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream()));
  BufferedWriter out = new BufferedWriter(new FileWriter(file));

  char[] cbuf = new char[255];
  while ((in.read(cbuf)) != -1) {
    out.write(cbuf);
  }
  in.close();
  out.close();

  FileInputStream fis = new FileInputStream(file);
  InputStreamReader fileInputStreamReader = new InputStreamReader(fis);
  Reader bufferedReader = new BufferedReader(fileInputStreamReader);
  JsonReader reader = new JsonReader(bufferedReader);

  reader.beginObject();
  Map<CurrencyPair, Integer> currencyPairs_MarketIds = new HashMap<CurrencyPair, Integer>();

  while (reader.hasNext()) {

    String name = reader.nextName();

    if (name.equals("success")) {
      reader.nextInt();
    }
    else if (name.equals("return")) {
      reader.beginObject();

      while (reader.hasNext()) {
        String pairString = reader.nextName();
        String[] currencies = pairString.split("/");
        CurrencyPair currencyPair = new CurrencyPair(currencies[0], currencies[1]);
        currencyPairs.add(currencyPair);

        reader.beginObject();
        while (reader.hasNext()) {
          if (reader.nextName().equals("marketid")) {
            Integer id = Integer.valueOf(reader.nextString());
            currencyPairs_MarketIds.put(currencyPair, id);
          }
          else {
            reader.skipValue();
          }
          if (sleep)
            Thread.sleep(25);
        }
        reader.endObject();
        if (sleep)
          Thread.sleep(25);
      }
    }
  }
  reader.close();
  file.delete();
zholmes1
  • 539
  • 5
  • 21
  • does the server use gzip? Have you tried any third party library? – Pavlos Sep 27 '14 at 01:34
  • Are you sure the deserialization is causing the OOM error in both cases ? Have you profiled the memory usage ? You do load the whole data in memory in your hash map in the end. Does it still occur if you comment out the `put` to the `HashMap` ? – desseim Sep 27 '14 at 01:39
  • I can't imagine that the HashMap would be causing the OOM exception. It only ends up having about 200 keys. – zholmes1 Sep 27 '14 at 01:56
  • I'm with desseim. Are you sure this is what's causing the OOM? 3.5 MB really isn't much.. the app I've been working on takes up ~150 MB. – Coeffect Sep 27 '14 at 02:02
  • The HashMap is definitely not where the error is occuring. It's happening on this line: while ((in.read(cbuf)) != -1) { – zholmes1 Sep 27 '14 at 02:06
  • Definitely not a problem with the JSON deserialization then. `BufferedReader` may grow its buffer when its mark is set with a big limit, but I don't see any of your code doing anything similar. I guess a full stack trace would give a bit more insight. – desseim Sep 27 '14 at 04:35

0 Answers0