Loading large JSON file in C#

Question

1) I need to download a large JSON file on a Windows Phone application, I would like to process the JSON as it is downloading, because otherwise the app just keeps downloading the JSON and this can take up to 10 seconds or more, is there any way to do this? I have searched for this for 2 days now and nothing seems to be working...

2) Is there any easy way to check for new items in this JSON file? It's hosted online so I would have to check if there are new items (nodes) added to the file.

@newStackExchangeInstance: Are you aware of any streaming JSON parsers? — SLaks, Jun 24 '13 at 20:25

score 1 · Answer 1 · answered Jun 24 '13 at 20:24

1

Just ask a server if there is something new, and make him send to you only actual new stuff. It will be much less, so less data to trasmit on wire, and less time to process and visualize (if need) it.
Or create JSON request, that gets data in packets, and after every packet arrive, process it. It will take longer to download, as during processing you will block request, but it will give feedback to a user. You can leverage this, by pushing packet in local JS stack and continue request for another packet, while processing the that one already available on top of it. But it becomes fairly complex scenario, imo, for this simple enough (seems to me) stuff.

answered Jun 24 '13 at 20:24

Tigran

61,654
8
86
123

Asking the server is no option I'm afraid, it's a rss-feed like JSON file that gets updated every time new items get added (since 2008), I am just able to get that JSON file, notging else... – Schaemelhout Jun 24 '13 at 20:29
I'm afraid at this point you have no much options, then to manage that in some clever way on client side. – Tigran Jun 24 '13 at 20:46
That's what I was afraid of, and was hoping if anyone here has ran into that particular problem before me and had any tips for me.. :) – Schaemelhout Jun 24 '13 at 20:48

score 0 · Answer 2 · answered Jun 25 '13 at 03:00

1. If you want to parse through a JSON object gradually, look into JSON.NET's JsonReader.

Be aware that on Windows Phone, you can't pull from a network stream on the UI thread, but you can if you push your work to a background thread (have a look at BackgroundWorker).

2. If your web server supports it, you could pass an If-None-Match / If-Modified-Since header and accept the 304 response to indicate that nothing has changed.

However, this only matches entire HTTP requests. If you need finer detail, you could split your requests into:

a) Request a list of IDs and last modified dates

b) Request the details of a list of IDs (determined by the app based on the modified dates)

Loading large JSON file in C#

2 Answers2