How Flutter can decode a very large json array, Flutter just decode the only first 20 items?

Question

Hello I've an api that return a big list the min length of this list could be 100 or 120 element, and flutter decode only first 20 element is the list. Now that it seems to you that I load a very large json array elements and don't care about the api fast the req only takes 300 mille sec as possible The Api Is : https://newsapi.org/. I use http package for loading data and thats my code.

Future getEverythingFunc({@required String query}) async {
    http.Response response =  await http.get(Uri.parse(ApiReq.everyThingForSearchingUrl(query:query)));
    if (response.statusCode >= 200 && response.statusCode <= 300) {
      print(response.body);
      return everythingModel = everythingModelFromJson(response.body);
    } else if (response.statusCode >= 400 && response.statusCode <= 500) {
      throw "Err while getting data ${response.statusCode}";
    } else {
      throw 'Unknown Err ${response.statusCode}';
    }
  }

Thanks for reading and hope you help me

Is it possible to provide all of the returned JSON so we can try and see what goes wrong? I guess you have checked by printing the `response.body` and verified that the JSON are fully valid and does not contain any charset issues. — julemand101, May 22 '21 at 22:23
Yes I printed and see that the response return it's full data back but the problem cause while parsing it just only accept the first 20 element or random 20 element and put them on my list in my model. — Omar Salem Bakry, May 22 '21 at 23:01
Ok, have you tested where you have pasted the full JSON directly into your dart code? Just to verify that the issue is not about some other problem? — julemand101, May 22 '21 at 23:06
Yes for sure! here's the problem everything works awesome everything is good the only strange thing that it only parsing 20 element into model's list so I ask my self the problem could be related to threading or the problem could happen because api licence ? but api if you checked it says for developer plan nothing to pay so that I think the problem could cause because I do everything in the main thread so I used compute method but same thing happen !. you could check my code on git send me you're mail or write it here to check my code if you wanna check it — Omar Salem Bakry, May 22 '21 at 23:30
I am not really a Flutter developer but I can properly move your code to some normal Dart program and test the JSON parsing. You can find my mail on my Github profile (after you have logged in): https://github.com/julemand101 . I am a little busy tomorrow so I cannot guarantee I will look at it before monday. :) — julemand101, May 22 '21 at 23:33
Okay i'll invite you soon thanks for your appreciate hope someone else help me as soon as possible . — Omar Salem Bakry, May 22 '21 at 23:49

score 0 · Accepted Answer · answered May 23 '21 at 02:06

I don't know how you are coding, but jsonDecode function is working fine. Have you read the doc from NewsAPI? when pageSize parameter is omitted the default is 20 results, and its maximum value for free subscription is 100. I've tested it, my code:

// @dart=2.13
import 'dart:async';
import 'dart:io';

import 'package:wnetworking/wnetworking.dart';

typedef JMap = Map<String, dynamic>;

class NewsAPI {
  static const _base = 'https://newsapi.org/v2';
  static const _apiKey = '111111111111111111111111111111';
  
  // 'pageSize' ranges from 0 to 100 for free subscription, more size is paid
  // source: https://newsapi.org/docs/endpoints/everything
  static FutureOr<void> fetchNews({required keyword, int pageSize=20}) async {
    final url = '$_base/everything?q=$keyword&pageSize=$pageSize&sortBy=publishedAt&apiKey=$_apiKey';
    List? news;

    stdout.write('Fetching news... ');
    await HttpReqService.getJson<JMap>(url)
      .then((response) {
        if (response != null) {
          news = response['articles'] as List;
        }
      })
      .whenComplete(() {
        print('done!');
        print('\nNews fetched: ${news!.length}');
      });
  }
}

void main(List<String> args) async {
  await NewsAPI.fetchNews(keyword: 'anime', pageSize: 100);
  print('Job done.');
}

Result:

Fetching news... done!

News fetched: 100
Job done.

Note

wnetworking package is not ready to publish yet, it contains operations related to API, etc. You can replace HttpReqService.getJson with your typical http.get but keep in mind the return value and exceptions.

Thanks a lot bro you solved my issue the only thing that i miss is reading 'pageSize' in docs, Thanks for your time — Omar Salem Bakry, May 23 '21 at 02:53

How Flutter can decode a very large json array, Flutter just decode the only first 20 items?

1 Answers1