Breaking json object with large data in HTTP GET

Question

I have an application built with Angularjs and CodeIgnitor. The angularjs sets $scope data using $resource service get methods in startup using ng-init.

Its working fine with less data but the application fails with large count of data. Is there any data size limit in GET response like the GET parameters limit?

The error:

Error: JSON.parse: end of data after property value in object

The script:

app.factory('FamilyFind', ['$resource', 'Data', function ($resource, Data) {
  return $resource(Data.rootUrl + 'admin/family_find/:id', {id: '@id'}, 
    {
      query: {
        isArray: true,
        method: 'GET'
      },
      update: {
        method: 'PUT'
      }
    }
  );
}]);

// load serverside data using http resource service

FamilyFind.get({}, function (response) { // success

   $scope.tableData = response.data; // store result to variable

}, function (error) { // ajax loading error

   Data.errorMsg(); // display custom error notification
   // error caught here
});

I am using philsturgeon CodeIgnitor RESTful API for back-end services.

CodeIgnitor Controller:

function family_find_get($id = '', $type = 'json') { 

  $this->response->format = $type;  
  $family             = $this->family_registration_model->find( $this->get() );  

  if($family) {  // success
     $this->response(array( 'data' => $family), 200); 
  } else  {  // error
     $this->response(array( 'data' => array()), 200);
  }    
}

CodeIgnitor Model:

function find($data) {
    $query = "SELECT dev_members.name, dev_family.house_name, dev_family.house_no, dev_family.place,
              dev_family.post, dev_family.phone,mem.male,mem.female 
              FROM dev_family
              JOIN dev_members ON dev_members.id = dev_family.family_head_id
              LEFT JOIN (SELECT family_id,SUM( gender = 'Male' ) AS male, SUM( gender = 'Female' ) AS female FROM dev_members) AS mem ON mem.family_id = dev_family.id WHERE dev_family.family_reg_no != ''";

    if( isset($data['cluster']) && $data['cluster'] != '' ) {
        $query.=" AND dev_family.cluster_id = ".$data['cluster']."";
    }

    $query = $this->db->query($query);
    return $query->result();
}

Edit:

When I debug the response from server-side, it shows the main json object not closing automatically. See sample data extracted from the 500+ records,

{"data":[{"name":"abc","house_name":"dfdfgdfg","house_no":"1","place":"School  Road","post":"dsfdfg","phone":"1111111","male":"278","female":"264"},{"name":"dgdfg","house_name":"dgdfg","house_no":"2","place":"School Road","post":"dfgdfg","phone":"111111","male":null,"female":null},{"name":"dgdfg","house_name":"dgdfg  dgd","house_no":"3","place":"School Road","post":"dfgdg","phone":"11111","male":null,"female":null},{"name":"dgg","house_name":"dgdfg","house_no":"4","place":"School Road","post":"dggd","phone":"11111","male":null,"female":null}]

The error in JSON Formatter:

Error: Parse error on line 1:
...null,"female":null}]
-----------------------^
Expecting '}', ',', got 'EOF'

And when I add the data object close at the end } using RESTClient add-on, it's working fine.But how can I solve this problem in server-side? And why it happens only with large data?

Update:

I have realized that the problem is with philsturgeon REST Controller library. Instead of,

$this->response(array( 'data' => $family), 200);

I have tried,

echo json_encode(array('data'=>$family));

And now it's working fine.Anyone knows how to solve this in philsturgeon REST Controller library?

Github Issue here.

score 3 · Answer 1 · answered Mar 30 '14 at 06:34

I don't think that the problem is in the data size itself. Response is not limited usually (though there are some possibilities that because of long time for operation or any other reason server closes the connection and the client get the cut data).

Please better check that you have correct data there. I faced some problem in my previous experience when some data (which is usually text data) with some special symbols like ',', ''', '{', '}' etc. can be corrupted. It depends how your server side framework can correctly handle these symbols so the client side can correctly process it. If you can print the output to the log at the server side and check how the JSON data is correct.

Breaking json object with large data in HTTP GET

1 Answers1