0

In my code I have to write multiple json strings to files. When I do so, the end of what's being written is cut off by a seemingly random amount of characters.

I originally tried using fwrite(), then tested using just file_put_contents(), but both methods of writing to file cut off the characters. That code snippet looks like this:

print_r( json_encode($verified_results) );
file_put_contents($results_file, json_encode($verified_results));
/*$fh = fopen($results_file, 'w'); // or die("can't open file");
$stringData = json_encode($verified_results) ;
fwrite($fh, $stringData);

fclose($fh);
print_r(preg_last_error());
*/

I know that the json string is intact post-write, as the print_r returns the full string before writing it to file.


The text that's being cut off is:
108 characters
,"elapsedTime":"1 minute ago","hotel_cat":null,"visible":true,"hash_id":"aed1b4c6c515ea040c2e49d874c883a1"}]


110 characters

 5","elapsedTime":"11 hours ago","hotel_cat":null,"visible":true,"hash_id":"22aa8da3d0b8ef44ec07f8521986fbac"}]

The first file is 47KB in size, and the cut off point leaves 108 characters at a random point (not due to an invalid character).

The second file is 52KB in size, and the cut off point leaves 110 characters, once again, at a random point.

What could be causing this with such a small file?


Edit:

(Full code)
foreach ($sift_usernames as $key => $user) {
    $username = $user['username'];

    if(is_dir(APP_ROOT."sift_users/".$username)) {

        if(is_file((APP_ROOT."sift_users/" . $username . "/operations.json"))) {
            $operations_file = APP_ROOT."sift_users/" . $username . "/operations.json";
            $contents = file_get_contents($operations_file);
            $file_contents = json_decode($contents, true);

            $get_general_settings = file_get_contents(APP_ROOT. "sift_users/".$username."/general_settings.json");
            $general_settings = json_decode($get_general_settings, true);

            if(!empty($file_contents)) {
                $file_contents = json_decode($contents, true);
                $user_operations = array();
                $testing = "";

                foreach($file_contents as $operation_name=>$op) {

                    if(!file_exists(APP_ROOT. "sift_users/". $username ."/operations_data")) {
                        mkdir((APP_ROOT. "sift_users/". $username ."/operations_data"), 0777, true);
                    }
                    if(!file_exists(APP_ROOT. "sift_users/". $username ."/operations_data/". $operation_name)) {
                        mkdir((APP_ROOT. "sift_users/". $username ."/operations_data/". $operation_name), 0777, true);
                    }

                    $results_filename = 'results_'.date('m-d-Y');
                    $search_results = social_search($operation_name, $general_settings);
                    if(!file_exists(APP_ROOT. "sift_users/". $username ."/operations_data/". $operation_name."/".$results_filename)) {

                        print_r( json_encode($search_results) );
                        file_put_contents((APP_ROOT. "sift_users/". $username ."/operations_data/". $operation_name."/".$results_filename), json_encode($search_results));
                        chmod((APP_ROOT. "sift_users/". $username ."/operations_data/". $operation_name."/".$results_filename), 0777);
                    } else {
                        $results_file = APP_ROOT. "sift_users/". $username ."/operations_data/". $operation_name."/".$results_filename;
                        $old_contents = file_get_contents($results_file);
                        $old_results = json_decode($old_contents, true);

                        $duplicate_result = false;
                        $verified_results = array();
                        foreach($search_results as $new_key => $new_result) {
                            foreach ($old_results as $old_key => $old_result) {
                                if($old_result['hash_id'] == $new_result->hash_id)
                                    $duplicate_result = true;
                            }

                            if(!$duplicate_result) {
                                array_push($verified_results, $new_result);
                            }
                        }
                        foreach($old_results as $key => $result) {
                            array_push($verified_results, $result);
                        }
                        print_r( json_encode($verified_results) );
                        file_put_contents($results_file, json_encode($verified_results));
                        /*$fh = fopen($results_file, 'w'); // or die("can't open file");
                        $stringData = json_encode($verified_results) ;
                        fwrite($fh, $stringData);

                        fclose($fh);
                        print_r(preg_last_error());
                        */
                    }
                    //chmod((APP_ROOT. "sift_users/operations_data/". $operation_name."/testing.json"), 0777);
                    $user_operations[$username][$operation_name] = $op;
                    $testing.= $operation_name." - ";
                    //deleteDir(APP_ROOT. "sift_users/". $username ."/operations_data");
                } 
            }
        }
    }
}
iRector
  • 1,935
  • 4
  • 22
  • 30
  • I guess that you are doing something with the file later on. if just `file_put_contents()` is used without options, then this would not happen – hek2mgl Aug 18 '13 at 02:50
  • @hek2mgl But I'm not. The `print_r` before writing the string to file returns the full string. The text in the file (written immediately after the `print_r`) is then missing the 100 or so characters – iRector Aug 18 '13 at 02:53
  • show the full php script. – hek2mgl Aug 18 '13 at 02:53
  • 1
    Does a disk have any free space left to write? – TheFrost Aug 18 '13 at 03:04
  • Not sure what you mean. If you mean the php.ini configurations, then yes, they're set at 100MB. If you mean the server's harddrive, it has a few hundred gigs left. – iRector Aug 18 '13 at 03:07
  • I mean, does the target partition have any free space where the file can be write? // By the way: put all the `APP_ROOT. "sift_users/". $username ."/operations_data"` in variable and use it. The code will be much smaller and more readable. – TheFrost Aug 18 '13 at 03:09
  • Interesting. Can you put some chars that are right before the "cut" moment in data you trying to write? I know you said that there is no invalid character there, but just to be sure. – TheFrost Aug 18 '13 at 03:15
  • @TheFrost Yeah, sorry, it still needs some cleanups.. I'm still not too sure what you mean. I know there's space on the server in terms of capacity. I also know that if I store a smaller file say around 17KB of text, it still cuts off `ago","hotel_cat":null,"visible":true,"hash_id":"eea469fc30fa78c8541affd79b09eb5a","userScreenName":"_asdjkhl"}]` – iRector Aug 18 '13 at 03:16
  • @TheFrost This is the text before what I just posted as being cut off: `{"postContent":"@OliverLaaance Hello Bro !!!!! Please Notice Or Follow Please :):'( #14","userName":"John Ericson \u2665","userID":1033835718,"id":"368933417699663872","userImage":"http:\/\/a0.twimg.com\/profile_images\/378800000015048082\/8196f9b715e0e87447cb3b6b741222c1_normal.jpeg","postTime":"Sun Aug 18 03:12:35 +0000 2013","postImage":null,"type":"twitter","query":"hello","offerType":null,"resultID":"box15","elapsedTime":"19 seconds ` – iRector Aug 18 '13 at 03:17
  • let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/35680/discussion-between-thefrost-and-irector) – TheFrost Aug 18 '13 at 03:18

1 Answers1

0

An old question but anyone else scrambling for a fix,

For me on my Dev server I changed the following in the PHP.ini file: max_input_vars = 5000 and up'd both the input time and memory limit:

max_input_time = 180 memory_limit = 512M

Pretty sure only the input vars was needed but not tested it

Bodkins
  • 51
  • 3