Saving JSON to file often breaks

Question

I was trying to find anwser but maybe I'm not using the right terms for my problem so I need to ask here:

I made a webapp that loads a larger array of objects from a JSON file. If necessary the user will update some part of the array, delete objects or can push new data.

Because of this when the user decides to save the array back to JSON file with JQUERY AJAX, in my PHP that handles the data I use 'w+' tag so I delete the content and write back the whole new and modified content.

The issue is that even if the JSON file is used by a single user sometimes the writing process breaks before the whole content would be writen to that cleared file. This couses important data loss.

// I do make security savings to backup files, but this means every time the saving breaks I have to copy the last backup JSON file over the original JSON file from where the webapp loads data and until I do that nobody can use my webapp.

This is my JQUERY AJAX in a .js file:

$.ajax
({
    url: 'json/save_json.php',
    type: 'POST',
    dataType : 'json',
    data: { pid: currentdate, data: JSON.stringify(allinfobj), userlevel: jslevel},
    complete: function(){alert("this worked out!")};

});

My save_json.php:

<?php
$datecode = $_POST['pid'];
$myFile = "work".$datecode.".json";
$userData = $_POST['userlevel'];
if($userData != 4 && $userData != 3){
    $fh = fopen($myFile, 'w+') or die("Error while opening the file.");
    flock($fh, LOCK_EX);   // acquire an exclusive lock
    $stringData = $_POST['data'];
    fwrite($fh, $stringData);
    flock($fh, LOCK_UN);
    fclose($fh);
}
?>

I use certain variables there like $userData or $datecode. Those have nothing to do with the fact that fwrite breaks. Hope it's not confusing anyone, I assure you those work well.

The resulting JSON file normally is something like:

[{"name":"Apple","color":"red","quality";"awful"};{"name":"Tesla","color":"metallic","quality";"improved"};{"name":"Shoe","color":"white","quality";"fake"};{"name":"Glass","color":"undefined","quality";"good"};{"name":"Jessica","color":"private","quality";"private"}]

But sometimes my result because of a break that I don't understand is this:

[{"name":"Apple","color":"red","quality";"awful"};{"name":"Tesla","color":"met

So next time the user can't load this JSON data beacuase it's missing content and it's not ever a correct array or JSON.

a pr a+ in PHP I think is not a solution because I often need to replace or delete older JSON objects in the array so I can't get away by appending to the existing data, or the new array might be shorter then the old one.

FYI: in my ajax call success and error tags won't do anything. I don't know why. complete works though, as you can see, I use it in my code.

I hope I made my self fairly clear and I will be very grateful for any suggestions that why writing to a JSON file breaks. Where should I look to understand and fix this. Thanky you!

What happens if I send a pid like `some_file.php\0` (where `\0` is the null character) and then as `data` put some `PHP` code. You may think that an attacker don't know about the prepended `work` string in the file name. But one can guess it in the `read_json.php` by doing the same thing and reading the exception results. — Antonio Ragagnin, Apr 09 '14 at 09:45
You are absolutely right. However I would like to focus now on the data break fenomenon and then of course improving security, but unquestionably good to point this insecurity out. — Laci, Apr 09 '14 at 09:58

Antonio Ragagnin · Answer 1 · 2014-04-09T08:51:31.420

Since you only write the file, then you just need w instead of w+.

But I don't think this will sovle the issue.

I think your server has a limit in size of your POST or GET requests.

To test it, I suggest you to see if the size of the broken JSON string is the same of the limit you set in your PHP configuration.

The maximum size of a request is set in max_input_vars of PHP.INI.

To test if the data comes correctly to your PHP script, try the following: append (in another file) the size of the JSON that you recieve. When it crashes, see if the size of the broken JSON string is the same of the lenght you written in the other file.

Also, you can send, via JS, the client-side lenght of the JSON string. You can then compare the reiceved lenght with the one you compute on the string you reiceve.

This will help in discriminating if teh problem is client or server side.

Thanks, I will give it a try, but the new JSON file, after the break occures, cuts so short that it's hard to believe this is the reason. Also after replacing the JSON file with the backup file (roughly same size) It will work for days even if it gets another 10 or 20KB's worth of new data. — Laci, Apr 09 '14 at 08:45

score 1 · Accepted Answer · edited May 23 '17 at 11:49

1

Check that the Post data is not being truncated before save it to a file. Check post_max_size if your Json Data is growing. Try too to change w+ to c Mode when you save the file. w+ can truncate the file.

http://us3.php.net/manual/en/function.fopen.php

This link can help you too:

fopen(file,w+) truncates the file before I can check if it's locked with flock()

edited May 23 '17 at 11:49

Community

1
1

answered Apr 09 '14 at 08:51

Jaime García Pérez

953
1
10
16

It looks like changing `w+` to `c` and after obtainging the exclusive lock using `ftruncate($fh,0)` and `fflush($fh)` fixed the issue as after more then 24 hours of intense use the partial truncation issue did not appear. For the time being I will mark this as correct answer. I upvoted all answers as they all helped me more or less understand the problem. Thank you. – Laci Apr 11 '14 at 08:25

Saving JSON to file often breaks

2 Answers2