How to append in json file at specific position

Question

I'm trying to append JSON objects to a file at specific position but I'm unable to do it as required. I get result but without "," separating the values, so they are separate JSON objects.

I want to loop through JSON objects and append the result to a file and separate it with "," and it is a valid JSON. Here is what I tried


auto js_text = R"(
{
  "a": "all_items",
  "b": []
}
        )";
    std::ofstream js_file("test.json");
    js_file << std::setw(4) << js_text << std::endl;
    std::ifstream in("test4.json");
    json js_file = json::parse(in);
    std::ifstream load_url("url_file.json");
    json url_file = json::parse(load_url);
   for (auto& endpoint : url_file["url_list"])
        {
    std::string url = endpoint["url"].get<std::string>(); 
    auto r = cpr::Get(cpr::Url{ url }); 
    json j = json::parse(r.text); // r gets a nested json objects from parsed url
    for (auto& td : j["b"])    
     {    
      json value = j["b"][0];
      json data = js_file;
      data["b"][0] = value;
      std::ofstream output;
      output.open("test.json",std::ios_base::app );
      output << std::setw(4) << data << std::endl;
}

Result I want is

{
  "a": "all_items",
  "b": [
    {
      "c": "xxx",
      "d": "yyy"
    },
    {
      "e": "zzz"
    }
  ]
}

Updated Code: after Botje valuable input.

     auto js_text = R"(
        {
           "a": "abc",
             "b": [ ]
           }
           )";

    json js_file = json::parse(js_text);

     std::ifstream load_url("url_file.json");
    json url_file = json::parse(load_url);

    for (auto& endpoint : url_file["url_list"])
     {
      std::string url = endpoint["url"].get<std::string>(); 
      auto r = cpr::Get(cpr::Url{ url }); 
      json j = json::parse(r.text); // j contains results from multiple HTTP requests that has json objects.
       for (auto& elem : j["b"])
            {
              json jd = j["b"];
              js_file["b"].emplace_back(std::move(td));
              std::cout << "jd:" << jd.dump(4) << std::endl;
         }
       }

      std::ofstream output;
      output.open("test.json",std::ios_base::app );
      output << std::setw(4) << js_file << std::endl;
    }

Hope this help others.

What does `r.text` look like? From your code I'm guessing it contains an array of objects at key `b` and you want to transplant that to the object in `js_text`? — Botje, Feb 14 '20 at 13:15
It should be fairly easy to convert the example into a [self-contained minimal example](https://stackoverflow.com/help/minimal-reproducible-example). I refuse to attempt to understand that code. And please format it properly. I bet you'll find the mistake yourself if you do that. — pasbi, Feb 14 '20 at 13:17
Trying to do an "in-place" update of a complex structure requires that you a) find the exact location in the original file that you want to start writing to, and b) write the output in a way that is still syntactically correct. Both of these problems are not easy. — Bob Dalgleish, Feb 14 '20 at 17:18

Botje · Answer 1 · 2020-02-19T12:51:06.870

1

As I understand your question, you want to make multiple HTTP requests and collect all objects under each response's "b" key into an array.

Here, we skip the HTTP part and assume each element of r_texts is one response object.

std::vector<std::string> r_texts = {
    R"({
        "b": [{
            "c": "xxx",
            "d": "yyy"
        }]
    })",
    R"({
        "b": [{
            "e": "zzz"
        }]
    })",
};

This produces the output you wanted, then:

auto js_text = R"(
{
  "a": "abc",
  "b": []
}
)";

int main() {
    json js_file = json::parse(js_text);
    for (auto& r_text: r_texts) {
        // r_text is now the contents of one "HTTP response"
        json j = json::parse(r_text);
        // Loop over all objects inside the "b" key
        for (auto & elem: j["b"]) {
            // And use emplace_back+move to detach the object from `j`
            // and move it to the back of `js_file["b"]`.
            js_file["b"].emplace_back(std::move(elem));
        }
    }

    std::ofstream output;
    output.open("test.json",std::ios_base::app );
    output << std::setw(4) << js_file << std::endl;
}

edited Feb 19 '20 at 12:51

answered Feb 14 '20 at 13:21

Botje

26,269
3
31
41

The about code results the same as I get. I wanted to write json objects inside b[ ] separated by "," ensuring json format. – Amy Feb 17 '20 at 12:28
Now I'm confused. This code produces output that matches exactly what you want, I think? Please provide edit your question with an example of what is in `r.text`, what you expect to see, and how it is different from what this code produces. Also note that converting a `nlohmann::json` variable to text will _always_ result in valid JSON. – Botje Feb 17 '20 at 12:58
I have updated my code. Kindly let me know what I'm misssing. Thanks in advance. – Amy Feb 18 '20 at 10:52
So for each document in `url_list`, you want to extract the values under the `"b"` key and insert that into your `js_text` template. How should your "result I want" be constructed from these individual responses? You only have one object there. Should the result be some kind of concatenation of the responses? Unique? Should you actually produce one file per response? Please describe how your data should be processed in actual English sentences instead of presenting examples. – Botje Feb 18 '20 at 11:03
I want to get the values under "b" from url_list and insert it to js_text. js_text will have huge json objects under "b". so each json array( { } ) that I get from url_list should be appended to js_text file inside"b [ ]" separated by "," until last obtained json array, such that they are json formatted. Hope this helps. Thanks. – Amy Feb 19 '20 at 12:31
I updated my answer. This assumes that either the example input you gave is wrong (as both responses contain two objects, so a simple concatenation should result in four objects, mine results in "what you wanted") or your specification is wrong (you did not say the objects should be deduplicated or made unique). – Botje Feb 19 '20 at 12:53
Thanks alot. "Emblace "was the thing I was missing and file appending was inside the for loop. "Emblace" was something new to me. I will update my code. – Amy Feb 20 '20 at 10:06
A `nlohmann::json` array is just a `std::vector` by default, so you can use all the methods it supports. `push_back` could also work, but copies, whereas `emplace_back` supports moved-from objects. – Botje Feb 20 '20 at 11:10
Finally, if this answer helped you, consider marking it as accepted. That keeps the answer separate from the question and helps others find this solution. – Botje Feb 20 '20 at 11:12

Bob Dalgleish · Answer 2 · 2020-02-14T20:14:56.740

Let's assume that:

std::string original_string;  // Original file contents as a string
json original_json; // JSON value parsed from original_string
json updated_json;  // original_json but updated with new values

Then you get the string representation like so:

std::string updated_string = updated_json.dump();

Now, compare the two strings character at a time until you get a difference. That is the offset where you can write the new contents of the file.

size_t update_offset = 0;
for (size_t i = 0; i < original_string.size(); ++i) {
  if (i < updated_string.size()) {
    // Update is shorter than original, something is wrong
  }
  if (original_string[i] != updated_string[i]) {
    update_offset = i;
    break;
  }
}

At this point, update_offset will have the number of bytes in common between the two strings; everything after this has changed. Now, we open the original file, seek to this offset and write the rest of the string.

std::ofstream output{"test.json", std::ofstream::out};
output.seekp(update_offset);
output << updated_string.substr(update_offset);
output.close();

The above sequence ensures that the contents of the output file are a syntactically correct representation of your JSON data.

How to append in json file at specific position

2 Answers2