Changing the key of JSON file whilst retaining data within that bracket section

Question

My first thought was a simple "search and replace of the underlying JSON code, inspecting it line by line. That will not work as the same "keyword" could be used in various places, but only one specific line needs to be changed.

Then I thought, read the existing data in, create a new entry, push the data into that block, and then delete the original key. The former works ... but leaves the original and all its data intact.

Here is a simplified version (don't think 500+ lines would be appreciated):

{
project 1:{
     ad_campaign:{
       ad_1{
          data1,
          data2
       },
       ad_2{
         data1,
         data2
       }
     }
}
project 2:{
     ad_campaign:{
     ad_1{
        data1,
        data2
        }
     ad_201{
       data1,
       data2
       }
     }
}
}

The "data" entries are 6 lines per ad, and there could be dozens of ads per project, and dozens of projects !!

They want to change Projec1 "ad_1" to "ad_101"

As you can see, both projects have ads named "ad_1", so a simple search and replace of the underlying text code file is a no-go.

If I use $project1->{"ad_campaign"}, I can get all the data. If I then use: $project1->{"ad_campaign"}=$new_ad_id ... it creates a new section (which I =could= then copy the data line - by - line) ... but I still cannot get rid of the original "ad_1" entry! ($project1->{"ad_campaign"}->{"ad_1"} =null (no bareword allowed) / "null" / "" have no effect - even when trying to remove the internal data first. That could result in data1:'',data2:'' etc)

What I really need is a way to do $customer1->{"ad_campaign"}->{"new_ad_id"} to simply change that third key. (I've Googled "change" / "replace" JSON key, but no results ... especially as I am using PERL)

One more "spanner in the works"; the file gets saved as a single line, (Using JSON::PP The "pretty" seems to add too many spaces - tab indents would seem better) so would be difficult to break apart, and do a line by line scan anyway. (Another idea was to reiterate through file, set a flag once hit "customer1", another loop, and another flag for "ad_campaign", and then final loop seeking "ad_1". Seems a bit inefficient ... plus all the decoded data is in a hash anyway!)

Please bear in mind, I have simplified this code. Between "Project1" and "ad_campaign" could be another 50 key:value pairs PER PROJECT

There is possibly some simple solution finding / changing the value via a hash "argument" ... but I've yet to find it!

Somewhere else I read: delete $JSON->{"project1"}->{"ad_campaign"}->{"ad_id"} but that didn't delete the original either!

Just had a closing thought: Maybe I could do something like "indexOf" to locate project1/ad_campaign etc, and then do split/splice, pushing one half of data, then split again etc until I find the word to replace. But again, that does seem overkill for what could be a pretty basic problem

This was only meant as a guide. The code I actually use passes JSONLINT, so presume that must be correct — Cristofayre, Feb 18 '23 at 18:25
Don't lie, now. It's not remotely close to being valid, and jsonlint craps out on the second line. Seventeen of the twenty-six lines have errors on them. It doesn't serve as a guide cause the errors are so numerous and ambiguous — ikegami, Feb 18 '23 at 18:43

pmqs · Answer 1 · 2023-02-18T19:45:37.373

Try code below.


use strict ;
use warnings;
use feature 'say';

use JSON;

my $data = <<EOM;
{
    "project 1": {
      "ad_campaign": {
        "ad_1": [
          "p1: ad:1 data1",
          "p1: ad:1 data2"
        ],
        "ad_2": [
          "p1: ad:2 data1",
          "p2: ad:2 data2"
        ]
      }
    },
    "project 2": {
      "ad_campaign": {
        "ad_1": [
          "p2: ad:1 data1",
          "p2: ad:1 data2"
        ],
        "ad_201": [
          "p2: ad:201 data1",
          "p2: ad:201 data2"
        ]
      }
    }
  }
EOM

my $in = from_json($data);

# create "ad_101" to reference to the existing "ad_1" data tree
$in->{"project 1"}{"ad_campaign"}{"ad_101"} = $in->{"project 1"}{"ad_campaign"}{"ad_1"};

# now delete the reference to "ad_1"
delete $in->{"project 1"}{"ad_campaign"}{"ad_1"};


# using pretty & canonical here to make the results easier to read
# don't use pretty if you want all JSON output as a single line
say to_json($in, {pretty => 1, canonical => 1});

output is

{
   "project 1" : {
      "ad_campaign" : {
         "ad_101" : [
            "p1: ad:1 data1",
            "p1: ad:1 data2"
         ],
         "ad_2" : [
            "p1: ad:2 data1",
            "p2: ad:2 data2"
         ]
      }
   },
   "project 2" : {
      "ad_campaign" : {
         "ad_1" : [
            "p2: ad:1 data1",
            "p2: ad:1 data2"
         ],
         "ad_201" : [
            "p2: ad:201 data1",
            "p2: ad:201 data2"
         ]
      }
   }
}

Key point to note is that nested hashes in perl use references, which are akin to pointers in C/C++ land. Means that after running this line

$in->{"project 1"}{"ad_campaign"}{"ad_101"} = $in->{"project 1"}{"ad_campaign"}{"ad_1"};

both the ad_1 and ad_101 keys reference the same data, namely

[
   "p1: ad:1 data1",
   "p1: ad:1 data2"
],

[EDIT - answering some of the comments]

I changed the data values to highlight that the reference in ad_1 and ad_101 are pointing to unique data in the JSON document. You didn't supply any data, so I made some up.

Use of from_json and to_json rather than decode_json & encode_json is purely to allow the pretty parameter to be controlled. Use the variant that suits your use-case. Check out the docs here

I'm not sure how the "p2:ad:201 data1" found itself into the mix, (my understanding of JSON is still very rudimentary) Within "ad_201{" I would use "camp_date:16","camp_month:2","camp_year:2023" etc. Only square bracket is "track:[0,0,0 ... (93 times, 3 entries per day)]," and similarly outside "camp", use "xxx:yyy," Unfortunately, I can't have two keys referencing the same data as the keys are used to populate a drop down menu, which is why the key they are replacing has to "disappear" — Cristofayre, Feb 18 '23 at 17:47
Looking further at the code, I see it uses "to_json" and "from_json" Is there any reason to use that over "decode_json xxx" which I currently use? (Which is also why I don't know how to handle pretty as I use: $JSData=decode_json(tmp) :LOAD and tmp=encode_json($JSData) :SAVE — Cristofayre, Feb 18 '23 at 17:50
Just set it up on a test script, and the "cross ref" of new pointing to old, and then delete to remove old appears to do the trick. Many thanks. (now off to research from_json/to_json and "say". Again, I encode_json, then open filehandle to print $tmp to file — Cristofayre, Feb 18 '23 at 18:04
[delete](https://perldoc.perl.org/functions/delete) returns the value deleted; so you could just do `$in->{"project 1"}{ad_campaign}{ad_101} = delete $in->{"project 1"}{ad_campaign}{ad_1}`. — Jim Davis, Feb 18 '23 at 18:23

score 2 · Answer 2 · answered Feb 18 '23 at 14:33

Another option is to use jq, rather than Perl

Assuming data.json contains this

{
    "project 1": {
      "ad_campaign": {
        "ad_1": [
          "p1: ad:1 data1",
          "p1: ad:1 data2"
        ],
        "ad_2": [
          "p1: ad:2 data1",
          "p2: ad:2 data2"
        ]
      }
    },
    "project 2": {
      "ad_campaign": {
        "ad_1": [
          "p2: ad:1 data1",
          "p2: ad:1 data2"
        ],
        "ad_201": [
          "p2: ad:201 data1",
          "p2: ad:201 data2"
        ]
      }
    }
}

this one-liner to do the rename

$ jq '."project 1".ad_campaign |= ( .ad_101 = .ad_1 | del(.ad_1) ) '  data.json

output is

{
  "project 1": {
    "ad_campaign": {
      "ad_2": [
        "p1: ad:2 data1",
        "p2: ad:2 data2"
      ],
      "ad_101": [
        "p1: ad:1 data1",
        "p1: ad:1 data2"
      ]
    }
  },
  "project 2": {
    "ad_campaign": {
      "ad_1": [
        "p2: ad:1 data1",
        "p2: ad:1 data2"
      ],
      "ad_201": [
        "p2: ad:201 data1",
        "p2: ad:201 data2"
      ]
    }
  }
}

to get jq to output as a single line, add the -c option to the commandline

$ jq -c  '."project 1".ad_campaign |= ( .ad_101 = .ad_1 | del(.ad_1) ) '  data.json 
{"project 1":{"ad_campaign":{"ad_2":["p1: ad:2 data1","p2: ad:2 data2"],"ad_101":["p1: ad:1 data1","p1: ad:1 data2"]}},"project 2":{"ad_campaign":{"ad_1":["p2: ad:1 data1","p2: ad:1 data2"],"ad_201":["p2: ad:201 data1","p2: ad:201 data2"]}}}

You say use jq rather than PERL. Not really an option as I have 20+ other scripts running PERL that call / interact with this "database". But thanks for the input — Cristofayre, Feb 18 '23 at 17:40
If you have a bunch of other programs that rely on the same low-level details, then you have a different problem. Refactor how they get their data so the manipulations all happen in common code. — brian d foy, Feb 19 '23 at 21:06
My website use a mix of several languages: 80% of the scripts, (about 100) are in PERL. But sometimes, PERL doesn't work as planned, (such as LWP::UserAgent) so I use PHP and Curl. As all the programs output a GUI to browser for information to be submitted, then I use HTML/Javascript and HTTPRequest. To date, the data was then stored in flat files, but when checking one script, found it would have to open and update 14 different files, so I incorporated them all into a curly bracket file (better not call it JSON). Still very new to the laltter — Cristofayre, Feb 21 '23 at 15:59
I do my designing offline, emulating the server setup. I could use jq offline ... but my shared server may not allow the jq installation, and I'd have to start again. NB: Why doesn't UserAgent work. I use Strawberry Perl ... and their CPAN module no longer connects to CPAN. (And they store the modules in weird places - in my case, Windows->Users->perl5->lib->perl5 (!!). Please don't think I'm "arguing" with your comment, I'm just painting the picture from my perspective — Cristofayre, Feb 21 '23 at 16:05

brian d foy · Answer 3 · 2023-02-20T10:13:12.470

The others have answered this question adequately, and I'd also choose jq (see Using jq how can I replace the name of a key with something else and the jq FAQ).

But, I want to emphasis what @pmqs is doing but not yelling from the rooftops.

It's just Perl

If you want to do something in Perl, you basically have scalars, arrays, and hashes. If you are already in Perl land, that's what you are manipulating.

You don't change a JSON key in Perl; you change a hash key. It doesn't matter that you got that hash from JSON or will output it as JSON. That's how you get to @pmqs's delete:

$hash->{$new_key} = delete $hash->{$old_key};

Everything beyond that is just input and output.

The right tool for the right job

You have a few other questions about the output, and those questions are unrelated to changing a hash key. When I deal with this, I simply ignore it. If it outputs it all on one line, I don't really care. I can get pretty output with jq:

% jq .

If I want to minify it, I can use testing:

% jq -r tostring data.json

You mentioned that you didn't like the indention. That's easy to change. jq uses 2 space indents, but I can make that smaller (or larger, or tabs) (see also How to restrict indentation while using jq filters on json file):

% jq --indent 1 . data.json

You say that you can't use jq because you are using Perl, but remember, Perl is the glue language of the internet. You can easily call other programs to do your work and you are expected to do so when something else can do the job better. Why wrestle with Perl and its modules when an existing tool already does a better job? If you aren't able to install it for whatever reason, that's fine. But saying that you can only choose one tool unnecessarily limits you.

Setting aside that, JSON::PP has settings for space_before, space_after, indent, and indent_length. You can adjust those how you like. If you have problems with those, you can ask a separate question.

Thank you for a very detailed reponse. I have to admit, I'm a "suck it and see" person. If I can't get something to work, I "Google" a solution, and hack it as best I can. I can just about follow that JSON is a "style" of writing, that actually create javascript objects (or hashes in this case) With PERL, I can understand curly brackets giving "associative array", and square brackets an indexed array, but hashes and how to "interact" with them is a mystery. As I say, I try it, then try again, and again till damn thing does work :-) — Cristofayre, Feb 21 '23 at 16:12
I'm so "basic", I use "$tmp=decode_json $xyz" and "$tmp=encode_json $xyz. Print FH $tmp" And with those simple statements, I don't understand how I could incorporate "pretty->" (I have seen things like new->pretty->UT8 etc, but as I say, I use the simple code) (Since it's only for testing, I either output using Dumper, or run the single line output through "jsonlint" to split it) When uploaded to server, it won't matter about the single line format. — Cristofayre, Feb 21 '23 at 16:17
The module docs will show you how to do things. But if it doesn't matter, don't pretty print at all. Use jq to reformat the data in dev if you want to look at it. — brian d foy, Feb 21 '23 at 21:24

Changing the key of JSON file whilst retaining data within that bracket section

3 Answers3

It's just Perl

The right tool for the right job