1

Is there way to validate the syntax of a JSON file in Unix and move the invalid files into a error folder.

Thanks, Kavin

Grokify
  • 15,092
  • 6
  • 60
  • 81
Kavin Palaniswamy
  • 181
  • 1
  • 10

2 Answers2

1

You can try using a python oneliner:

python -c "import json;file = open('foo.json');json.loads(file.read());file.close()"

With some tweak, you can turn it into an alias:

testjson(){
    python -c "import json;file = open('$1');json.loads(file.read());file.close()"
}

And use something like this:

ls | while read file;
  testjson $file || mv $file ~/bad_json/
done;

Some sample with this:

echo "{}" > foo.json

echo "{" > bar.json

testjson bar.json && echo ok 
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
    obj, end = self.scan_once(s, idx)
ValueError: Expecting object: line 1 column 2 (char 1)

testjson bar.json || echo ko
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
    obj, end = self.scan_once(s, idx)
ValueError: Expecting object: line 1 column 2 (char 1)
ko

testjson foo.json && echo ok
ok

testjson foo.json || echo ko

Note: just be careful with special filenames containing ' or \n

Blusky
  • 3,470
  • 1
  • 19
  • 35
  • Hi @Blusky , thanks for the quick response. I did try the code above but , it was also moving the valid json file, what could be wrong ? Adding the code that i used in the comment below. – Kavin Palaniswamy Nov 19 '18 at 20:49
  • `#!/bin/bash contec_strr=/app/comcast-me-sw/public/contec_strr cp_path=/app/kavin-test/public/Files/JSON testjson(){ python -c "import json;file = open('$1');json.loads(file.read());file.close()" } cd $contec_strr ls *.json | while read file; do testjson $file || cp $file $cp_path done;` – Kavin Palaniswamy Nov 19 '18 at 20:50
  • Are you sure your "valid json" are valid ? If python detects it as invalid, you should have an error printed, can you share it ? – Blusky Nov 20 '18 at 06:29
  • 1
    Hi @Blusky, really sorry for the late reply. I did get it to work, the code mentioned above had issues. You method works flawlessly !! Thanks again for the help, much appreciated. – Kavin Palaniswamy Nov 27 '18 at 02:36
0

Validating (to some extent) 1660 jsonld files in my test_0009 directory. All have "0" exit code (= "OK"):

[pjalajas@sup-pjalajas-2 504Project]$ find test_0009 -iname "*.jsonld" | parallel "jq '.' 2> /dev/null ; echo $?" | cut -d\  -f1 | sort | uniq -c | sort -k1nr
   1660 0
petjal
  • 19
  • 3