370

In my Unix shell script, when I execute a curl command, the result will be displayed as below which I am redirecting to file:

{"type":"Show","id":"123","title":"name","description":"Funny","channelTitle":"ifood.tv","lastUpdateTimestamp":"2014-04-20T20:34:59","numOfVideos":"15"}

But, I want this output to put in the readable JSON format like below in the file:

{"type":"Show",
"id":"123",
"title":"name",
"description":"Funny",
"channelTitle":"ifood.tv",
"lastUpdateTimestamp":"2014-04-20T20:34:59",
"numOfVideos":"15"}

How do I format the output this way?

Matthias Braun
  • 32,039
  • 22
  • 142
  • 171
Jams
  • 3,769
  • 3
  • 18
  • 13

12 Answers12

765

A few solutions to choose from:

json json is a fast CLI tool for working with JSON. It is a single-file node.js script with no external deps (other than node.js itself).

$ echo '{"type":"Bar","id":"1","title":"Foo"}' | json
{
  "type": "Bar",
  "id": "1",
  "title": "Foo"
}

Require:

# npm install -g json

json_pp: command utility available in Linux systems for JSON decoding/encoding

echo '{"type":"Bar","id":"1","title":"Foo"}' | json_pp -json_opt pretty,canonical
{
   "id" : "1",
   "title" : "Foo",
   "type" : "Bar"
}

You may want to keep the -json_opt pretty,canonical argument for predictable ordering.


: lightweight and flexible command-line JSON processor. It is written in portable C, and it has zero runtime dependencies.

echo '{"type":"Bar","id":"1","title":"Foo"}' | jq '.'
{
  "type": "Bar",
  "id": "1",
  "title": "Foo"
}

The simplest jq program is the expression ., which takes the input and produces it unchanged as output.

For additional jq options check the manual


python yq yq: Command-line YAML/XML/TOML processor - jq wrapper for YAML, XML, TOML documents

$ echo '{"type":"Bar","id":"1","title":"Foo"}' | yq
{
  "type": "Bar",
  "id": "1",
  "title": "Foo"
}

The version go yq doesn't work here


With xidel Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents.

$ echo '{"type":"Bar","id":"1","title":"Foo"}' | xidel -e '$json'
{
  "type": "Bar",
  "id": "1",
  "title": "Foo"
}

with :

echo '{"type":"Bar","id":"1","title":"Foo"}' | python -m json.tool
{
    "id": "1",
    "title": "Foo",
    "type": "Bar"
}

with and :

echo '{"type":"Bar","id":"1","title":"Foo"}' | node -e "console.log( JSON.stringify( JSON.parse(require('fs').readFileSync(0) ), 0, 1 ))"
{
 "type": "Bar",
 "id": "1",
 "title": "Foo"
}
Gilles Quénot
  • 173,512
  • 41
  • 224
  • 223
  • is this the way ?curl -i -vs -X POST -H "$SESSION_TOKEN" -H "$AUTH_TOKEN" -H "Accept:$ACCEPT_HEADER" -H "Content-Type:text/plain" "http://$BASE_URI/api" | json_pp >> jsonoutput.json – Jams Dec 01 '14 at 22:57
  • 1
    Absolutely. Have you tested ? – Gilles Quénot Dec 01 '14 at 22:59
  • yes, but nothing been written to the file "jsonoutput.json". Its size is zero. – Jams Dec 01 '14 at 23:04
  • 1
    `json_pp` seems to format data in reverse order than it's expected. Is there a way to un-reverse data? – Antonio Aug 23 '16 at 01:39
  • +1 for the bash and node.js portion. I was trying to do something different (pipe JSON data into node.js). I was able to do this via `cat` and node using `cat ./example.json | node <<< "var o = $(cat); console.log(o);"` Worked like a charm – Robert Dundon Dec 15 '16 at 16:26
  • jq is great. can you explain why you added the 'dot' at the end ? ('jq .') – programmer Jun 06 '17 at 09:23
  • 2
    json_pp is a Perl command utility which converts between some input and output formats (one of them JSON). The default input format is json and the default output format is json with pretty option. – bajocode Aug 18 '17 at 05:24
  • How to do this with c++ – Sai Dec 10 '20 at 10:56
  • 1
    Pipe to Python for the win ! – Pierre Mar 28 '21 at 17:48
  • I like the jq solution, but may I know what's the purpose of the quotes around the period? – Someone Special Apr 09 '21 at 09:54
  • @Jams, the command your wrote above worked for me on Mac, it outputs a full json file, you may need to specify the output argument explicitly to `json` like so: `curl https://jsonplaceholder.typicode.com/todos/1 | json_pp -f json -t json > todo.json` this outputs `todo.json` for me. – Ibrahim.H Mar 13 '23 at 07:15
97

I am guessing that you want to prettify the JSON output. That could be achieved using python:

curl http://localhost:8880/test.json | python -mjson.tool > out.json
Koray Tugay
  • 22,894
  • 45
  • 188
  • 319
0xbb
  • 1,106
  • 6
  • 4
  • 1
    Thanks for answering. But i am not using Python, im using shell. can u guide me on shell – Jams Dec 01 '14 at 22:46
  • @Jams that executes python from the shell giving your a pretty printed output. – agconti Aug 04 '16 at 20:19
  • 3
    Great solution as python comes preinstalled. – Gayan Weerakutti Mar 13 '17 at 07:15
  • 1
    That python script gives a "broken pipe" error when I add a " | head" to it. – Crouching Kitten Oct 18 '17 at 09:50
  • @CrouchingKitten I don't see that problem myself (maybe different versions), but that may be due to the `head` command cutting off the output being piped to it before reaching the end, thus "breaking" the pipe. If you allow `head` to consume the entire data stream from the pipe, I'd expect the error to not show up. – code_dredd Oct 14 '19 at 18:01
  • Best thing about this solution is that Python comes pre-installed. This is specially helpful for the places where you have no control over the applications that are installed in the system (e.g. You are not in sudo users' list) – Romeo Sierra Jul 20 '20 at 02:23
54

This is to add to of Gilles' Answer. There are many ways to get this done but personally I prefer something lightweight, easy to remember and universally available (e.g. come with standard LTS installations of your preferred Linux flavor or easy to install) on common *nix systems.

Here are the options in their preferred order:

Python Json.tool module

echo '{"foo": "lorem", "bar": "ipsum"}' | python -mjson.tool

pros: almost available everywhere; cons: no color coding


jq (may require one time installation)

echo '{"foo": "lorem", "bar": "ipsum"}' | jq

cons: needs to install jq; pros: color coding and versatile


json_pp (available in Ubuntu 16.04 LTS)

echo '{"foo": "lorem", "bar": "ipsum"}' | json_pp

For Ruby users

gem install jsonpretty
echo '{"foo": "lorem", "bar": "ipsum"}' | jsonpretty
Zack
  • 3,819
  • 3
  • 27
  • 48
Zhenhua
  • 2,389
  • 15
  • 10
  • Thanks @zhenhua! I used `cat myFile.json | python -mjson.tool >> myFilePretty.json` then opened in my editor (vim, don't hate) and got colors. – essa Jun 14 '18 at 03:11
41

You can use the json node module:

npm i -g json

then simply append | json after curl. curl http://localhost:8880/test.json | json

Michael Johansen
  • 964
  • 6
  • 18
NanoNova
  • 829
  • 10
  • 19
  • I would remove the comment and part about sudo if you don't recommend it haha. I agree it's not necessary. Thanks for the answer – eazy_g Jul 27 '20 at 22:24
  • 6
    If you don't want to install the `json` module, you could use it directly with `npx`: `curl http://localhost:8880/test.json | npx json` – Michael Johansen Nov 30 '20 at 12:17
  • this should be the accepted response IMO, this is the easiest way to go. no options required, outputs exactly what you'd expect – nico Mar 02 '22 at 20:20
37

You can install jq and make the query like below:

curl http://localhost:8080/observations/station/221 | jq

enter image description here

Arefe
  • 11,321
  • 18
  • 114
  • 168
28
python -m json.tool
Curl http://127.0.0.1:5000/people/api.json | python -m json.tool

can also help.

Vega
  • 27,856
  • 27
  • 95
  • 103
Vishnu
  • 293
  • 3
  • 10
17

I found json_reformat to be very handy. So I just did the following:

curl http://127.0.0.1:5000/people/api.json | json_reformat

that's it!

Ahsan
  • 3,845
  • 2
  • 36
  • 36
13

Motivation: You want to print prettify JSON response after curl command request.

Solution: json_pp - commandline tool that converts between some input and output formats (one of them is JSON). This program was copied from json_xs and modified. The default input format is json and the default output format is json with pretty option.

Synposis: json_pp [-v] [-f from_format] [-t to_format] [-json_opt options_to_json1[,options_to_json2[,...]]]

Formula: <someCommand> | json_pp

Example:

Request

curl -X https://jsonplaceholder.typicode.com/todos/1 | json_pp 

Response

{
   "completed" : false,
   "id" : 1,
   "title" : "delectus aut autem",
   "userId" : 1
}
Zack
  • 3,819
  • 3
  • 27
  • 48
avivamg
  • 12,197
  • 3
  • 67
  • 61
5

Check out curljson

$ pip install curljson
$ curljson -i <the-json-api-url>
mitnk
  • 3,127
  • 2
  • 21
  • 28
4

A lot more features (slice, filter and map and transform structured ) apart from formatting.

https://stedolan.github.io/jq/

curl 'https://api.github.com/repos/stedolan/jq/commits?per_page=5' | jq '.[0]'
{
  "sha": "d25341478381063d1c76e81b3a52e0592a7c997f",
  "commit": {
    "author": {
      "name": "Stephen Dolan",
      "email": "mu@netsoc.tcd.ie",
      "date": "2013-06-22T16:30:59Z"
    },
    "committer": {
      "name": "Stephen Dolan",
      "email": "mu@netsoc.tcd.ie",
      "date": "2013-06-22T16:30:59Z"
    },
    "message": "Merge pull request #162 from stedolan/utf8-fixes\n\nUtf8 fixes. Closes #161",
    "tree": {
      "sha": "6ab697a8dfb5a96e124666bf6d6213822599fb40",
      "url": "https://api.github.com/repos/stedolan/jq/git/trees/6ab697a8dfb5a96e124666bf6d6213822599fb40"
    },
    "url": "https://api.github.com/repos/stedolan/jq/git/commits/d25341478381063d1c76e81b3a52e0592a7c997f",
    "comment_count": 0
  },
  "url": "https://api.github.com/repos/stedolan/jq/commits/d25341478381063d1c76e81b3a52e0592a7c997f",
  "html_url": "https://github.com/stedolan/jq/commit/d25341478381063d1c76e81b3a52e0592a7c997f",
  "comments_url": "https://api.github.com/repos/stedolan/jq/commits/d25341478381063d1c76e81b3a52e0592a7c997f/comments",
  "author": {
    "login": "stedolan",
    "id": 79765,
    "avatar_url": "https://avatars.githubusercontent.com/u/79765?v=3",
    "gravatar_id": "",
    "url": "https://api.github.com/users/stedolan",
    "html_url": "https://github.com/stedolan",
    "followers_url": "https://api.github.com/users/stedolan/followers",
    "following_url": "https://api.github.com/users/stedolan/following{/other_user}",
    "gists_url": "https://api.github.com/users/stedolan/gists{/gist_id}",
    "starred_url": "https://api.github.com/users/stedolan/starred{/owner}{/repo}",
    "subscriptions_url": "https://api.github.com/users/stedolan/subscriptions",
    "organizations_url": "https://api.github.com/users/stedolan/orgs",
    "repos_url": "https://api.github.com/users/stedolan/repos",
    "events_url": "https://api.github.com/users/stedolan/events{/privacy}",
    "received_events_url": "https://api.github.com/users/stedolan/received_events",
    "type": "User",
    "site_admin": false
  },
  "committer": {
    "login": "stedolan",
    "id": 79765,
    "avatar_url": "https://avatars.githubusercontent.com/u/79765?v=3",
    "gravatar_id": "",
    "url": "https://api.github.com/users/stedolan",
    "html_url": "https://github.com/stedolan",
    "followers_url": "https://api.github.com/users/stedolan/followers",
    "following_url": "https://api.github.com/users/stedolan/following{/other_user}",
    "gists_url": "https://api.github.com/users/stedolan/gists{/gist_id}",
    "starred_url": "https://api.github.com/users/stedolan/starred{/owner}{/repo}",
    "subscriptions_url": "https://api.github.com/users/stedolan/subscriptions",
    "organizations_url": "https://api.github.com/users/stedolan/orgs",
    "repos_url": "https://api.github.com/users/stedolan/repos",
    "events_url": "https://api.github.com/users/stedolan/events{/privacy}",
    "received_events_url": "https://api.github.com/users/stedolan/received_events",
    "type": "User",
    "site_admin": false
  },
  "parents": [
    {
      "sha": "54b9c9bdb225af5d886466d72f47eafc51acb4f7",
      "url": "https://api.github.com/repos/stedolan/jq/commits/54b9c9bdb225af5d886466d72f47eafc51acb4f7",
      "html_url": "https://github.com/stedolan/jq/commit/54b9c9bdb225af5d886466d72f47eafc51acb4f7"
    },
    {
      "sha": "8b1b503609c161fea4b003a7179b3fbb2dd4345a",
      "url": "https://api.github.com/repos/stedolan/jq/commits/8b1b503609c161fea4b003a7179b3fbb2dd4345a",
      "html_url": "https://github.com/stedolan/jq/commit/8b1b503609c161fea4b003a7179b3fbb2dd4345a"
    }
  ]
}
Tasawar Hussain
  • 650
  • 7
  • 21
1

With :

curl <...> | xidel - -se '$json'

xidel can probably retrieve the JSON for you as well.

Reino
  • 3,203
  • 1
  • 13
  • 21
0

In real cases sometimes you need use -i option in curl to get headers

$ curl https://petstore.swagger.io/v2/pet/1 -i
HTTP/2 200 
date: Wed, 26 Jul 2023 16:46:39 GMT
content-type: application/json
access-control-allow-origin: *
access-control-allow-methods: GET, POST, DELETE, PUT
access-control-allow-headers: Content-Type, api_key, Authorization
server: Jetty(9.2.9.v20150224)

{"id":1,"category":{"id":0,"name":"zbyszek"},"name":"pies","photoUrls":["string"],"tags":[{"id":0,"name":"pies"}],"status":"available"}

in that case, pure jq/json_pp/json_reformat will fail:

curl https://petstore.swagger.io/v2/pet/1 -i | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   138    0   138    0     0    108      0 --:--:--  0:00:01 --:--:--   109
parse error: Invalid numeric literal at line 1, column 7

I offer to use ijq https://github.com/1frag/ijq that ignores non-json lines

curl https://petstore.swagger.io/v2/pet/1 -i | ijq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    51    0    51    0     0     41      0 --:--:--  0:00:01 --:--:--    41
HTTP/2 404 
date: Wed, 26 Jul 2023 16:51:58 GMT
content-type: application/json
access-control-allow-origin: *
access-control-allow-methods: GET, POST, DELETE, PUT
access-control-allow-headers: Content-Type, api_key, Authorization
server: Jetty(9.2.9.v20150224)
{
    "code": 1,
    "message": "Pet not found",
    "type": "error"
}
ifrag
  • 111
  • 4