145

I am using curl in a bash script to fetch the response of a service as below,

response=$(curl -isb -H "Accept: application/json" "http://host:8080/some/resource")

Service response is of json type and on browser I could perfectly fine response.
However curl response has other unwanted things (such as set-cookie, content-length header in this case) and sometimes the actual response is eaten up.

Here is the output of echo $response >

 Set-Cookie: rack.session=BAh7CEkiD3Nlc3Npb25faWQGOgZFVEkiRWJlY2JiOTE2M2Q1ZWI4NThjMDdi%0AYjRiOWRjMGMxMGEwYTBkMjE3NmJhZDVjYzY4YjY4ZTlmMTE2ZGVkYWE3MTMG%0AOwBGS
SIJY3NyZgY7AEZJIiVhZmQ2MmUyZGMxMzFmOGEwMjg3NDlhNWM3YmVm%0AN2FjNwY7AEZJIg10cmFja2luZwY7AEZ7B0kiFEhUVFBfVVNFUl9BR0VOVAY7%0AAFRJIi00MTc0OGM2MWNkMzljZTYxNzY3ZjU0
Y2I5OTdiYWRkN2MyNTBkYmU4%0ABjsARkkiGUhUVFBfQUNDRVBUX0xBTkdVQUdFBjsAVEkiLWRhMzlhM2VlNWU2%0AYjRiMGQzMjU1YmZlZjk1NjAxODkwYWZkODA3MDkGOwBG%0A--ee97a62095e7d42129
 tontent-Length: 354c8; path=/; HttpOnly

This is breaking my response parsing logic.
I have seen this happening intermittently which is weird.

Is there a way to get "only" json response from cURL output?
I went through the curl documentation but could not see any thing/ or I could have missed it.

Nicholas K
  • 15,148
  • 7
  • 31
  • 57
Rishi
  • 5,869
  • 7
  • 34
  • 45

2 Answers2

188

You are specifying the -i option:

-i, --include

(HTTP) Include the HTTP-header in the output. The HTTP-header includes things like server-name, date of the document, HTTP-version and more...

Simply remove that option from your command line:

response=$(curl -sb -H "Accept: application/json" "http://host:8080/some/resource")
Remy Lebeau
  • 555,201
  • 31
  • 458
  • 770
  • True. But strange this is; it works directly from command line. But from bash script it behaves different, I m not able to see anything in the echo response except the letter "t" – Rishi Jun 17 '14 at 00:43
  • 4
    Changed echo "response is $response" from echo $response and now I m able to see the correct response. Dunno why would that be. Thanks! – Rishi Jun 17 '14 at 00:55
  • 1
    One learning is when used inside a command always "quote" a echo variable.. responsenew=$(echo "$response" | grep "abc") – Rishi Jun 17 '14 at 16:14
  • 1
    Rishi, the quotes around $response avoid breaking the contents at new lines and spaces – Pablo Adames Nov 21 '17 at 22:15
51

I was executing a get request an also want to see just the response and nothing else, seems like magic is done with -silent,-s option.

From the curl man page:

-s, --silent Silent or quiet mode. Don't show progress meter or error messages. Makes Curl mute. It will still output the data you ask for, potentially even to the terminal/stdout unless you redirect it.

Below the examples:

curl -s "http://host:8080/some/resource"
curl --silent "http://host:8080/some/resource"

Using custom headers

curl -s -H "Accept: application/json" "http://host:8080/some/resource"

Using POST method with a header

curl -s -X POST -H "Content-Type: application/json" "http://host:8080/some/resource" -d '{ "myBean": {"property": "value"}}'

You can also customize the output for specific values with -w, below the options I use to get just response codes of the curl:

curl -s -o /dev/null -w "%{http_code}" "http://host:8080/some/resource"
DeadlyChambers
  • 5,217
  • 4
  • 44
  • 61
Enrique Palacio
  • 1,521
  • 13
  • 8