1

I have the following PHP script, which is suppose to go to a URL and fetch the JSON output from that page:

function nexusCheck($nexusUrl, $toolName, $verboseBool, $httpCode, $connectTime, $totalTime) {
    global $date;
    $username = 'user';
    $password = 'password';
    $URL = "$nexusUrl" . "/service/siesta/capabilities";
    $handle = curl_init();
    $timeout = 30;
    curl_setopt($handle, CURLOPT_URL, $URL);
    curl_setopt($handle, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt($handle, CURLOPT_CONNECTTIMEOUT, $timeout);
    curl_setopt($handle, CURLOPT_HTTPAUTH, CURLAUTH_ANY);
    //curl_setopt($handle, CURLOPT_HEADER, array('Accept: application/json' , 'Content-Type: application/json'));
    curl_setopt($handle, CURLOPT_HEADER, "Accept: application/json");
    curl_setopt($handle, CURLOPT_USERPWD, "{$username}:{$password}");
    //var_dump(curl_setopt($handle, CURLOPT_USERPWD, "{$username}:{$password}"));
    $result = curl_exec($handle);

    //echo encode_json($result);
    //echo decode_json($result);
    echo($result);

    curl_close($handle);
}

So as mentioned the output should be JSON, however, right now the output is set as only text. I am not sure why this is happening because when I actually do the actual curl on my terminal the output comes out as JSON!

My cURL statement is: curl -X GET -H "Accept: application/json" -u username:password "http://nexus.company.net/service/siesta/capabilities".

Any ideas why the output is just text? What am I doing wrong here?

EDIT: To clarify, I meant that the result that shows on the page isn't encoded as JSON data structure. However, when I do that curl statement in the terminal the output is encoded properly.

EDIT2: Okay so the output looks like this:

000d43932f6eb3f3truesmartproxy.event.publishrepositoryio-aws-internal-releasesfeedEnabledfalseIO AWS Company Internal Hosted ReleasestruefalseSmart Proxy: PublishActiveCategorySmart ProxyRepositoryIO AWS Company Internal Hosted Releases00028650f5fea946truesmartproxy.event.publishrepositoryio-thirdpartyfeedEnabledfalseIO 3rd partytruefalseSmart Proxy: PublishActiveCategorySmart ProxyRepositoryIO 3rd party000d43932f6eb3f1truesmartproxy.event.publishrepositoryio-aws-internal-snapshotsfeedEnabledfalseIO AWS Company Internal Hosted SnapshotstruefalseSmart Proxy: PublishActiveCategorySmart ProxyRepositoryIO AWS Company Internal Hosted Snapshots000d43932f6eb3f2falsesmartproxy.event.subscriberepositoryio-aws-internal-snapshotspreemptiveFetchfalsedeletetruefeedEnabledfalseIO AWS Company Internal Hosted SnapshotsfalsefalseSmart Proxy: SubscribeDisabledCategorySmart ProxyRepositoryIO AWS Company Internal Hosted Snapshots000d43932f6eb3f0truesmartproxy.event.publishrepositorypublicfeedEnabledfalsePublic RepositoriestruefalseSmart Proxy: PublishActiveCategorySmart ProxyRepositoryPublic Repositories00028650f5fea949truesmartproxy.event.subscriberepositoryio-core-releasespreemptiveFetchtruedeletetruefeedEnabledfalseIO Core Services ReleasestruefalseSmart Proxy: SubscribeActive<h2>Monitor State</h2> <br/> <div> <pre>CONNECTED</pre> </div> <br/> <h2>Remote Node ID</h2> <br/> <div> <pre>8FA5D941-31598C8F-B7C4203F-EDD413CA-B94B6C03</pre> </div> <br/> <h2>Remote Connection URI</h2> <br/> <div> <pre>ssl://nexus.company.net:51773</pre> </div> <br/> <h2>Remote Repository ID</h2> <br/> <div> <pre>core-release-proxy</pre> </div> <br/> <h2>Handshake Status</h2> <br/> <div> <pre>ENABLED</pre> </div> <br/> <h2>Remote Certificate Fingerprint</h2> <br/> <div> <pre>8F:A5:D9:41:31:59:8C:8F:B7:C4:20:3F:ED:D4:13:CA:B9:4B:6C:03</pre> </div> <br/> <h2>Remote Certificate PEM</h2> <br/> <div> <pre>-----BEGIN CERTIFICATE----- MIIDijCCAnKgAwIBAgIGAVQ0GItxMA0GCSqGSIb3DQEBBQUAMIGEMS0wKwYDVQQD DCRmNWQ3ODQ1MS1jNmJkLTQzOTQtYTU2OS1iODY5OWE4ZDk1MmUxDjAMBgNVBAsM BLAHBLAHBLAHBLAHBLAHBLHKLHJKDLJFKDLJKFLDJKFLDJKLFDJKFDJKLFDJKLFJ usgJWixHE5e3JZD7ZaBlhmKcBf/gF6HWNOrhh6exV/PTUnQreLaMSQ6gt/2zP/3e mCHR0iMOXPDCI8nKCGxPuNGoTR/aY5ZBihJeCPTSRoV6+6G0ATCqwM6QxePJk/yT 64f8rHRdD+6sJKr3lzBVXTIYGz0bVK+AKCRA+MaeQT0bcYU6UJMGF5jKPKUPEg== -----END CERTIFICATE----- </pre> </div> <br/> <h2>Remote Certificate Detail</h2> <br/> <div> <pre> [0] Version: 3 SerialNumber: 1461162904433 IssuerDN: CN=f5d78451-c6bd-4394-a569-b8699a8d952e,OU=Nexus,O=Sonatype,L=Silver Spring,ST=MD,C=US Start Date: Wed Apr 20 14:35:04 UTC 2016 Final Date: Fri Mar 27 14:35:04 UTC 2116 SubjectDN: CN=f5d78451-c6bd-4394-a569-b8699a8d952e,OU=Nexus,O=Sonatype,L=Silver Spring,ST=MD,C=US Public Key: RSA Public Key modulus: 98a2f8f258a8348f5b3bcee954cdf59dccb052c9bf17e1a514cdff759accc44e7f42e2773blahblahblahblajblahlbhalbhlahblhalbhalhblahblahblahblahlbhalbhalhblahblac99f586c2c2035b37889aff642120cdd147d54d3e09d8ad5c213b8bfae3133633660df75e1 public exponent: 10001 Signature Algorithm: SHA1WITHRSA Signature: aac0ce90c5e3c993fc93eb87fcac745d0feeac24 aaf79730555d32181b3d1b54af80282440f8c69e 413d1b71853a5093061798ca3ca50f12 </pre> </div> <br/> <h2>Shared Connection Detail</h2> <br/> <div> <pre>ActiveMQConnection {id=ID:ip-10-96-2-8-44490-1465326994959-58:1,clientId=ID:ip-10-96-2-8-44490-1465326994959-57:1,started=true}</pre> </div> <br/> CategorySmart ProxyRepositoryIO Core Services Releases00028650f5fea948truesmartproxy.event.publishrepositoryio-core-releasesfeedEnabledfalseIO Core Services ReleasestruefalseSmart Proxy: PublishActiveCategorySmart ProxyRepositoryIO Core Services Releases00028650f5fea947truesmartproxy.event.subscriberepositoryio-thirdpartypreemptiveFetchtruedeletetruefeedEnabledfalseIO 3rd partytruefalseSmart Proxy: SubscribeActive<h2>Monitor State</h2> <br/> <div> <pre>CONNECTED</pre> </div> <br/> <h2>Remote Node ID</h2> <br/> <div> <pre>8FA5D941-B94B6C03</pre> </div> <br/> <h2>Remote Connection URI</h2> <br/> <div> <pre>ssl://nexus.company.net:51773</pre> </div> <br/> ..................................................................................................................................................

Instead of (what I get from the command line):

[ {
  "capability" : {
    "id" : "000d43932f6eb3f3",
    "notes" : null,
    "enabled" : true,
    "typeId" : "smartproxy.event.publish",
    "properties" : [ {
      "key" : "repository",
      "value" : "io-aws-internal-releases"
    }, {
      "key" : "feedEnabled",
      "value" : "false"
    } ]
  },
  "description" : "IO AWS Company Internal Hosted Releases",
  "active" : true,
  "error" : false,
  "typeName" : "Smart Proxy: Publish",
  "stateDescription" : "Active",
  "status" : null,
  "tags" : [ {
    "key" : "Category",
    "value" : "Smart Proxy"
  }, {
    "key" : "Repository",
    "value" : "IO AWS Company Internal Hosted Releases"
  } ]
}, {
  "capability" : {
    "id" : "00028650f5fea946",
    "notes" : null,
    "enabled" : true,
    "typeId" : "smartproxy.event.publish",
    "properties" : [ {
      "key" : "repository",
      "value" : "io-thirdparty"
    }, {
      "key" : "feedEnabled",
      "value" : "false"
    } ]
  },
  "description" : "IO 3rd party",
  "active" : true,
  "error" : false,
  "typeName" : "Smart Proxy: Publish",
  "stateDescription" : "Active",
  "status" : null,
  "tags" : [ {
    "key" : "Category",
    "value" : "Smart Proxy"
  }, {
    "key" : "Repository",
    "value" : "IO 3rd party"
  } ].........................................................................................................................

EDIT3: So rid said it better, I think what's happening is when I try to make the same request in PHP as you do with command line cURL, the server returns a different format when you make the request from PHP.

EDIT4: So using HTTPHEADER instead of HEADER and sending the header as an array did the trick, however there's a small issue where if the value of a key has HTML tags the browser formats the page using those tags. So like this:

This is how it should look like:

  "stateDescription" : "Active",
  "status" : "<h2>Node ID</h2>\n<br/>\n<div>\n    <pre>8FA5D941-31598C8F-B7C4203F-EDD413CA-B94B6C03</pre>\n</div>\n\n<br/>\n\n<h2>Fingerprint</h2>\n<br/>\n<div>\n    <pre>8F:A5:D9:41B9:4B:6C:03</pre>\n</div>\n\n<br/>\n\n<h2>Certificate</h2>\n<br/>\n<div>\n    <pre>-----BEGIN CERTIFICATE-----\nMIIDijCCAnKgAwIBAgIGAVQ0GItxMA0GCSqGSIb3DQEBBQUAMIGEMS0wKwYDVQQD\nDCRmNWQ3ODQ1MS1jNmJkLTQzOTQtYTU2OS1iODY5OWE4ZDk1MmUxDjAMBgNVBAsM\nBU5leHVzMREw6exV/PTUnQreLaMSQ6gt/2zP/3e\nmCHR0iMOXPDCI8nKCGxPuNGoTR/aY5ZBihJeCPTSRoV6+6G0ATCqwM6QxePJk/yT\n64f8rHRdD+6sJKr3lzBVXTIYGz0bVK+AKCRA+MaeQT0bcYU6UJMGF5jKPKUPEg==\n-----END CERTIFICATE-----\n</pre>\n        </div>\n\n<br/>\n\n<h2>Detail</h2>\n<br/>\n<div>\n    <pre>[\n[\n  Version: V3\n  Subject: C=US, ST=MD, L=Silver Spring, O=Sonatype, OU=Nexus, CN=f5d78451-c6bd-4394-a569-b8699a8d952e\n  Signature Algorithm: SHA1withRSA, OID = 1.2.840.113549.1.1.5\n\n  Key:  Sun RSA public key, 2048 bits\n  modulus: 19268587064139719085609236135424479219819536352682784106558145114346679106038989267437271230004063018121065830291975586740542592294259900243345540709378803877292510984799053572694764430270406900807137\n  public exponent: 65537\n  Validity: [From: Wed Apr 20 14:35:04 UTC 2016,\n               To: Fri Mar 27 14:35:04 UTC 2116]\n  Issuer: C=US, ST=MD, L=Silver Spring, O=Sonatype, OU=Nexus, CN=f5d78451-c6bd-4394-a569-b8699a8d952e\n  SerialNumber: [    01543418 8b71]\n\n]\n  Algorithm: [SHA1withRSA]\n  Signature:\n0000: 72 16 A7 0E 0A 83 74 97   BC 9E FD FD F6 53 34 33  r.....t......S43\n0010: AF 0D 9B 67 84 73 2E A2   93 79 81 7F 41 B8 91 C9   C0 CE 90 C5 E3 C9 93  F.z....0........\n00D0: FC 93 EB 87 FC AC 74 5D   0F EE AC 24 AA F7 97 30  ......t]...$...0\n00E0: 55 5D 32 18 1B 3D 1B 54   AF 80 28 24 40 F8 C6 9E  U]2..=.T..($@...\n00F0: 41 3D 1B 71 85 3A 50 93   06 17 98 CA 3C A5 0F 12  A=.q.:P.....<...\n\n]</pre>\n</div>\n",
  "tags" : [ {
    "key" : "Category",
    "value" : "Smart Proxy"
  } ]

Instead I get this:

enter image description here

Community
  • 1
  • 1
Fadi
  • 1,329
  • 7
  • 22
  • 40
  • 3
    technically json **IS** "just text". it's a text-encoding of a JS data structure. – Marc B Sep 13 '16 at 17:23
  • Well, okay my bad.. I meant that it's not encoded with a JSON data structure (which it does when I do it directly from the terminal). – Fadi Sep 13 '16 at 17:24
  • 1
    it's your job to call json_encode on the result, not curls. whoever programmed your terminal or browser programmed it to do that. now that youre the programmer, it is your job. – Robert Parham Sep 13 '16 at 17:26
  • 1
    what **IS** the output? `{"foo":"bar"}` and `42` are both completely valid json "text". – Marc B Sep 13 '16 at 17:30
  • Ok look at the 2nd edit.. I hope that explains it better.. I added lots of `.....` at the end to show that there's a lot more information that comes after that I just didn't want to show it all. (becuase like you get the idea lol). – Fadi Sep 13 '16 at 17:31
  • And @RobertParham I tried json_decode but that didn't work.. :( – Fadi Sep 13 '16 at 17:32
  • 1
    Oh, so what you mean is that when you try to make the same request in PHP as you do with command line cURL, the server returns a different format when you make the request from PHP? – rid Sep 13 '16 at 17:34
  • Yes! That's it! Lol.. – Fadi Sep 13 '16 at 17:35
  • Maybe the PHP cURL does a POST request by default? – rid Sep 13 '16 at 17:37
  • Try setting `CURLOPT_HTTPGET` to `1`. – rid Sep 13 '16 at 17:38
  • PHP cURL defaults to GET – Trey Sep 13 '16 at 17:38
  • @Trey, makes sense. Not sure what else could be different from the command line version though. – rid Sep 13 '16 at 17:39
  • @rid That's what's confusing me also.. I can't figure out why I get a different response from PHP than the command line.. :( – Fadi Sep 13 '16 at 17:40
  • 1
    This is the point where a debug proxy like [Charles](http://charlesproxy.com) could elucidate the mystery. – rid Sep 13 '16 at 17:41
  • Not sure why a GET vs POST would strip characters, the only other thing I can think of is wrapping the response in `
    `, but it shouldn't matter since JSON is printable in HTML
    – Trey Sep 13 '16 at 17:42
  • @Trey, me neither, I was just thinking the server could probably return a different format for a different method, since that's the only difference I'm seeing. – rid Sep 13 '16 at 17:42
  • @Trey, I don't think it's stripping characters, maybe that's a different format altogether, with fields separated by non-printable character or something like that. – rid Sep 13 '16 at 17:44
  • 1
    I think you're not using `CURLOPT_HEADER` right. `CURLOPT_HEADER` seems to be a boolean value. Try using `CURLOPT_HTTPHEADER` instead, and send it an array, not a string. – rid Sep 13 '16 at 17:46
  • So like this: `curl_setopt($handle, CURLOPT_HTTPHEADER, array("Accept: application/json"));` ? – Fadi Sep 13 '16 at 17:50
  • 1
    That should work, yes. – rid Sep 13 '16 at 17:53
  • That did work! However, now because there's a key called "status" that has some html tags (along with some other text) as the value, the browser is considering those as actual HTML tags and messing up the JSON format because of this.. Is there a way around this? I'll add an `EDIT4` to the OP to show you what I'm talking about. – Fadi Sep 13 '16 at 17:59
  • 1
    Do you want to just display the JSON data in the browser, not parse it and do something with it? – rid Sep 13 '16 at 18:00
  • Right. Parsing it is also something I'm looking to do (which I believe I can do easily now that this is working).. But, displaying it also is something I wanted to do. Also can you put your HTTPHEADER comment as an answer so I can mark it as the answer? :) – Fadi Sep 13 '16 at 18:06
  • 1
    If you want to escape HTML to display in the browser, use [`htmlspecialchars()`](http://php.net/htmlspecialchars). – rid Sep 13 '16 at 18:07
  • That worked like a charm.. Please post those as the answer so I can accept it as an answer! :) – Fadi Sep 13 '16 at 18:11

1 Answers1

-1

problem 1: you're not sending the Accept: application/json header because you are not using CURLOPT_HTTPHEADER, you are using CURLOPT_HEADER, and you aren't using an array, but a string. fix both of those.

problem 2: you don't specify any CURLOPT_USERAGENT, and it has become increasingly popular to block requests not containing an user-agent header.. you should fix that too, just to be sure.

and protip: enable CURLOPT_ENCODING as an empty string, your transfers will magically become faster by doing this.

hanshenrik
  • 19,904
  • 4
  • 43
  • 89