I am quite new to API testing and any topics in networking in general.
I'm attempting to retrieve BC, Canada school names and rankings from a website. The target data is in the right table, available after a prompt to choose a province (here I selected British Columbia). I am using the chrome developer tools to analyze the requests/responses after selecting the province, however I am getting gibberish when expecting a JSON response.
After choosing the province, 3 XMLHttpRequests are made to compareschoolrankings.org/api/v1/ with response headers of
content-type: application/json
I assume the responses to these requests hold my target data, however the response content is gibberish when I would otherwise expect it in json format, in example:
3dd3U2FsdGVkX1/TDJgJ2Kpx3ekEf3yT9DaZMp8nDRMZJlP85M8RWOruj5tm1Qu6c2UF1ifJVFMU8+XbeXvIbWZ/Or ... bo/XkaOUHOnWGMhpFIC8mYz
Here is one request (it's header) that I expect is requesting the target data:
:authority: www.compareschoolrankings.org
:method: GET
:path: /api/v1/schools.json?province=bc&ht=NzQ1Mjg
:scheme: https
accept: application/json, text/plain, */*
accept-encoding: gzip, deflate, br
accept-language: en-US,en;q=0.9
cookie: _ga=GA1.2.217437611.1609538525; _gid=GA1.2.1976080400.1609538525; _gat_UA-3850680-10=1; _hjTLDTest=1; _hjid=1d1327a2-dd14-4388-b08f-ef670f6178cf; _hjFirstSeen=1
referer: https://www.compareschoolrankings.org/
sec-fetch-dest: empty
sec-fetch-mode: cors
sec-fetch-site: same-origin
user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.66 Safari/537.36
x-requested-with: XMLHttpRequest
Here is the corresponding response header:
accept-ranges: bytes
age: 14440
cache-control: max-age=21600, public
content-encoding: gzip
content-language: en
content-length: 448428
content-type: application/json
date: Fri, 01 Jan 2021 22:02:13 GMT
etag: W/"1609524093"
expires: Sun, 19 Nov 1978 05:00:00 GMT
last-modified: Fri, 01 Jan 2021 18:01:33 GMT
server: nginx
strict-transport-security: max-age=300
vary: Accept-Encoding, Cookie
via: 1.1 varnish, 1.1 varnish, 1.1 varnish
x-cache: MISS, HIT, HIT
x-cache-hits: 0, 1, 1
x-content-type-options: nosniff
x-drupal-cache: MISS
x-frame-options: SAMEORIGIN
x-generator: Drupal 8 (https://www.drupal.org)
x-pantheon-styx-hostname: styx-fe3fe4-h-6c4765d776-86gmx
x-served-by: cache-yyz4534-YYZ, cache-sea4433-SEA, cache-sea4483-SEA
x-styx-req-id: 623d1ec1-4c5b-11eb-bbc4-620e110c7f7f
x-timer: S1609538534.794192,VS0,VE0
x-ua-compatible: IE=edge
Question: Why is the response not in a JSON format when both the request and response headers indicate the content to be so? Where should I be looking to retreive my target data?
Any help or references would be much appreciated!
