Python HTTP Request Returns 404 or Bytes

Question

I'm trying to use a Python script to call the API detailed in the link below:

https://developer.wmata.com/docs/services/gtfs/operations/5cdc51ea7a6be320cab064fe?

When I use the code below, it always returns a 404 error:

import requests
import json


def _url(path):
    return "http://api.wmata.com" + path


def pull_data():
    return requests.get(_url("/gtfs/bus-gtfsrt-tripupdates.pb"), params=params)


def jprint(obj):
    # create a formatted string of the Python JSON object
    text = json.dumps(obj, sort_keys=True, indent=4)
    print(text)


# authenticate with your api key
params = {
    "apiKey": "mykey",
}

response = pull_data()
print(response)
jprint(response.json())

I have also tried using the python code provided in the link, but it returns meaningless response content as shown below. Any attempts to decode the content have been unsuccessful.

Request-Context: appId=cid-v1:2833aead-1a1f-4ffd-874e-ef3a5ceb1de8
Cache-Control: public, must-revalidate, max-age=5
Date: Thu, 11 Feb 2021 22:05:31 GMT
ETag: 0x8D8CED90CC8419C
Content-Length: 625753
Content-MD5: fspEFl7LJ8QbZPgf677WqQ==
Content-Type: application/octet-stream
Expires: Thu, 11 Feb 2021 22:05:37 GMT
Last-Modified: Thu, 11 Feb 2021 22:04:49 GMT



1.0�Ԗ��

1818410080�


181841008020210211*52!�Ԗ�"19032("�Ԗ�"18173(#�Ԗ�"7779($�Ֆ�"18174(%�Ֆ�"7909(&�Ֆ�"7986('�Ֆ�"8039((�Ֆ�"8130()�֖�"8276(+�֖�"8313(,�ז�"8403(-�ז�"8452(.�ז�"8520(/�ؖ�"8604(1�ؖ�"8676(
7070 �Ӗ�(����������

1814174080�


181417408020210211*P129�Ԗ�"2373(:�Ԗ�"2387(;�Ԗ�"17296(=�Ֆ�"17212(>�֖�"2444(?�֖�"2493(@�֖�"2607(A�֖�"14633(B�֖�"2784(C�ז�"2832(D�ז�"2843(E�ז�"2848(F�ז�"2875(G�ؖ�"2945(H�ؖ�"2987(I�ؖ�"21946(K�ٖ�"14636(L�ٖ�"3122(M�ٖ�"3227(N�ٖ�"3308(O�ٖ�"3411(P�ٖ�"3500(Q�ٖ�"3539(R�ٖ�"14637(S�ږ�"3685(T�ږ�"15195(U�ږ�"15196(V�ۖ�"4243(W�ۖ�"4443(X�ۖ�"4517(Y�ۖ�"4631([�ܖ�"11962(
8002 �Ӗ�(/�

1825989080�


182598908020210211*7Y�Ӗ�"2158(�Ԗ�"2215(�Ԗ�"2259(�Ԗ�"2292(�Ԗ�"2299(�Ֆ�"18701(�Ֆ�"2310( �Ֆ�"2245(!�Ֆ�"2174("�Ֆ�"1987(#�֖�"1937(%�֖�"1864(
3191 �Ӗ�(��


1819988080�

Any guidance or direction would be greatly appreciated!

score 1 · Answer 1 · answered Feb 11 '21 at 22:14

1

Try changing your URL so that it is using https instead of http. The documentation that you have linked at Bus RT Trip Updates seems to indicate that https is required.

Change this:

def _url(path):
return "http://api.wmata.com" + path

to make it this:

def _url(path):
return "https://api.wmata.com" + path

answered Feb 11 '21 at 22:14

MCcodemasher

98
4

Thanks for the help but I received a 401 response with the message "Access denied due to missing subscription key. Make sure to include subscription key when making requests to an API." However, I tried including the API key both in the headers and in the query params with no success. – Mike Feb 11 '21 at 22:40

serghei · Accepted Answer · 2021-02-11T22:58:02.430

1

Change as pull_data function as follows:

def pull_data():
    return requests.get(_url("/gtfs/bus-gtfsrt-tripupdates.pb"), headers=headers)

Then rename params module global variable to headers .

headers = {"apiKey": "mykey"}

WMATA looks for a apiKey in the headers, not in the query params.

Update: I noticed they use api_key for some samples, and apiKey for another ones. For example see: https://developer.wmata.com/docs/services/gtfs/operations/5cdc51ea7a6be320cab064fe

Update 2: Notice the content type in the response headers :

print(response.headers['content-type'])
# application/octet-stream

it is not a JSON. You can get contents as follows:

print(response.content)

Worked example:

import requests

API_URL = 'https://api.wmata.com'


def _prepare_url(path):
    return f'{API_URL}/{path.lstrip("/")}'


def pull_data(**options):
    url = _prepare_url('/gtfs/bus-gtfsrt-tripupdates.pb')
    return requests.get(url, **options)


response = pull_data(headers={'api_key': 'secret'})
print(response.content)

edited Feb 11 '21 at 22:58

answered Feb 11 '21 at 22:26

serghei

3,069
2
30
48

1

Thanks for the help, but I'm still getting the same 404 response with your changes. – Mike Feb 11 '21 at 22:33
@Mike I noticed they use `api_key` for some samples, and `apiKey` for another ones. Could you try `api_key`? – serghei Feb 11 '21 at 22:39
1

Okay using api_key in combination with @MCcodemasher's suggestion returned a successful 200 response! However, response.json() returns the following error: "JSONDecodeError: Expecting value: line 3 column 1 (char 3)." Could this have something to do with it being an application/octet-stream? – Mike Feb 11 '21 at 22:48
json.loads(response) returns this error: "TypeError: the JSON object must be str, bytes or bytearray, not Response" – Mike Feb 11 '21 at 22:54
response.content returns a meaningless stream: "b'\n\r\n\x031.0\x10\x00\x18\xf9\xb9\x9a\x81\x06\x12\xc6\x03\n\n1832238090\x1a..." – Mike Feb 12 '21 at 15:23
@Mike Yes, the response is really hard to read. However, this is exactly what the API returns. This is why you shouldn't use `response.json()`. In general `response.headers['content-type']` tells you about response format and this is **not** a JSON. In any case, working with the return type is beyond the scope of the question you originally asked. I hope my answer helped you correctly prepare the request and get a response from the API server. – serghei Feb 13 '21 at 10:28

Python HTTP Request Returns 404 or Bytes

2 Answers2

Linked