1

I am facing this error while making request to fetch json from api.

I can get json data using the "/v1/articles' path.

conn = http.client.HTTPSConnection("api.xxxx.com.tr")
    headers = {
    'accept': "application/json", 
    'apikey': "cd6b6c96799847698d87dec9f9a731d6"
    }
    filter = "daily"
    conn.request("GET", "/v1/articles", headers=headers)
    reader = codecs.getreader("utf-8")
    res = conn.getresponse()
    data = json.load(reader(res))
    json.dumps(data)
    return data

But i am having JSONDecodeError if i set filter. Code:

conn = http.client.HTTPSConnection("api.xxxx.com.tr")
    headers = {
    'accept': "application/json", 
    'apikey': "cd6b6c96799847698d87dec9f9a731d6"
    }
    conn.request("GET", "/v1/articles?$filter=Path eq '/daily/'", headers=headers)
    reader = codecs.getreader("utf-8")
    res = conn.getresponse()
    data = json.load(reader(res))
    json.dumps(data)
    return data

I tried same filter using Postman with no error and i can get Json data.

Returned Json data from Postman:

[
    {
        "Id": "40778196",
        "ContentType": "Article",
        "CreatedDate": "2018-03-20T08:28:05.385Z",
        "Description": "İspanya'da 2016 yılında çalınan lüks otomobil, şasi numarası değiştirilerek Bulgaristan üzerinden getirildiği Türkiye'de bulundu.",
        "Files": [
            {
                "FileUrl": "http://i.xxxx.com/i/xxxx/98/620x0/5ab0c6a9c9de3d18a866eb54.jpg",
                "Metadata": {
                    "Title": "",
                    "Description": ""
                }
            }
        ],
        "ModifiedDate": "2018-03-20T08:32:12.001Z",
        "Path": "/gundem/",
        "StartDate": "2018-03-20T08:32:12.001Z",
        "Tags": [
            "ispanya",
            "Araç",
            "Hırsız",
            "Dolandırıcı"
        ],
        "Title": "İspanya'da çalınan lüks araç Türkiye'de bulundu!",
        "Url": "http://www.xxxx.com.tr/gundem/ispanyada-calinan-luks-arac-turkiyede-bulundu-40778196"
    }
 ]

I can not figure out the problem. It would be great if anyone help me about this issue. Thank you.

darkfolcer
  • 172
  • 1
  • 3
  • 17
  • it is very hard to solve cause I don't have access to your resource, I can only suggest using a more robust client like **requests** that might solve the issue http://docs.python-requests.org/en/master/ – shahaf Mar 20 '18 at 08:39
  • `conn.request("GET", "/v1/articles?$filter=Path eq '/daily/'", headers=headers)` Are you sure the url should have `$`? – DeepSpace Mar 20 '18 at 08:39
  • @DeepSpace yes url should have it. – darkfolcer Mar 20 '18 at 08:41
  • I really, really doubt that. And it shouldn't have spaces or quotes either. What is the API you are connecting to? – Daniel Roseman Mar 20 '18 at 09:13
  • There are other odd things about this code too. For instance, `json.dumps(data)` on its own does *literally nothing*. – Daniel Roseman Mar 20 '18 at 09:13
  • @DanielRoseman api docs: https://developers.hurriyet.com.tr/docs/versions/1.0/resources/article If it shouldn't have spaces or quotes how come the postman response me with json data? – darkfolcer Mar 20 '18 at 10:26
  • That page shows examples of URLs, and they don't have $ or spaces - those characters are encoded properly. – Daniel Roseman Mar 20 '18 at 13:10

2 Answers2

4

I finally figured out the problem! Using the requests library have solved my problem now I can filter the api request.

data = requests.get('https://api.xxxxx.com.tr/v1/articles', headers =
headers, params={"$filter":"Path eq '/xxxxxx/'"}).json()

I am leaving this answer here for anyone else who can need this solution in the future. Thanks for all your suggestions.

darkfolcer
  • 172
  • 1
  • 3
  • 17
3

The problem is in the following line

data = json.load(reader(res))

when your response is not a json string, JSONDecodeError occurs. so, add an additional logic to see if the response is None or a json string. First thing, print the reader(res) and see what the return is

rawwar
  • 4,834
  • 9
  • 32
  • 57
  • I have tried to print reader(res) this is what returned. – darkfolcer Mar 20 '18 at 08:51
  • can you look into this link, https://stackoverflow.com/questions/35036921/asyncio-decode-utf-8-with-streamreader – rawwar Mar 20 '18 at 09:53
  • oh, can you pickle that output and share it here, may be i can take a look – rawwar Mar 20 '18 at 11:09
  • i included the traceback into the question. thank you for the effort. – darkfolcer Mar 20 '18 at 11:25
  • no, not the traceback. to correct it we need to decode the object that is being returned in the response. so, if you could save the object as pickle file and upload it here, i can try getting the json out of it – rawwar Mar 20 '18 at 11:33