Python, jsonpath_ng: Exception: Parse error at 1:4 near token ? (?)

Question

How do I get a specific list of information from JSON through text field input?

Here is my sample JSON data (retrieved via URL). Actually, there are hundreds of items.

{
"items": [
    {
        "timestamp": "2020-04-03T23:24:27+08:00",
        "carpark_data": [
            {
                "carpark_info": [
                    {
                        "total_lots": "91",
                        "lot_type": "C",
                        "lots_available": "36"
                    }
                ],
                "carpark_number": "HE12",
                "update_datetime": "2020-04-03T23:23:34"
            },
            {
                "carpark_info": [
                    {
                        "total_lots": "583",
                        "lot_type": "C",
                        "lots_available": "474"
                    }
                ],
                "carpark_number": "HLM",
                "update_datetime": "2020-04-03T23:23:23"
            },
            {
                "carpark_info": [
                    {
                        "total_lots": "322",
                        "lot_type": "C",
                        "lots_available": "126"
                    }
                ],
                "carpark_number": "RHM",
                "update_datetime": "2020-04-03T23:23:35"
            },
            {
                "carpark_info": [
                    {
                        "total_lots": "97",
                        "lot_type": "C",
                        "lots_available": "85"
                    }
                ],
                "carpark_number": "BM29",
                "update_datetime": "2020-04-03T23:23:43"
            },
            {
                "carpark_info": [
                    {
                        "total_lots": "96",
                        "lot_type": "C",
                        "lots_available": "75"
                    }
                ],
                "carpark_number": "Q81",
                "update_datetime": "2020-04-03T23:23:21"
            },
            {
                "carpark_info": [
                    {
                        "total_lots": "173",
                        "lot_type": "C",
                        "lots_available": "55"
                    }
                ],
                "carpark_number": "C20",
                "update_datetime": "2020-04-03T23:23:24"
            }

        ]
    }
]}

I am writing the following code to get carpark_data when the user keys in a respective carpark_number in the text field, for example: HLM

My expected output is

[
  {
    "carpark_info": [
      {
        "total_lots": "583",
        "lot_type": "C",
        "lots_available": "474"
      }
    ],
    "carpark_number": "HLM",
    "update_datetime": "2020-04-03T23:23:23"
  }
]

I have been trying as follows, but not sure if it is possible..

from kivymd.uix.dialog import MDInputDialog
from urllib import parse
from kivy.network.urlrequest import UrlRequest
from kivy.app import App
import json
from jsonpath_ng import jsonpath,parse


class SearchMenu(MDInputDialog):
   title = 'Search by Address'
   text_button_ok = 'Search'

   def __init__(self):
       super().__init__()
       self.size_hint = [.9, .3]
       self.events_callback = self.callback

   def callback(self,*args):
       address = self.text_field.text
       self.get()
       print(address)

   def get(self):
       url= "https://api.data.gov.sg/v1/transport/carpark-availability"
       UrlRequest(url,on_success=self.success, on_failure=self.failure, on_error=self.error)


   def success(self,urlrequest, result):
       print("ok")
       data=[match.value for match in parse(('$..carpark_data[?(@.carpark_number =address)]')).find(result)]
       json_data=json.dumps(data, indent =2 ,separators=(',', ':'))
       print(json_data)

   def failure(self,urlrequest, result):
       print(result)
       print("failure")

   def error(self,urlrequest, result):
       print(result)
       print("error")

Answer 1

Good question. I think you are pretty close; using this code worked for me:

import json
from jsonpath_ng import jsonpath
from jsonpath_ng.ext import parse
json_str = '''{"items":[{"timestamp":"2020-04-03T23:24:27+08:00","carpark_data":[{"carpark_info":[{"total_lots":"91","lot_type":"C","lots_available":"36"}],"carpark_number":"HE12","update_datetime":"2020-04-03T23:23:34"},{"carpark_info":[{"total_lots":"583","lot_type":"C","lots_available":"474"}],"carpark_number":"HLM","update_datetime":"2020-04-03T23:23:23"},{"carpark_info":[{"total_lots":"322","lot_type":"C","lots_available":"126"}],"carpark_number":"RHM","update_datetime":"2020-04-03T23:23:35"},{"carpark_info":[{"total_lots":"97","lot_type":"C","lots_available":"85"}],"carpark_number":"BM29","update_datetime":"2020-04-03T23:23:43"},{"carpark_info":[{"total_lots":"96","lot_type":"C","lots_available":"75"}],"carpark_number":"Q81","update_datetime":"2020-04-03T23:23:21"},{"carpark_info":[{"total_lots":"173","lot_type":"C","lots_available":"55"}],"carpark_number":"C20","update_datetime":"2020-04-03T23:23:24"}]}]}'''
json_obj = json.loads(json_str)
jsonpath_expr = parse('$..carpark_data[?(@.carpark_number == "HLM")]')
print([match.value for match in jsonpath_expr.find(json_obj)])

First, I think you have some minor syntax issues/typos in your path query

• the equal operator is actually == (I know, the readme on Github tells otherwise! It's already reported.)
• you need to wrap the search string with double quotes, i.e. in your case it shout read like this parse('$..carpark_data[?(@.carpark_number == "'+address+'")]'
• (also, your sample JSON was missing the closing } it's probably only absent in your post but not your code, right?)

Second, probably most important, you need to use a different parse import when using path filters [?(..)]:

from jsonpath_ng import jsonpath
from jsonpath_ng.ext import parse   

I literally found this out by reading the jsonpath_ng tests on Github and by trial and error. Fun!