EvaluateJsonPathAttributeCustom - Nifi

Question

I have to extract attributes from a json file that I receive from an api call using InvokeHTTPCustom. JSON FILE has the following sample data :

[
    {
        "input_index": 0,
        "candidate_index": 0,
        "delivery_line_1": "1 Santa Claus Ln",
        "last_line": "North Pole AK 99705-9901",
        "delivery_point_barcode": "997059901010",
        "components": {
            "primary_number": "1",
            "street_name": "Santa Claus",
            "street_suffix": "Ln",
            "city_name": "North Pole",
            "state_abbreviation": "AK",
            "zipcode": "99705",
            "plus4_code": "9901",
            "delivery_point": "01",
            "delivery_point_check_digit": "0"
        },
        "metadata": {
            "record_type": "S",
            "zip_type": "Standard",
            "county_fips": "02090",
            "county_name": "Fairbanks North Star",
            "carrier_route": "C004",
            "congressional_district": "AL",
            "rdi": "Commercial",
            "elot_sequence": "0001",
            "elot_sort": "A",
            "latitude": 64.75233,
            "longitude": -147.35297,
            "coordinate_license": 1,
            "precision": "Rooftop",
            "time_zone": "Alaska",
            "utc_offset": -9,
            "dst": true
        },
        "analysis": {
            "dpv_match_code": "Y",
            "dpv_footnotes": "AABB",
            "dpv_cmra": "N",
            "dpv_vacant": "N",
            "dpv_no_stat": "Y",
            "active": "Y",
            "footnotes": "L#"
        }
    },
    {
        "input_index": 1,
        "candidate_index": 0,
        "addressee": "Apple Inc",
        "delivery_line_1": "1 Infinite Loop",
        // truncated for brevity
    }
]

I have extracted all the required data such as address, state, city, primary_number, etc. However, when I try to extract latitude,longitude from metadata, it leads to failure in EvaluateJsonPathAttributeCustom processor. Other attributes, which are in string format, get extracted correctly. However, this being not a string, might be issue, is my diagnosis. How do I get this working? I need to extract longitudes and latitudes. Please give detail explanation as I am new to nifi. Configuration in nifi for EvaluateJsonPathAttributeCustom:

Attribute Name Input : x**.json
Attribute Name Output : latitude
JsonPathExpresssion : $[0].metadata.latitude
Splitif.. : False

score 0 · Answer 1 · answered Sep 03 '21 at 12:13

0

One way to do this is by using the JOLT https://jolt-demo.appspot.com/. I would recommend using the JoltTransformJSON NiFi Processor as it can really help make things easy to pull out only the data that you want. I have tried your specific request and it will work to pull out those data. You can configure JOLT to pull any data you require and it might be easier once you get the hang of it.

    [{
  "operation": "shift",
  "spec": {
    "*": {
      "metadata": {
        "latitude": "latitude",
        "longitude": "longitude"
      }
    }
  }
  }]

answered Sep 03 '21 at 12:13

Mike R

464
5
16

how can I do it using evaluatejsonpath>? – Shubham Patwa Sep 03 '21 at 12:23
If you wanted to use EvaluateJSONPath, you could use $.*.metadata.latitude and $.*.metadata.longitude to get the specific values – Mike R Sep 03 '21 at 12:27
We are back to the question again. If I use evaluateJSONPATH with this '$.*.metadata.latitude' then it leads to failure – Shubham Patwa Sep 03 '21 at 12:32
PLease check the datatype of latitude. Is it related to that? – Shubham Patwa Sep 03 '21 at 12:33
If you use $[0].metadata.latitude, and you have more than 1 entry (1 latitude), it will only return the 1st latitude. If it is valid json, then you could always split each json to go from >1 records in each array to 1 record. That would make it easier to parse – Mike R Sep 03 '21 at 12:35
There is only one entry . I am already splitting it before. – Shubham Patwa Sep 03 '21 at 12:39
It is not returning the latitude, thats the problem. – Shubham Patwa Sep 03 '21 at 12:40
Do you have to use evaluatejsonpath? You could use the JOLT to get all the records you need, perform the same functionality, and do a 1:1 replacement. All you would need to do is pass each individual json element to the transform jolt processor and it would provide what you want for each record. You would just need to add a splitjson for $.* – Mike R Sep 03 '21 at 12:40
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/236730/discussion-between-shubham-patwa-and-mike-r). – Shubham Patwa Sep 03 '21 at 12:42

EvaluateJsonPathAttributeCustom - Nifi

1 Answers1