1

I'm using a Microsoft REST API to query a Azure application, oauth and request goes without problem.

The response from InvokeHTTP has this format

{"@odata.context":"https://****.dynamics.com/api/data/v9.1/$metadata#endpoint","value":[ here comes the actual JSON result in format {
  "@odata_etag" : "W/\"555598\"", "field":"value...},...]
,"@odata.nextLink":"https://****.dynamics.com/api/data/v9.1/endpoint?$skiptoken.....}

I need to extract the nextLink for pagination and Value to continue the flow and store the result.

When I try to parse with inferAvroSchema so I can start working with, it throws this error:

Illegal initial character: @odata.etag

My Idea was to inferAvroSchema, then EvaluateJsonPath to extract the odata tags and then extract the values.

I tried using EvaluateJsonPath on the result asking to create an attribute for $.@odata.context but it doesn't find the item either, I'm sure is something about the @.

I can also replace all the @ of the incoming flow for another char, but don't know if that makes sense.

I'm feeling that I'm not using a correct approach, but NIFI + odata doesn't give me results on google or here. I'm open to any suggestions!

thank you!

Sandra Rossi
  • 11,934
  • 5
  • 22
  • 48
Alejandro
  • 519
  • 1
  • 6
  • 32

2 Answers2

2

Schema fields cannot contain @. You could replace the @, however you must be sure not to replace it in actual content like email addresses. Another solution is to transform the API response using JoltTransformJSON processor, such that your flow can work with it:

enter image description here

GenerateFlowFile:

enter image description here

For the JoltTransformJSON processor provide following Jolt specification:

[
  {
    "operation": "shift",
    "spec": {
      "\\@odata.nextLink": "next"
    }
  }
]

Leave the default values for the other properties. You can play around with Jolt here: http://jolt-demo.appspot.com/

EvaluateJsonPath:

enter image description here

Result:

enter image description here

Notice that the url is now part of the flowfile attributes.

DarkLeafyGreen
  • 69,338
  • 131
  • 383
  • 601
1

Your hunch is correct, you can only have valid characters for the field names on the schema type you are using, avro or JSON. You could get NiFi to remove illegal characters with the replacetext proceasor, have a read here on what is valid: http://avro.apache.org/docs/current/spec.html#names

Chaffelson
  • 1,249
  • 9
  • 20
  • Than you, I accepted @Upvote response because it went a little forward explaining how to extract the fields – Alejandro Nov 11 '19 at 17:58