2

In NiFi I'm processing a flowfile containing the following attribute:

Key: 'my_array'
    Value: '[u'firstElement', u'secondElement']'

I'd like to split flowFile on this array to process each element separately (and then merge). I tried to use SplitJson processor, but it requires JSON content to operate on, so I used AttributesToJSON before it. Unfortunately the produced flowFile's content is:

{"my_array": "[u'firstElement', u'secondElement'"}

And I receive the error

The evaluated value [u'firstElement', u'secondElement'] of $['my_array'] was not a JSON Array compatible type and cannot be split.

Is it possible to convert my_array string to the correct JSON array? Do I need to use ExecuteScript or is there some simpler way?

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
trivelt
  • 1,913
  • 3
  • 22
  • 44
  • the `[u'firstElement', u'secondElement']` has not a valid json format. i think it's a kind of python formatting. – daggett Jun 26 '19 at 08:51
  • Yes, these values come from Python, but I think that the main problem is having `"[u'firstElement', u'secondElement']"` instead of `[u'firstElement', u'secondElement']` – trivelt Jun 26 '19 at 08:53
  • 1
    with doublequotes - it's just a string. without - not a valid json. could you produce json from python? – daggett Jun 26 '19 at 09:21
  • I tried to convert it to the list in Python, but it causes error `org.python.core.PyList cannot be cast to java.lang.String `, so it seems that Python script has to return string. – trivelt Jun 26 '19 at 09:51
  • 1
    use `json.dumps(...)` to convert python data to json-formatted string – daggett Jun 26 '19 at 10:06

2 Answers2

3

How about ReplaceText with Replacement Strategy of Always Replace and Replacement Value of ${my_array} and then SplitJSON? This will replace your FlowFile's content with this attribute's value and then you could SplitJSON on it.

Ben Yaakobi
  • 1,620
  • 8
  • 22
  • Thank you, I used your solution along with the one proposed by @dagett. I changed Python script to set `json.dumps(...)` as attribute's value and than I put `ReplaceText` with`Replacement Value = {"my_array": ${my_array}}` – trivelt Jun 26 '19 at 11:33
0

Suppose I want to string : "Hashtags": "['tag1','tag2']" (as part of my resultant json in Nifi,) to be changed into : "Hashtags": ['tag1','tag2'].

what I do is :

Apache Nifi replaceText sample

I use ReplaceText with Replacement Strategy : Regex Replace and Replacement Value : a regex Expression. This will replace FlowFile's matched content with this attribute's value and then you could continue your process.

smbanaei
  • 1,123
  • 8
  • 14
  • That does indeed work, thanks for the tip. It sucks that NiFi doesn't properly format JSON lists in AttributesToJson processor, instead it turns them into strings. – Wyllys Dec 08 '20 at 23:08