0

all.

I have a query in a ExecuteSQLRecord processor which produces a JSON column in the result:

SELECT t.id, json_agg(json_build_object("value1", t.column1, "value2", t.column2)) AS "test" FROM table t GROUP BY t.id 

The processor uses a JSONRecordSetWriter to write the results. However, the processor returns a String from the database, which causes issues in further processing that expects a JSON value:

{
   ...
   "test": "[{\"value1\": \"column1value\", \"value2\": \"column2value\"}, ...]"
}

Is there a way of turning it to a proper JSON value, i.e., in this format:

{
   ...
   "test": [{"value1": "column1value", "value2": "column2value"}, ...]
}

without resorting to ReplaceText processors?

I have tried to replace the quotes in the SQL query, to use jsonb_agg() and jsonb_build_object(), and to use a QueryDatabaseTableRecord but none worked.

Thanks in advance.

gcolucci
  • 438
  • 1
  • 5
  • 21
  • 1
    Don't build json on the level of database. Or do it fully on the level of database. Your current problem could be solved with scripting. – daggett Apr 20 '20 at 23:07
  • You might look into JOLT but I think that would require Nifi be interpreting the string as JSON as a precondition. Not sure what database you're using but there might be some regex functions available in SQL as well to format it. MongoDB might be a better more native option for JSON data storage than an RDBMS. – firefly2442 May 03 '20 at 05:35
  • Thanks @daggett, but some of the values themselves are already JSONs... – gcolucci May 04 '20 at 20:10
  • Thanks @firefly2442, I couldn't do it with JOLT due to the reason you mentioned. It's not an option to change the database now, but I appretiate the suggestion... I guess I'll have to stick with the `ReplaceText`s for now. – gcolucci May 04 '20 at 20:11
  • i can show how to script it with groovy – daggett May 04 '20 at 20:52
  • https://stackoverflow.com/a/64404894/1743724 – smbanaei Oct 17 '20 at 16:46

0 Answers0