1

I'm trying to use PDI to read data from an API (json) and now I'm simply trying to use json input to get a few specific fields but the get fields button on the input step gives me.

ERROR (version 8.3.0.0-371, build 8.3.0.0-371 from 2019-06-11 11.09.08 by buildguy) : Index 1 out of bounds for length 1

enter image description here

all the steps execute fine, and produce data - just not the json input step doesn't wnat to give me the fields option! - I've tired the text file and json oput and both write valid json so IDK whats going on....

PS. this is my first time using PDI

ISSUE 2:

It looks like PDI uses jayway for its json path parsing so I've been using this site https://jsonpath.herokuapp.com/ jayway selection which gives me my expected path. When I put that into the 'fields' of the json input dialog I only get the FIRST instance of that path value vs it actually parsing the json and giving me every instance, and can't figure out why though I assume it has something to do with PDI's row based view on things but I also don't know how to get it to understand that its json and it should be giving me back all values that match that path.

enter image description here

UPDATE 1: I've been looking at this https://forums.pentaho.com/threads/135882-Parsing-JSON-data-without-knowing-field-names/ it seems like this Modified Java Script Value step might be the way to go. Will continue testing.

UPDATE 2

OK - Used the MJSV as posted above along with a select fields step and finally able to get the key's

var obj = JSON.parse(mydata);
var keys = Object.keys(obj);
for (var i = 0; i < Object.keys(obj).length; i++) {
    var row = createRowCopy(getOutputRowMeta().size());
    var idx = getInputRowMeta().size();
    row[idx++] = keys[i];
    putRow(row);
}
trans_Status = SKIP_TRANSFORMATION;
Tony
  • 8,681
  • 7
  • 36
  • 55
  • Do you have a sample JSON that you can add to the post? Also, try nested JSON input steps, with the first one getting $.results.* into one string per row, then a second that parses the fields per row. – Cyrus Feb 06 '20 at 15:03
  • 1
    The "Select Fields" button in the JSON Input step is generally going to work from a file, not from a REST source. Reason being is that the previous step isn't going to execute to provide you the document you need for the Select Fields process to function. So my question is, how do you have the "File" tab of the JSON Input step configured? What may work better for you is 1) downloading the file 2) configure the JSON Input step the way you want using the file 3) re-implement the REST source once you have all the correct JSON pathing setup from the file. – eicherjc Feb 06 '20 at 15:41
  • Hi @eicherjc the file tab is confirmed as input from a previous step so nothing file based - I tried writing the output to a file and then going to the json input step but once again it does not produce the results properly. – Tony Feb 06 '20 at 15:57
  • Hey Tony, yeah I would expect that if you're doing input from a previous step for the Select Fields option to not work. However if it's still not working when you're doing the input directly from a file I would question if the JSON is possibly malformed in some way. Is it possible to give us a sample of the JSON? – eicherjc Feb 06 '20 at 19:58

0 Answers0