Pentaho Kettle (PDI) table input step with field substitution running slower than using literal

Question

I'll go straight to the point. I have a table input step, which reads records with a query that includes a where clause, as follows:

SELECT * id, name, surname, creation_date
FROM users
WHERE creation_date > ?

If a put a literal (i.e. '2017-04-02T00:00:00.000Z') in palce of the question mark, this step reads all new values, which could be thousands, in millis. If I use the field substitution and use the incoming value, it takes minutes.

Do you know why this could be happening? Do you know how to solve the issue?

Thank you very much for your time.

What is the source of incoming values? Maybe you have multiple incoming rows? — Rudolf Yurgenson, Apr 10 '17 at 08:28
Hi @RudolfYurgenson, no, the incoming value is a single column single row value, that is correct. — vantesllar, Apr 11 '17 at 11:23

vantesllar · Answer 1 · 2017-04-11T11:33:17.883

0

I found a workaround, not a solution for this particular issue, but it works: instead getting the value from the previous step and use field substitution (? in the query), I read the value in a previous transformation in the job, store it in the variables space, and read it from there using variable substitution ('${variable_name}' in the query), it works just as fast as if the value were hardcoded.

edited Apr 11 '17 at 11:33

answered Apr 11 '17 at 11:24

vantesllar

469
4
13

Pentaho Kettle (PDI) table input step with field substitution running slower than using literal

1 Answers1