Luigi passing small data between tasks

Question

Using Luigi I want to do a query on my postgres DB to get a lat/lng coordinate pair then pass that pair onto another task.

I have a task QueryPostgres() that takes a SQL string as an input and performs the query. Is there a more concise way of passing the query result onto the next task rather than writing it to a local file? After all it is only two numbers I need to pass.

Thanks!

Answer 1

You could write the result out into another table in the database, redis, etc. Check the luigi.contrib package for a wide variety of output target types. http://luigi.readthedocs.io/en/stable/api/luigi.contrib.html