18

Is it possible to have stdin data go into a pandas DataFrame?

Currently I'm saving the data in an intermediate json file and then doing:

pandas.read_json('my_json_file.json')

but was wondering if it's possible to pipe the stdin directly in the python script. I found this: How to read from stdin or from a file if no data is piped in Python? but not sure how to do a line-by-line insertion in a pandas DF.

Community
  • 1
  • 1
Fra
  • 4,918
  • 7
  • 33
  • 50
  • Person after my own heard. The bash shell provides so much functionality free and these monolithic programs reinvent the wheel (inferiorly). – Sridhar Sarnobat Jul 08 '23 at 20:35

1 Answers1

26

Just use the sys.stdin as a file object (which it actually is) and pass it to pandas read_xy method.

$ cat test.py
import sys
import pandas as pd

df = pd.read_json(sys.stdin)
print df

$ cat data.json
{"a": [1,2,3,4], "b":[3,4,5,6]}

$ python test.py < data.json
   a  b
0  1  3
1  2  4
2  3  5
3  4  6
Viktor Kerkez
  • 45,070
  • 12
  • 104
  • 85