2

Pandas have read_sql to read a query from database directly in database.

query = "select top 100 * from TABLE"
df=pd.read_sql(query, redshift_conn)

Can I do the same thing using Vaex? Vaex is not having to_sql so, I was converting vaex dataframe into pandas and then using pandas to_sql to dump the data into database. Should I do the same for reading too?

Jithendra Yenugula
  • 51
  • 6
  • Yeah, that is the way to go. Load the data to pandas, then pass it to vaex. If your data is big, do it in chunks, and then convert to memory mappable file that vaex likes (like arrow, hdf5, parquet), and go from there, – Joco Oct 12 '21 at 14:22

0 Answers0