2

Does anyone know why polars (or maybe my pycharm setup or python debugger) limits the number of rows in the output? This drives me nuts.

Here is the polars code i am running but I do suspect its not polars specific as there isnt much out there on google (and chatgpt said its info is too old haha).

import polars as pl

df = pl.scan_parquet('/path/to/file.parquet')
result_df =(
        df
        .filter(pl.col("condition_category") == 'unknown')
        .groupby("type")
        .agg(
            [
                pl.col("type").count().alias("counts"),
            ]
        )
    ).collect()
print(result_df)

polars printout of query result

rickhg12hs
  • 10,638
  • 6
  • 24
  • 42
theStud54
  • 705
  • 1
  • 8
  • 19
  • 1
    It's a string column, so hopefully [set_fmt_str_lengths](https://pola-rs.github.io/polars/py-polars/html/reference/api/polars.Config.set_fmt_str_lengths.html#polars.Config.set_fmt_str_lengths) works? – Wayoshi Apr 17 '23 at 13:48
  • Tried and didnt work: ```with pl.Config() as cfg: cfg.set_fmt_str_lengths(200000) print(result_df)``` – theStud54 Apr 17 '23 at 13:52

1 Answers1

2

Looks like the following will work. Thanks to @wayoshi for sharing this. I will say that the defaults are way too conservative!

with pl.Config() as cfg:
    cfg.set_tbl_rows(1000)
    print(result_df)

or throw this at the top of your script if you prefer to not manage contexts.

import polars as pl

# Configure Polars 
cfg = pl.Config()
cfg.set_tbl_rows(2000)
theStud54
  • 705
  • 1
  • 8
  • 19