Python - Polars - value counts on string column

Question

How to apply word count on Polars DataFrame I have a string column and I want to make a word count on all text. Thanks

DataFrame example:

0                             Would never order again.
1    I'm not sure it gives me any type of glow and ...
2    Goes on smoothly a bit sticky and color is clo...
3        Preferisco altri prodotti della stessa marca.
4         The moisturizing advertised is non-existent.

If I am using pandas it goes like this

df.Description.str.split(expand=True).stack().value_counts().reset_index()

Results:

           index  0
0             the  2
1             and  2
2           brown  2
3              is  2
4             any  1
5             The  1
6    moisturizing  1
7            like  1
8             I'm  1
9             not  1
10         closer  1
11         stessa  1
12       prodotti  1
13  non-existent.  1
14     advertised  1
15              I  1
16             of  1
17          order  1

...

Can you edit your post to include a sample of the DataFrame you are working with? — Eric Lozano, Feb 07 '22 at 08:19

ritchie46 · Accepted Answer · 2022-02-07T14:16:18.183

You can do something like this:

csv = """
0,                            Would never order again.
1,   I'm not sure it gives me any type of glow and ...
2,   Goes on smoothly a bit sticky and color is clo...
3,       Preferisco altri prodotti della stessa marca.
4,        The moisturizing advertised is non-existent.
""".encode()

(pl.read_csv(csv, has_header=False, new_columns=["idx", "lines"])
    .select(pl.col("lines").str.split(" ").flatten().alias("words"))
    .groupby("words").agg(pl.count())
    .sort("count", reverse=True)
    .filter(pl.col("words").str.lengths() > 0)  
)

Or like this:

(pl.read_csv(csv, has_header=False, new_columns=["idx", "lines"])
    .select(pl.col("lines").str.split(" ").flatten().alias("words"))
    .to_series()
    .value_counts()
    .filter(pl.col("words").str.lengths() > 0)  
)

Both output:

shape: (35, 2)
┌────────┬───────┐
│ words  ┆ count │
│ ---    ┆ ---   │
│ str    ┆ u32   │
╞════════╪═══════╡
│ is     ┆ 2     │
├╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┤
│ and    ┆ 2     │
├╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┤
│ order  ┆ 1     │
├╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┤
│ it     ┆ 1     │
├╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┤
│ ...    ┆ ...   │
├╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┤
│ Goes   ┆ 1     │
├╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┤
│ The    ┆ 1     │
├╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┤
│ stessa ┆ 1     │
├╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌┤
│ altri  ┆ 1     │
└────────┴───────┘

Python - Polars - value counts on string column

1 Answers1

Linked