Changing style of pandas.DataFrame: Permanently?

Question

When I change the style of a pandas.DataFrame, for instance like so

        # color these columns
        color_columns = ['roi', 'percent_of_ath']
        (portfolio_df
            .style
            # color negative numbers red
            .apply(lambda v: 'color: red' if v < 0 else 'color: black',
                   subset=color_columns)
            # color selected cols light blue
            .apply(lambda s: 'background-color: lightblue',
                    subset=color_columns))

the styles applied to the dataframe are not permanent.

To make them stick I can assign the output of the (portfolio_df ... part to the same dataframe like so:

portfolio_df = (portfolio_df ...

Displaying this overwritten portfolio_df in a Jupyter Notebook, I can see the beautifully styled DataFrame. But trying to change the style from within a function that is imported from a module, I fail. I construct the DataFrame in the function, change the style, return the (now) styled DataFrame from the function, display it in the Jupyter Notebook, I see a non-styled DataFrame.

Edit

Inspecting the type of the return value of the styling operation

s = (portfolio_df.style.apply(...

I see this:

>>> type(s)
pandas.io.formats.style.Styler

So the operation does not return a DataFrame, but a ...Styler object. I was erroneously thinking that I can re-assign this return value to my original DataFrame, thus overwrite it and make the style change permanent.

Question

Is the operation of applying a style to a DataFrame a destructive or non-desctructive operation? The answer seems to be that the style is not changed permanently. Now, how can I make it change permanently?

Edit 2

Viewing the source code of Pandas, I looked at the docstring for class Styler (see [1]):

    If using in the Jupyter notebook, Styler has defined a ``_repr_html_``
    to automatically render itself. Otherwise call Styler.render to get
    the generated HTML.

So in a Jupyter notebook, Styler has a method that auto renders the dataframe, respecting the applied style.

Otherwise (in iPython) it creates HTML.

Assigning the return value of the applied style to a variable

s = (portfolio_df.style.apply(...

I can use it in an Jupyter notebook to render the new style.

What I understand is this: I cannot output my dataframe into a Jupyter notebook and expect it to render the new style. But I can output s to show the new style.

[1] class Styler in

pandas/pandas/io/formats/style.py

Docstring, line 39.

Please edit your post to include the full traceback of the error — G. Anderson, May 16 '19 at 21:17
I have a similar use case, in which I want to apply a style and convert to latex. — layser, Jun 11 '20 at 14:45
Could you elaborate *But trying to change the style from within a function that is imported from a module, I fail.* with your code? — Ynjxsjmh, Jun 03 '22 at 06:41

MYK · Answer 1 · 2022-11-16T16:44:04.623

I can give you two recommendations:

1. Write a simple function to display your dataframes

This is by far the simplest and least hacky solution. You could write:

def my_style(df:pd.DataFrame, color_columns:list[str]=['roi', 'percent_of_ath']):
    return (df
            .style
            .applymap(lambda v: 'color: red' if v < 0 
                                 else None, subset=color_columns)
           )

This lets you write code like:

df.pipe(my_style) # This will output a formatted dataframe

Or

from IPython.display import display 

# This will print a nicely formatted dataframe
def my_display(df:pd.DataFrame, style=my_style):
    display(df.pipe(style))

2. Overwrite the Pandas _repr_html_ method

I don't advice this, but it is what you are asking for ;)

from pandas._config import get_option
from pandas.io.formats import format as fmt

def _my_repr_html_(self) -> str | None:
        """
        Return a html representation for a particular DataFrame.

        Mainly for IPython notebook.
        """
        if self._info_repr():
            buf = StringIO()
            self.info(buf=buf)
            # need to escape the <class>, should be the first line.
            val = buf.getvalue().replace("<", r"&lt;", 1)
            val = val.replace(">", r"&gt;", 1)
            return "<pre>" + val + "</pre>"

        if get_option("display.notebook_repr_html"):
            max_rows = get_option("display.max_rows")
            min_rows = get_option("display.min_rows")
            max_cols = get_option("display.max_columns")
            show_dimensions = get_option("display.show_dimensions")

            formatter = fmt.DataFrameFormatter(
                self,
                columns=None,
                col_space=None,
                na_rep="NaN",
                formatters=None,
                float_format=None,
                sparsify=None,
                justify=None,
                index_names=True,
                header=True,
                index=True,
                bold_rows=True,
                escape=True,
                max_rows=max_rows,
                min_rows=min_rows,
                max_cols=max_cols,
                show_dimensions=show_dimensions,
                decimal=".",
            )
            # return fmt.DataFrameRenderer(formatter).to_html(notebook=True)
            return self.pipe(my_style).to_html(notebook=True) # <<<< !!! HERE !!! 
        else:
            return None
        
df.pipe(_my_repr_html_)

pd.DataFrame._repr_html_ = _my_repr_html_

Be careful! This sample code does not handle very long or wide DataFrames.

Edit:

The code above for overwriting repr_html has a minimal edit of the pandas code. This is a minimal working example:

def my_style(df:pd.DataFrame, color_columns:list[str]=['roi', 'percent_of_ath']):
    return (df.style.applymap(
            lambda v: 'color: red' if v < 0 else None, subset=color_columns)
           ) 

def _my_repr_html_(self) -> str | None:
    return self.pipe(my_style)._repr_html_() # <<<< !!! HERE !!! 
        
pd.DataFrame._repr_html_ = _my_repr_html_

score 0 · Answer 2 · answered Mar 02 '23 at 21:24

I put a startup script for my pandas options in my ipython profile.

For example here is what my ipython startup looks like

$ ls ~/.ipython/profile_default/startup/
10-pandas.py  11-import-modules.py  README

My pandas setup looks like

def setup_pandas_options(max_rows=50):
    import pandas as pd
    pd.set_option('display.expand_frame_repr', True)
    pd.set_option('display.max_rows', max_rows)
    pd.set_option('display.max_columns', 100)
    pd.set_option('display.max_colwidth',180)
    pd.set_option('display.width', 200)

Then my inports startup looks like:

try:
    import os
    import sys
    import numpy as np
    import pandas as pd
    setup_pandas_options()
except ModuleNotFoundError:
    pass

You may need to setup a profile initially.

score -4 · Answer 3 · answered May 16 '19 at 21:19

-4

try using this function

df.style.applymap()

answered May 16 '19 at 21:19

Mathanraj-Sharma

354
1
3
7