1

I try to export many holoviews plots as png in a loop. My problem is, that after the export of a singular image, the memory is not freed and so, after some iterations, the process kills itself or my pc freezes because the RAM overloads (OOM).

Here is a code that reproduces the problem. I found out, that "df_test.hvplot()" by itself clears the memory like it should, but as son as "hv.render()" is used the memory usage starts to increase without freeing.

In the code there are all my approaches i found online to clean up at the end of the loop, but none of them work.

import numpy as np
import pandas as pd

import holoviews as hv
import hvplot.pandas
from bokeh.io import export_png
import bokeh

import gc


print(f"holoviews version: {hv.__version__}")
print(f"bokeh version: {bokeh.__version__}")
print(f"pandas version: {pd.__version__}")

# Directory (not necessary for testing)
directory_name = 'path_testfolder/'
export_name = 'test'


df_test = pd.DataFrame(np.random.normal(loc=0.5, scale=0.5, size =(80_000_000, 2)), columns=['x', 'y'])
for i in range(100):
    # Create Plot
    p1 = df_test.hvplot(kind="scatter", x="x", y="y", rasterize=True, datashade=True, width=800, height=800)
    p1_bokeh = hv.render(p1)
    
    # Export
    # file_out_path = directory_name + "/" + export_name + "_" + str(i) + ".png"
    # a = export_png(p1_bokeh, filename=file_out_path, height=800, width=800)
    
    # Clear Memory (Apporaches)
    p1_bokeh.renderers = []
    p1_bokeh = None
    p1 = None
    del p1_bokeh
    del p1
    # del a
    
    bokeh.io.curdoc().clear()  
    bokeh.io.state.State().reset()
    bokeh.io.reset_output()
    
    gc.collect()
mosc9575
  • 5,618
  • 2
  • 9
  • 32
Debroize
  • 29
  • 2
  • I'd suggest seeing if you can reproduce a situation with pure Bokeh, and then file and issue on the appropriate tracker (if it's only when `hvplot` is used then it should be filed on he `hvplot` issue tracker). – bigreddot Nov 15 '22 at 19:43

0 Answers0