How to automate html reports generation using Luigi and Papermill

Question

I am trying to combine Luigi and papermill functionality to automate my reports. (courtesy this beautiful article : this

The reports are being generated but I run into two problems:

Issue 1: The output from papermill .ipynb file has some plotly graphs which does not display in the output file. I found the issue is related to the output Jupyter notebook not being trusted. I tired adding the command (below command 1): os.system(f"""jupyter trust {self.output().path}""") but I do not know how I can include it in the below script.

Issue 2: What can I do to generate a report out of the output file in html without code in it e.g. : !jupyter nbconvert --to html --no-input --no-prompt KPIcalculation__2022-10-28.ipynb

The below is the code at I am trying to execute.

class DataLoaderTask(luigi.Task):
    
    last_date_of_report = luigi.Parameter(default=datetime.date.today())
    def requires(self):
        return None
    def run(self):
        nb = pm.execute_notebook(input_path = 'C:/Metrics/WeeklyLoad.ipynb',
                    output_path='C:/Logs/'+'WeeklyLoad_'+str(self.last_date_of_report)+'.ipynb')
    def output(self):        
        return luigi.LocalTarget('C:/Logs/'+'WeeklyLoad_'+str(self.last_date_of_report)+'.ipynb')


class MetricsCalc(luigi.Task):
    
    last_date_of_report = luigi.Parameter(default=datetime.date.today())
    link = luigi.Parameter("C:/Metrics/")
    def requires(self):
        return DataLoaderTask()
    def run(self):
        dp1 = pm.execute_notebook(input_path = 'C:/Metrics/KPIcalculation.ipynb',
                    output_path='C:/Logs/'+'KPIcalculation_'+' '+ str(self.last_date_of_report) + '.ipynb',
                                 parameters=dict(link=self.link)) # Command 
    def output(self):
        return luigi.LocalTarget('C:/Logs/'+'KPIcalculation_'+' '+ str(self.last_date_of_report) + '.ipynb')


if __name__ == '__main__':
    luigi.run(['MetricsCalc','--local-scheduler'])

How to automate html reports generation using Luigi and Papermill

0 Answers0