2

FPDF is a library that allows to convert a pandas dataframe to nicely formatted pdf reports. Is there a feature in foundry code repo or code workbook to write pdf files into foundry from a spark or pandas dataframe ?

i have a requirement to create a nicely formatted pdf report from a foundry dataset filtered to few rows.

with the help of user https://stackoverflow.com/users/4922673/jackfischer i was able to get the requirement working, However the code overwrites the existing the file, how to incrementally update the datasets with new files everytime the code is ran. I am using Code Workbook templating feature to pass parameter to the logic and everytime a new parameter is passed, how can the logic create new file

example :

  1. samplefile.txt
  2. samplefile2.txt
Asher
  • 175
  • 11

1 Answers1

1

While I'm not familiar with the FPDF library specifically, Foundry supports generating files from datasets in transforms or Code Workbooks.

To create a single Pandas-based PDF from your dataset, convert your dataset to Pandas and get an output file handle from Foundry such as. In Code Workbooks,

def pdf_dataset(input_df):
    output = Transforms.get_output()
    pd = input_df.toPandas()
    output_fs = output.filesystem()
        with output_fs.open(output_file_path, "wb") as f:
            # use FDPF as needed

Alternatively, you can create a PDF per-row in parallel via Spark. This can be done most easily by structuring your data such that the parameters needed to generate each PDF are colocated in rows and from there you can run a Python function on to generate the PDF and write it out of Python memory to the destination dataset.

In a Code Workbook this would resemble

def pdf_dataset(input_df):
    output = Transforms.get_output()

    def generate_pdf(row):
        output_fs = output.filesystem()
        with output_fs.open(output_file_path, "wb") as f:
            # use FDPF as needed
            
    input_df.rdd.foreach(generate_pdf)
jackfischer
  • 86
  • 2
  • 8
  • Thank you for that , it works as you mentioned, i tested the first line of code with sample text file in code workbook. I converted it into a template, but everytime i run the code with a new parameter, the existing file is overwritten, how can i keep adding new files ? – Asher Jan 13 '22 at 04:01
  • 2
    Try a combination of: 1. Make your computation incremental. You may need to use Code Repositories (not Code Workbook) for this. There is a migration functionality to move your code from CWB to CR. In your case it might then suffice to add the `incremental()` decorator to your transform. 2. Add a variable based on the current timestamp at build time and use it in the file name of the PDF you're creating. Then every time your build runs, you get a differentiated file output. – Benjamin Ahnert Jan 13 '22 at 10:06