1

I'm trying to retrieve some data and mail it out from within a Google Cloud Function using SendGrid

I tried convert the data as a list of dictionaries (with the same, flat, structure) to CSV as detailed here but this fails as the filesystem is read only.

To counter this, I use io.StringIO() to store the CSV in-memory.

However, I get the following error/stack trace during execution:

Traceback (most recent call last): 
File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", 
 line 383, in run_background_function_function_handler.invoke_user_function(event_object) 
File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", 
 line 217, in invoke_user_function return call_user_function(request_or_event) 
File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", 
 line 214, in call_user_function event_context.Context(**request_or_event.context)) 
File "/user_code/main.py",  line 267, in session_updated summarize_session(sessionid, doctorid)  File "/user_code/main.py", line 375, in summarize_session send_email([docemail], data) 
File "/user_code/main.py", line 328, in send_email response = sg.send(message) 
File "/env/local/lib/python3.7/site-packages/sendgrid/sendgrid.py",  
 line 98, in send response = self.client.mail.send.post(request_body=message.get()) 
File "/env/local/lib/python3.7/site-packages/python_http_client/client.py", 
 line 251, in http_request data = json.dumps(request_body).encode('utf-8')  
File "/opt/python3.7/lib/python3.7/json/__init__.py", 
 line 231,  in dumps return _default_encoder.encode(obj) 
File "/opt/python3.7/lib/python3.7/json/encoder.py",  
 line 199, in encode chunks = self.iterencode(o, _one_shot=True)  
File "/opt/python3.7/lib/python3.7/json/encoder.py", 
 line 257, in iterencode return _iterencode(o, 0)  
File "/opt/python3.7/lib/python3.7/json/encoder.py", 
 line 179,  in default raise TypeError(f'Object of type {o.__class__.__name__} '  
TypeError: Object of type StringIO is not JSON serializable

The code is as follows:

def send_email(to_emails, datadict):
    message = Mail(
    from_email='info@domain.com',
    #to_emails= to_emails,
    to_emails= ['dude@domain.com'],
    subject='Summary of your session',
    html_content='<strong>Data Summary</strong>\
        <p>This email is a summary of your session. Please check the attachment for  details.    </p>')
    sg = SendGridAPIClient(os.environ.get('SENDGRID_API_KEY'))
    keys = datadict[0].keys()
    try:
        output_file = io.StringIO()
        #output_file = io.BytesIO()  # results in a typeerror 
        # https://stackoverflow.com/questions/34283178/typeerror-a-bytes-like-object-is-required-not-str-in-python-and-csv
        dict_writer = csv.DictWriter(output_file, keys)
        dict_writer.writeheader()
        dict_writer.writerows(datadict)

        print("Attaching")
        message.attachment = [
        Attachment(FileContent(output_file),
                FileType('text/csv'),
                FileName('sessiondata.csv'),
                Disposition('inline'),
                ContentId('SessionData')),
        ]
    except Exception as e:
        print("Exception:")
        print(e)
        return

    response = sg.send(message)

How do I convert the list of dictionaries to a CSV and attach it to an email without opening a physical file on the filesystem?

kilokahn
  • 1,136
  • 2
  • 18
  • 38
  • What do you understand from that error message? _Is it possible_ That depends on the format of the dictionary, eh. – AMC Jan 25 '20 at 22:43
  • The dict is a flat one, if that is what you're asking. The erroir message tells me that once the CSV is written, attachment fails because its in a format that is not JSON serializable. I tried using BytesIO but get a different error instead : https://stackoverflow.com/questions/34283178/typeerror-a-bytes-like-object-is-required-not-str-in-python-and-csv – kilokahn Jan 25 '20 at 22:46
  • I’m pretty sure you read the StringIO like you would a file, just remember to seek() back to the beginning before doing so. – AMC Jan 25 '20 at 22:48
  • I don't think seeking has anything to do with causing the Exception. From the Stacktrace, the attachment is seen to be passed to json.dumps by the SendGrid library, and found to be in a format that is not JSON serialisable. – kilokahn Jan 25 '20 at 22:57
  • Sorry if my comment wasn’t clear, I meant that if you do seek and read, you’ll get the contents as a string, which should be JSON serializable. – AMC Jan 26 '20 at 03:55

0 Answers0