AWS Lambda and Pygsheets

Question

This is slightly related to this previous question .

Similar to that time, I am trying to use pygsheets on AWS Lambda and I'm having trouble with the json file. Like what he discussed, the json file is included in the zip.

What I'm having trouble with now is a write error. When I invoke the function I get the following error.

{
    "errorMessage": "[Errno 30] Read-only file system: 'sheets.googleapis.com-python.json'",
    "errorType": "OSError",
    "stackTrace": [
        "  File \"/var/lang/lib/python3.8/imp.py\", line 234, in load_module\n    return load_source(name, filename, file)\n",
        "  File \"/var/lang/lib/python3.8/imp.py\", line 171, in load_source\n    module = _load(spec)\n",
        "  File \"<frozen importlib._bootstrap>\", line 702, in _load\n",
        "  File \"<frozen importlib._bootstrap>\", line 671, in _load_unlocked\n",
        "  File \"<frozen importlib._bootstrap_external>\", line 783, in exec_module\n",
        "  File \"<frozen importlib._bootstrap>\", line 219, in _call_with_frames_removed\n",
        "  File \"/var/task/handler.py\", line 151, in <module>\n    gc = py.authorize(client_secret='google-sheets-credentials.json')\n",
        "  File \"/var/task/pygsheets/authorization.py\", line 129, in authorize\n    credentials = _get_user_authentication_credentials(client_secret, scopes, credentials_directory, local)\n",
        "  File \"/var/task/pygsheets/authorization.py\", line 64, in _get_user_authentication_credentials\n    with open(credentials_path, 'w') as file:\n"
    ]
}

The user who had trouble with the json file seemed to have gotten things working and yet I've read that lambda functions are immutable. Can anyone clear this up for me? If possible, how can I get past this issue?

Edit:

This is the portion of code that is relevant to the question. As I stated, the json file is included in the zip that I uploaded to AWS Lambda. The other question on stack overflow that I linked to got things working so my guess is I need to put one or both of the two json files in a different directory, but I'm not sure as there are very little details in their answer.

import pygsheets as py

gc = py.authorize(client_secret='google-sheets-credentials.json')

Pretty much all of the code relating to the error was included in the error. Just in case I included the two lines that are needed to create the error — cpuser, Feb 10 '21 at 04:51

score 0 · Answer 1 · answered Feb 10 '21 at 05:03

0

After Auth flow pygsheets will try to create a credential and save it for future calls. But you can skip this by providing this json file yourself.

First run pygsheets locally, it will generate the credentials json. Upload it to lambda and pass its path as credentials_directory.

answered Feb 10 '21 at 05:03

Nithin

5,470
37
44

I see. I had the file uploaded, but I wasn't using gc = py.authorize(credentials_directory=path). I tried using the path '/var/task' but that doesn't seem to be the location that the function is located. – cpuser Feb 10 '21 at 06:45
I confirmed that '/var/task/' is the correct path. If I call gc = py.authorize(credentials_directory='/var/task/') I get the same error message. "errorMessage": "[Errno 30] Read-only file system: '/var/task/sheets.googleapis.com-python.json'", – cpuser Feb 10 '21 at 08:20

Tim Roberts · Answer 2 · 2023-01-19T18:08:42.747

The issue is caused by Pygsheets trying to write the credentials to the root directory (/var/task/ in your Lambda) when it authorises.

Lambda's are read only apart from the /tmp/ folder (hence the error message when it tries to write to the root directory). You therefore need to set the credentials_directory to /tmp/ so that Pygsheets knows to write the credentials to this folder.

You'll also want to copy your existing credentials into that same /tmp/ folder when you deploy the Lambda. You could do that in your Dockerfile, but I plumped for a slightly hacky solution of copying over the tokens in Python before authorising Pygsheets. I have a bunch of other access tokens so put them all in /tmp/tokens/ for tidiness.

token_folderpath = <LOCAL_TOKEN_FOLDERPATH>
if <SOME_CHECK_THAT_YOU_ARE_RUNNING_IN_LAMBDA>:
    if not os.path.exists("/tmp/tokens/"):
        shutil.copytree(token_folderpath, "/tmp/tokens/")
    token_folderpath = "/tmp/tokens/"

google_sheets = pygsheets.authorize(
    client_secret=os.path.join(
        token_folderpath, "sheets.googleapis.com-python.json"
    ),
    credentials_directory=token_folderpath
)

AWS Lambda and Pygsheets

2 Answers2