How to connect Google Sheets from Google Cloud Datalab using Python SDK and service account?

Question

I'm trying to fetch data from a shared Google Sheet using Python SDK from Google Cloud Datalab.

I did the following:

1. Created a new Notebook on Google Cloud Datalab
2. Obtained a service account which is used by the Notebook, as described on the Key concepts & components page: “To display the cloud project and service account names, click the user icon in the top-right corner of the Cloud Datalab notebook or notebook listing page in your browser”.
3. Shared the Google Sheet with the service account email.
4. Run any of the following code snippets

Using Google APIs Python SDK

import google.auth
from googleapiclient.discovery import build

credentials, project = google.auth.default(
    scopes=['https://www.googleapis.com/auth/spreadsheets.readonly'])

service = build('sheets', 'v4', credentials=credentials)
sheet = service.spreadsheets()
result = sheet.values().get(spreadsheetId=GOOGLE_SHEET_ID,
                            range=SHEET_RANGE).execute()
values = result.get('values', [])

Got error:

HttpError: <HttpError 403 when requesting https://sheets.googleapis.com/v4/spreadsheets/<my sheet id here>/values/<my sheet range here>?alt=json returned "Request had insufficient authentication scopes.">

Using gspread

%%bash
pip install gspread

import gspread
import google.auth

credentials, project = google.auth.default(
    scopes=['https://www.googleapis.com/auth/spreadsheets.readonly'])

gc = gspread.authorize(credentials)
sheet = gc.open_by_key(GOOGLE_SHEET_ID)
worksheet = sheet.get_worksheet(MY_WORKSHEET)

Got error:

APIError: {'status': 'PERMISSION_DENIED', 'message': 'Request had insufficient authentication scopes.', 'details': [{'metadata': {'method': 'google.apps.sheets.v4.SpreadsheetsService.GetSpreadsheet', 'service': 'sheets.googleapis.com'}, 'reason': 'ACCESS_TOKEN_SCOPE_INSUFFICIENT', 'domain': 'googleapis.com', '@type': 'type.googleapis.com/google.rpc.ErrorInfo'}], 'code': 403}