ImportError: Importing the devappserver sandbox module from user application code is not permitted

Question

I'm working on a Google App Engine application written in Python (Standard Environment) and I need to whitelist some additional modules in the development server environment.

I have been doing it using this code in the appengine_config file for a long time, and it worked very well:

from google.appengine.tools.devappserver2.python import sandbox
sandbox._WHITE_LIST_C_MODULES += ['_ssl', '_socket']

As answered in this question, some time ago the Google Cloud SDK was updated and the previous import caused an ImportError. The import just needed to be changed to:

from google.appengine.tools.devappserver2.python.runtime import sandbox

With this change, everything worked well again, until I updated the Cloud SDK to version: 186.0.0.

Now it seems a "SandboxAccessPreventionImportHook" class has been added to the sandbox module so it can't be imported from the App Engine application. This is the error the application is raising:

ImportError: Importing the devappserver sandbox module (google.appengine.tools.devappserver2.python.runtime.sandbox) from user application code is not permitted.

Does anyone have an idea on how to bypass this? Or is there another way to whitelist modules in the development server environment?

Thanks!!!

What problem are you trying to solve that requires adding module to the whitelist? — Dave W. Smith, Jan 31 '18 at 19:48
@DaveW.Smith most likely outbound https requests, this just started happening to me too — Alex, Jan 31 '18 at 21:12
If I dont have that `_WHITE_LIST_C_MODULES` code in place i experience this issue https://github.com/requests/requests/issues/4078. I'm currently trying to see if I can have my devserver load request 2.16 (the version before started happening) instead of up-to-date version my production code uses — Alex, Jan 31 '18 at 21:47

karloskar · Answer 1 · 2018-02-02T10:06:01.230

2

We had the exact same issue as OP but unfortunately Alex's solution didn't work for us.

While a really really hacky solution this is what worked for us. Please be aware that the modification of the sandbox must be redone after an update.

Modify the sandbox.py file located at {appengine_sdk_dir}/google/appengine/tools/devappserver2/python/runtime/sandbox.py and add _ssl and _socket like in Alex's first example to the _WHITE_LIST_C_MODULES list.

_WHITE_LIST_C_MODULES = [
    // keep existing items
    '_ssl',
    '_socket'
]

Then we removed the imports and sandbox overrides from appengine_config.py.

vendor.add('lib')

if os.environ.get('SERVER_SOFTWARE', '').startswith('Development'):
    import imp
    import os.path
    import inspect

    # Use the system socket.
    real_os_src_path = os.path.realpath(inspect.getsourcefile(os))
    psocket = os.path.join(os.path.dirname(real_os_src_path), 'socket.py')
    imp.load_source('socket', psocket)

# handle requests_toolbelt's monkeypatch as you see fit.

Let's hope there is a day when this is no longer necessary!

edited Feb 02 '18 at 10:06

answered Feb 02 '18 at 06:32

karloskar

2,091
1
15
10

Thanks for sharing this and confirmed: this let's us locally use the Boto3 library to talk to the AWS API. Pretty average that Google does not feel like fixing this obvious issue in their GAE SDK. – Soeren Feb 27 '18 at 03:47
Somehow, this one worked :) Although, I still don't understand the mechanics of it (-_-') – Divij Sehgal Apr 01 '18 at 10:10
1

This worked me for but led to the following error: ``` File ".../lib/pathlib2/__init__.py", line 5, in import ctypes File "/usr/local/Cellar/python@2/2.7.15/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ctypes/__init__.py", line 29, in if int(_os.uname()[2].split('.')[0]) < 8: ValueError: invalid literal for int() with base 10: '' ``` ..caused by value `('Linux', '', '', '', '')` being returned by `fake_uname` and was fixed by removing the override for uname/whitelisted in _MODULE_OVERRIDE_POLICIES in sandbox.py. – Johan Andersson Dec 03 '19 at 12:48

Alex · Answer 2 · 2018-01-31T23:44:12.103

Context

So for me the root of this issue was the dev_appserver.py's inability to send outbound https requests / sockets.

The solution then was to put this in my appengine_config.py:

vendor.add('lib')

if os.environ.get('SERVER_SOFTWARE', '').startswith('Development'):
    import imp
    import os.path
    import inspect
    try:
        from google.appengine.tools.devappserver2.python import sandbox
    except ImportError:
        from google.appengine.tools.devappserver2.python.runtime import sandbox

    sandbox._WHITE_LIST_C_MODULES += ['_ssl', '_socket']
    # Use the system socket.

    real_os_src_path = os.path.realpath(inspect.getsourcefile(os))
    psocket = os.path.join(os.path.dirname(real_os_src_path), 'socket.py')
    imp.load_source('socket', psocket)
else:
    # Doing this on dev_appserver/localhost seems to cause outbound https requests to fail
    import requests
    from requests_toolbelt.adapters import appengine as requests_toolbelt_appengine

    # Use the App Engine Requests adapter. This makes sure that Requests uses
    # URLFetch.
    requests_toolbelt_appengine.monkeypatch()

Today I upgraded my cloud sdk and started getting

ImportError: Importing the devappserver sandbox module (google.appengine.tools.devappserver2.python.runtime.sandbox) from user application code is not permitted.

If I remove the _WHITE_LIST_C_MODULES stuff, I then get this error when using the python requests library to make outbound https requests:

  File "/Users/alexindaco/google-cloud-sdk/platform/google_appengine/google/appengine/api/urlfetch.py", line 293, in fetch
    return rpc.get_result()
  File "/Users/alexindaco/google-cloud-sdk/platform/google_appengine/google/appengine/api/apiproxy_stub_map.py", line 613, in get_result
    return self.__get_result_hook(self)
  File "/Users/alexindaco/google-cloud-sdk/platform/google_appengine/google/appengine/api/urlfetch.py", line 413, in _get_fetch_result
    rpc.check_success()
  File "/Users/alexindaco/google-cloud-sdk/platform/google_appengine/google/appengine/api/apiproxy_stub_map.py", line 579, in check_success
    self.__rpc.CheckSuccess()
  File "/Users/alexindaco/google-cloud-sdk/platform/google_appengine/google/appengine/api/apiproxy_rpc.py", line 157, in _WaitImpl
    self.request, self.response)
  File "/Users/alexindaco/google-cloud-sdk/platform/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 222, in MakeSyncCall
    self._MakeRealSyncCall(service, call, request, response)
  File "/Users/alexindaco/google-cloud-sdk/platform/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 241, in _MakeRealSyncCall
    request_pb.set_request(request.Encode())
  File "/Users/alexindaco/google-cloud-sdk/platform/google_appengine/google/net/proto/ProtocolBuffer.py", line 103, in Encode
    self.Output(e)
  File "/Users/alexindaco/google-cloud-sdk/platform/google_appengine/google/net/proto/ProtocolBuffer.py", line 347, in Output
    self.OutputUnchecked(e)
  File "/Users/alexindaco/google-cloud-sdk/platform/google_appengine/google/appengine/api/urlfetch_service_pb.py", line 481, in OutputUnchecked
    out.putDouble(self.deadline_)
  File "/Users/alexindaco/google-cloud-sdk/platform/google_appengine/google/net/proto/ProtocolBuffer.py", line 592, in putDouble
    a.fromstring(struct.pack("<d", v))
error: required argument is not a float

I then found that stack trace in this issue https://github.com/requests/requests/issues/4078 which seemed to indicate that this only started happening after python-requests version 2.16.0

Solution

All my third party libs were installed to a folder in my project root called lib using

pip install -t lib

Now, I have lib & localhost_libs and I did:

pip install -t localhost_libs requests==2.16

My appengine_config.py now has this instead:

vendor.add('lib')

if os.environ.get('SERVER_SOFTWARE', '').startswith('Development'):
    vendor.add('localhost_libs')
    import pkg_resources
    pkg_resources.require("requests==2.16.0")
import requests
print "requests.__version__", requests.__version__

from requests_toolbelt.adapters import appengine as requests_toolbelt_appengine

# Use the App Engine Requests adapter. This makes sure that Requests uses
# URLFetch.
requests_toolbelt_appengine.monkeypatch()
print "Appengine config done"

Edit: Modified solution to use pkg_resources and not require a prod_libs folder

score 0 · Answer 3 · edited Mar 06 '18 at 14:11

0

The location of the sandbox module got moved into the runtime module.

from google.appengine.tools.devappserver2.python import runtime
runtime.sandbox._WHITE_LIST_C_MODULES += ['_ssl', '_socket']

edited Mar 06 '18 at 14:11

Christian

3,551
1
28
24

answered Mar 06 '18 at 08:56

Паша Гагин

9
1

3

Please add some explanation. Code only answers are often flagged as low-quality posts. – cezar Mar 06 '18 at 11:39

Axa Cheng · Answer 4 · 2018-03-10T12:36:47.833

The way @karloskar mentioned was fixed my case.

However, after I modified sandbox.py to allow _ssl and _socket to be WHITE_LIST, I got another library import issue - ImportError: No module named google.auth.

For who also got above google.auth import error, you could consider to update your appengine_config.py like below. That's the way fixed my import issue.

# appengine_config.py

import os
import google
from google.appengine.ext import vendor

lib_directory = os.path.dirname(__file__) + '/lib'

# Change where to find the google package (point to the lib/ directory)
google.__path__ = [os.path.join(lib_directory, 'google')] + google.__path__

# Add any libraries install in the "lib" folder.
vendor.add(lib_directory)

Reference1: App Engine not finding google.auth file in dev_appserver.py

Reference2: https://github.com/GoogleCloudPlatform/google-auth-library-python/issues/169#issuecomment-315417916

While a link is a welcome reference, you should state the main points of your answer here on SO. Links often break after a while. — Mr. T, Mar 10 '18 at 09:30

ImportError: Importing the devappserver sandbox module from user application code is not permitted

4 Answers4

Context

Solution