Google App Engine: importing modules within the handler's scope vs global scope

Question

My project consists of a lot of imports which are placed at 'global' scope.

from google.appengine.ext import ndb
from handlers import SomeHandler
import logging
import datetime  # will only use this module ONCE

I want to use the datetime module just once inside a specific handler. Should I import datetime within the handler or should I leave it in global scope?

import datetime  # Should we import here?

class BookHandler(webapp2.RequestHandler):
    import datetime   # Or, should we import here?
    def get(self):
        today = datetime.datetime.now()

It seems like importing locally shows clearer dependency relationships. Is there any performance issue or other drawbacks to consider?

score 1 · Answer 1 · answered Oct 12 '17 at 15:57

you should defiantly import modules at the beginning of your file which will bound it to the scope of that file. I think what you are doing is called 'lazy loading' the modules and it can causes bugs at runtime, if the module is not installed or imported correctly.

By the way, the way python works is that every time you import a module the interpreter looks if the module has already been imported. If it has been imported then it will set a reference to it. In another word, it doesn't create a new instance of it.

What i recommend is to create a file for your handler class and import datetime and anything else you want at the beginning of that file.

score 1 · Answer 2 · answered Oct 12 '17 at 16:56

There is no problem importing inside the handler (or even inside the get() function if you prefer) - I'm using it massively.

Advantages of lazy-loading:

reduced app loading time
the average memory footprint of your application can be much lower than the total memory footprint required should you load all modules at init time, even those rarely used - lower cost

Disadvantages of lazy-loading:

non-deterministic app loading time
potential hard-to-reproduce bugs hit at delayed module loading since the exact conditions are unknown

Related (in the lazy-loading sense only): App Engine: Few big scripts or many small ones?

score 1 · Answer 3 · edited Jun 20 '20 at 09:12

Hiding imports like this is an optimisation; whenever considering whether to optimise, verify that the proposed optimisation is really going to be effective.

Let's consider the specific example of datetime first. Here's a simple app:

import sys

import webapp2


class Handler(webapp2.RequestHandler):

    def get(self):
        if 'datetime' in sys.modules:
            self.response.write('The datetime module is already loaded.\n')
        else:
            self.response.write('The datetime module is not already loaded\n.')
        self.response.write('%d modules have been loaded\n' % len(sys.modules))
        count = sum(1 for x in sys.modules.values() if '/lib64' in repr(x))
        self.response.write('%d standard library modules have been loaded\n' % count)
        gcount = sum(1 for x in sys.modules.values() if 'appengine' in repr(x))
        self.response.write('%d appengine modules have been loaded\n' % gcount)

application = webapp2.WSGIApplication([('/', Handler)])

If we visit the '/' url we see this output:

The datetime module is already loaded.

706 modules have been loaded

95 standard library modules have been loaded

207 appengine modules have been loaded

Even in this minimal app, datetime has already been imported by the SDK^*. Once Python has imported a module, futher imports only cost a single dictionary lookup, so there is no benefit in hiding the import. Given that the SDK has already imported 95 standard library modules and 207 SDK modules it follows that there is unlikely to be much benefit in hiding imports of commonly used standard library or SDK modules.

This leaves the question of imports of application code. Handlers can be lazy-loaded by declaring them as strings in routes, so that they are not imported until the route is visited:

app = webapp2.Application([('/foo', 'handlers.FooHandler')])

This technique permits optimising startup time without hiding imports in classes or methods, should you find that it is necessary.

The cost of lazy-loading, as the other answers point out, can be unexpected runtime errors. Moreover, if you choose of hide imports it can also decrease code readability, potentially cause structural problems, for example masking circular dependencies, and sets a poor example for less experienced developers, who may assume that hiding imports is idiomatic rather than an optimisation.

So, when considering optimising in this way:

verify that optimisation is necessary: not all applications require absolute maximum performance
verify that the right problem is being addressed; RPC calls on App Engine tend to dominate response times
profile to verify that the optimisation is effective
consider the costs in code maintainability

^*Relying on the SDK's sys.modules being similar to that of the cloud runtime is, I hope, a reasonable assumption.

Google App Engine: importing modules within the handler's scope vs global scope

3 Answers3