AppEngine Development Environment Module Restrictions

AppEngine has some very tight but obvious restrictions on what types of Python modules can be invoked from application code. The general rule of thumb is that modules that need filesystem access or C code can’t be used. So, which modules are allowed or disallowed? Which modules are partially implemented, or defined and completely empty (yes, there are/were some)?

Unfortunately, the only official list of such modules is very dated.

There was a point, in the not-too-distant past, that the reigning perception of AppEngine’s module support was that the development environment does no such restriction, leaving a dangerous and scary gap between what will definitely run on your system and what you can be sure will run in production.

It turns out that there is some protection in the development environment.. Maybe even complete protection.

The google/appengine/tools/devappserver2/python/sandbox.py module appears to be wholly responsible for the loading of modules. At the top, there’s a sys.meta_path assignment. This is what appears as of version 1.8.4:

  sys.meta_path = [
      StubModuleImportHook(),
      ModuleOverrideImportHook(_MODULE_OVERRIDE_POLICIES),
      BuiltinImportHook(),
      CModuleImportHook(enabled_library_regexes),
      path_override_hook,
      PyCryptoRandomImportHook,
      PathRestrictingImportHook(enabled_library_regexes)
      ]

This defines a series of module “finders” responsible for resolving imported modules. This is where restrictions are imposed. The following are descriptions/insights about each one.

StubModuleImportHook: Replaces complete modules with different ones.
ModuleOverrideImportHook: Adjust partially white-listed modules (symbols may be added, removed, or updated).
BuiltinImportHook: Imposes a white-list on builtin modules. This raises an ImportError on everything else.
CModuleImportHook: Imposes a white-list on C modules.
path_override_hook: Has an instance of PathOverrideImportHook. It looks like this module looks for modules in special paths (the kind scattered in the.
PyCryptoRandomImportHook: Fixes the loading of Crypto.Random.OSRNG.new .
PathRestrictingImportHook: Makes sure any remaining imports come out of an accessible path.

If you have a question of what specific modules are involved, look in the sandbox.py module mentioned above. The first four finders are relatively concrete. Most of their modules are expressed in lists.

Advertisements