I'm Jeffrey Godwyll

This is my personal approach to using pure third-party python packages on Google App Engine.

Installing pure third-party python libs in GAE is very well documented.

With a little tweak you should even have zero issues working in a virtualenv, the modern and right way to work with python web applications. Virtualenvs create an isolated Python environment ensuring that application specific packages are isolated from system packages.

This approach(the one outlined in the docs) is quite easy and better. But occasionally I just want to be able to quickly test some functionality from my python console and be sure too that this will work in my GAE dev environment. Some prefer to use GAE's interactive console. I do too at times, but nothing beats cool terminal colors and stuff :D.

How I Vendor

One common path people follow is to install packages twice. One in their custom lib folder for the development server and another in their virtual environment . But I find it simpler to symlink the virtual environment's site packages to my third-party folder named lib. The concept is simple in that we essentially make our application's dependencies available in this folder and with a little vendoring awesomeness (or hell depending on whichever way you want to look at it) in the configuration of our GAE project, we're able to use these dependencies locally, in dev and in production. For what it's worth, I think it's awesome.

To vendor like I do follow these:

1. Go into your project's root

2. Create virtualenv with a desired name and activate it. I use env.

   $ virtualenv env
   $ source env/bin/activate
   

3. Symlink the virtualenv's site packages to the lib folder

   $ ln -s env/lib/python2.7/site-packages lib
   

   After, you should have a top level structure resembling something like this:

   .
   ├── appengine_config.py
   ├── app.yaml
   ├── env
   ├── lib -> env/lib/python2.7/site-packages
   ├── main.py
   └── requirements.txt
   

4. pip install your packages. They get auto added into the lib folder as well For example to install Flask you can just use:

   $ pip install flask
   

   This works well with a requirement's file:

   $ pip install -r requirements.txt
   

   Even better this allows you to move about with your requirements file that I'm sure you've come to love and just can't live without.

5. Edit or create appengine_config.py to enable vendoring

   from google.appengine.ext import vendor
   
   # Add any libraries installed in the "lib" folder.
   vendor.add('lib')
   

6. In the skip files section of your app.yaml be sure to skip the upload of your virtual environment as shown on the last line in the following example

   skip_files:
   - ^(.*/)?#.*#$
   - ^(.*/)?.*~$
   - ^(.*/)?.*\.py[co]$
   - ^(.*/)?.*/RCS/.*$
   - ^(.*/)?\..*$
   - ^env$ #virtual environment's folder
   

Once the packages are properly set up for App Engine to access, you can just fire up a python shell in your virtualenv to test stuff out. We can thus be pretty certain that any nifty stuff we try in the virtualenv will be the same thing we can put in our application's logic and expect the same results.