Google App Engine Middleware (gaem) is a collection of utilities for
hosting external sites on Google App Engine.
I created gaem because I wanted to host my blog on an unreliable server and let
Google App Engine serve the content while my server was down or in maintenance
mode.
The basic design is similar to that of Django's middleware. A request
travels through a list of components which can either manipulate the request or
provide a response. The response is filtered back through the components and
served to the requester.
A basic implementation looks something like
import gaem
import gaem.middleware
import gaem.middleware.cache
import gaem.middleware.dos
import gaem.middleware.fetch
URL = 'http://www.example.org'
MIDDLEWARE_CLASSES = (
gaem.middleware.cache.Memcache(),
gaem.middleware.cache.Datastore(),
gaem.middleware.dos.DoS(),
gaem.middleware.fetch.Fetch(url=URL),
)
application = gaem.Application(MIDDLEWARE_CLASSES)
if __name__ == '__main__':
application.run()
When a request first hits this application it is handled by the Memcache
component. This basic caching component checks the Google App Engine Memcache
service to see if the page is in the cache. If so it simply returns the cached
response, otherwise the request proceeds to the Datastore component which does
pretty much the same thing except it uses Google Datastore for persistence. If
both caching components fail the the request goes through a DoS protection
component and is finally retrieved from the backend server by the Fetch
component.
Using a setup like this gaem acts as a rudimentary reverse proxy and while most
of the functionality I've implemented thus far is for this type setup, I plan
on adding features not traditionally found in a reverse proxy.
Current Features
Possible Future Features
- Initial and periodic crawler (pre-cache and other hooks)
- RESTful API for managing content (put, delete, etc...)
- Web interface
- XSS protection
- Some ESI support
But what makes gaem really cool for developers is how easy it is to create
custom components.
A basic component looks like
from gaem import middleware
class DoesNothing(middleware.Base):
def process_request(self, request):
return request
def process_response(self, response, request):
return response
A default gaem component should inherit from gaem.middleware.Base and
implement process_request and/or process_response.
The process_request function lets you either manipulate the request (ex:
strip the "cookie" header) and pass it along or use the request to create and
return a response (ex: do a fetch to the backend server). The
process_response function lets you filter the response (ex: translate body
into pig latin) and do side affects (ex: store response in memcache) before the
response is sent to the requester.
The Request/Response objects are borrowed from WebOb and can be
manipulated using the interfaces defined in the WebOb documentation.
gaem is currently pretty rough and likely to change over the next couple of
months, but if you're a Python hacker I'd love to hear some feedback
(contact me or submit an issue).
P.S. You can see the application I created for this site in the examples directory.