Lowering Tg2 memory usage by running multiple instances of an app inside same WSGI daemon process

Friday, August 6th, 2010 | Software Development

I was recently trying to deploy one app multiple times inside the same
WSGIProcessGroup and WSGIApplicationGroup %{GLOBAL} to reduce memory
usage.
This works quite well except for all the SQLAlchemy sessions which end being
attached to the engine of the last wsgi script started.

The best solution that I have been able to get so far is to create a proxy interface to the application engine. This way each wsgi script gets binded to the same engine, but the engine itself keeps track of all the available real engines and responds to the script requests sending them to the right real engine.

class MultiSiteEngine(object):
   def __init__(self):
       self.engines = {}
 
   def __getattr__(self, name):
       if name == 'engines':
           return object.__getattribute__(self, name)
 
       if not self.engines.has_key(config['sqlalchemy.url']):
           self.engines[config['sqlalchemy.url']] = engine_from_config(config, 'sqlalchemy.')
 
       return getattr(self.engines[config['sqlalchemy.url']], name)

To make this work just allocate a multi_engine = MultiSiteEngine() inside your tg2 app model __init__ and change init_model method to pass multi_engine instead of the engine itself.

Tags:

blog comments powered by Disqus

Search