[Repoze-dev] subprocess app

[Ian Bicking]

I saw Chris mention process management in the #repoze channel, but he'd 
left.  Anyway, I thought you guys might find this to be interesting: 
/home/ianb/src/colorstudy/OLPCServer/olpcserver/spawn.py (not a final 
resting place; that whole package is now in 
http://dev.laptop.org/git?p=library/library-activity;a=tree;f=webserver/olpcwebserver;h=d6277c04e2b94a05408586387a349855ff3f2c72;hb=HEAD 
and I move it into WSGIProxy as well.  The process management isn't 
great, and maybe something like zdrun would be better, but for this 
particular case I also want to keep it very small (in WSGIProxy it could 
be larger).

So basically it's an application that starts a subprocess on demand, and 
can also shut it down when idle.  It could probably be fit together with 
Paste Deploy to run the whole thing like:

   [app:in-subprocess]
   use = egg:XXX#subprocess
   application = other_section (or config:other_file.ini)
   # and possibly:
   virtualenv = /path/to/virtualenv

And it would figure out all the stuff to load up the subprocess, put in 
the right filtering of the environment, and do it with the right 
interpreter if you give a virtualenv.  Right now you have to do that 
stuff somewhat manually.  It would also be great if it could use named 
sockets instead of ports, or even just stdin/stdout or something like 
that (maybe with SCGI?) so that there's less stuff globally allocated 
like port numbers.

I've been putting some of this sort of stuff in WSGIProxy 
(http://svn.pythonpaste.org/Paste/WSGIProxy/trunk), but that package is 
kind of a mess -- I've extracted stuff into it, but not really 
consolidated what's there.

I've been thinking about this sort of thing for a while, but never got 
around to it, but for a server running on the XO/OLPC laptop 
specifically it seemed particularly useful, since it lets the main 
server be very small while occasionally-used applications can be started 
on demand then cleaned up entirely.

-- 
Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org