[Repoze-dev] Plone on repoze + configuration details

[Chris McDonough]

On Sep 23, 2007, at 2:19 PM, Mike Naberezny wrote:

> Chris McDonough wrote:
>> This is Apache + mod_wsgi + repoze.zope2 (unreleased) + Zope  
>> 2.10.4/ZEO + Plone 3.0.1.
>
> That is exciting!

It is. ;-)  Thank you!

>
> How stable is mod_wsgi now with its child processes?

This is my first experience with it, so I probably don't have enough  
data to say.  It's configured to run in "daemon mode" currently,  
which is documented to be the most stable (but least performant)  
configuration.  The docs for this mode say:

"""
An alternate mode of operation is 'daemon' mode. This mode operates  
in similar ways to FASTCGI/SCGI solutions, whereby distinct processes  
can be dedicated to run a WSGI application. Unlike FASTCGI/SCGI  
solutions however, a separate infrastructure is not needed when  
implementing the WSGI application and everything is handled  
automatically by mod_wsgi.

Because the WSGI applications in daemon mode are being run in their  
own processes, the impact on the normal Apache child processes used  
to serve up static files and host applications using Apache modules  
for PHP, Perl or some other language is minimal. Daemon processes may  
if required also be run as a distinct user ensuring that WSGI  
applications cannot interfere with each other or access information  
they shouldn't be able to.

When using 'daemon' mode of mod_wsgi, each process group can be  
individually configured so as to run in a manner similar to either  
'prefork', 'worker' or 'winnt' MPMs for Apache. This is achieved by  
controlling the number of processes and threads within each process  
using the 'processes' and 'threads' options of the WSGIDaemonProcess  
directive.
"""

Comparative performance info between modes is here:  http:// 
code.google.com/p/modwsgi/wiki/PerformanceEstimates

I'd have thought I'd see "python" processes representing the WSGI  
loader app in the process table, but I don't.  They seem to appear in  
the process table as "apache2":

PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
    45 root       5 -10     0    0    0 S  0.0  0.0   0:00.00 aio/0
6448 root      16   0 13048 6836  10m S  0.0  0.7   0:00.15 apache2
6449 www-data  16   0 11428 5536 9496 S  0.0  0.5   0:00.05 apache2
6451 www-data  25   0  130m  99m  11m S  0.0  9.8   0:00.00 apache2
6453 www-data  25   0  129m  98m  11m S  0.0  9.7   0:00.00 apache2
6455 www-data  25   0  131m 100m  11m S  0.0  9.9   0:00.00 apache2
6457 www-data  25   0  128m  97m  11m S  0.0  9.7   0:00.00 apache2
6459 www-data  25   0  230m 9436  10m S  0.0  0.9   0:00.00 apache2
6460 www-data  25   0  230m 9428  10m S  0.0  0.9   0:00.00 apache2
6548 www-data  16   0  229m 8928  10m S  0.0  0.9   0:00.00 apache2

(Yup, Zope luvs it the memory)

One other thing I haven't figured out to do yet is to have mod_wsgi  
(or the loader script) do initialization of application data when  
Apache starts up.  This would be useful because currently when the  
server is started, doesn't initialize the application(s) until it  
receives a request; each process is initialized "on demand", so  
currently after a restart, it'll be common for some user to come in  
and wait up to 30 seconds (Plone startup dance) to get a single  
resource to load.  Only after all the processes are initialized as a  
result of enough concurrent requests does response time become  
predictable.

>   If one of the daemons hangs or leaks memory, what happens?

I havent tested a hang yet.  I suspect it uses a mechanism a lot like  
our event listener protocol when it tries to dispatch a request: the  
hung process is probably marked as "busy" and it becomes an orphan  
which is just never consulted again.  There's a rudimentary killoff- 
after-x-requests config knob, but no "max bytes" setting in  
mod_wsgi.  I don't see any way to get info about the state of the  
daemon children.

It would also be feasible to set up Apache without mod_wsgi and use  
mod_proxy+mod_rewrite to proxy requests to a cluster of ZServer-based  
Zope processes.  Then you can "control the horizontal and the  
vertical" with respect to the processes that comprise Zope (use  
supervisor to kill one of them off if it uses too much memory, put  
Pound in front of the cluster to do session affinity, etc).  This is  
the typical way larger Zope deployments are configured now.  It also  
solves the initialization problem.

- C

_______________________________________________
Repoze-dev mailing list
Repoze-dev at lists.repoze.org
http://lists.repoze.org/mailman/listinfo/repoze-dev