.\" Automatically generated by Pod::Man v1.37, Pod::Parser v1.32 .\" .\" Standard preamble: .\" ======================================================================== .de Sh \" Subsection heading .br .if t .Sp .ne 5 .PP \fB\\$1\fR .PP .. .de Sp \" Vertical space (when we can't use .PP) .if t .sp .5v .if n .sp .. .de Vb \" Begin verbatim text .ft CW .nf .ne \\$1 .. .de Ve \" End verbatim text .ft R .fi .. .\" Set up some character translations and predefined strings. \*(-- will .\" give an unbreakable dash, \*(PI will give pi, \*(L" will give a left .\" double quote, and \*(R" will give a right double quote. | will give a .\" real vertical bar. \*(C+ will give a nicer C++. Capital omega is used to .\" do unbreakable dashes and therefore won't be available. \*(C` and \*(C' .\" expand to `' in nroff, nothing in troff, for use with C<>. .tr \(*W-|\(bv\*(Tr .ds C+ C\v'-.1v'\h'-1p'\s-2+\h'-1p'+\s0\v'.1v'\h'-1p' .ie n \{\ . ds -- \(*W- . ds PI pi . if (\n(.H=4u)&(1m=24u) .ds -- \(*W\h'-12u'\(*W\h'-12u'-\" diablo 10 pitch . if (\n(.H=4u)&(1m=20u) .ds -- \(*W\h'-12u'\(*W\h'-8u'-\" diablo 12 pitch . ds L" "" . ds R" "" . ds C` "" . ds C' "" 'br\} .el\{\ . ds -- \|\(em\| . ds PI \(*p . ds L" `` . ds R" '' 'br\} .\" .\" If the F register is turned on, we'll generate index entries on stderr for .\" titles (.TH), headers (.SH), subsections (.Sh), items (.Ip), and index .\" entries marked with X<> in POD. Of course, you'll have to process the .\" output yourself in some meaningful fashion. .if \nF \{\ . de IX . tm Index:\\$1\t\\n%\t"\\$2" .. . nr % 0 . rr F .\} .\" .\" For nroff, turn off justification. Always turn off hyphenation; it makes .\" way too many mistakes in technical documents. .hy 0 .if n .na .\" .\" Accent mark definitions (@(#)ms.acc 1.5 88/02/08 SMI; from UCB 4.2). .\" Fear. Run. Save yourself. No user-serviceable parts. . \" fudge factors for nroff and troff .if n \{\ . ds #H 0 . ds #V .8m . ds #F .3m . ds #[ \f1 . ds #] \fP .\} .if t \{\ . ds #H ((1u-(\\\\n(.fu%2u))*.13m) . ds #V .6m . ds #F 0 . ds #[ \& . ds #] \& .\} . \" simple accents for nroff and troff .if n \{\ . ds ' \& . ds ` \& . ds ^ \& . ds , \& . ds ~ ~ . ds / .\} .if t \{\ . ds ' \\k:\h'-(\\n(.wu*8/10-\*(#H)'\'\h"|\\n:u" . ds ` \\k:\h'-(\\n(.wu*8/10-\*(#H)'\`\h'|\\n:u' . ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'^\h'|\\n:u' . ds , \\k:\h'-(\\n(.wu*8/10)',\h'|\\n:u' . ds ~ \\k:\h'-(\\n(.wu-\*(#H-.1m)'~\h'|\\n:u' . ds / \\k:\h'-(\\n(.wu*8/10-\*(#H)'\z\(sl\h'|\\n:u' .\} . \" troff and (daisy-wheel) nroff accents .ds : \\k:\h'-(\\n(.wu*8/10-\*(#H+.1m+\*(#F)'\v'-\*(#V'\z.\h'.2m+\*(#F'.\h'|\\n:u'\v'\*(#V' .ds 8 \h'\*(#H'\(*b\h'-\*(#H' .ds o \\k:\h'-(\\n(.wu+\w'\(de'u-\*(#H)/2u'\v'-.3n'\*(#[\z\(de\v'.3n'\h'|\\n:u'\*(#] .ds d- \h'\*(#H'\(pd\h'-\w'~'u'\v'-.25m'\f2\(hy\fP\v'.25m'\h'-\*(#H' .ds D- D\\k:\h'-\w'D'u'\v'-.11m'\z\(hy\v'.11m'\h'|\\n:u' .ds th \*(#[\v'.3m'\s+1I\s-1\v'-.3m'\h'-(\w'I'u*2/3)'\s-1o\s+1\*(#] .ds Th \*(#[\s+2I\s-2\h'-\w'I'u*3/5'\v'-.3m'o\v'.3m'\*(#] .ds ae a\h'-(\w'a'u*4/10)'e .ds Ae A\h'-(\w'A'u*4/10)'E . \" corrections for vroff .if v .ds ~ \\k:\h'-(\\n(.wu*9/10-\*(#H)'\s-2\u~\d\s+2\h'|\\n:u' .if v .ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'\v'-.4m'^\v'.4m'\h'|\\n:u' . \" for low resolution devices (crt and lpr) .if \n(.H>23 .if \n(.V>19 \ \{\ . ds : e . ds 8 ss . ds o a . ds d- d\h'-1'\(ga . ds D- D\h'-1'\(hy . ds th \o'bp' . ds Th \o'LP' . ds ae ae . ds Ae AE .\} .rm #[ #] #H #V #F C .\" ======================================================================== .\" .IX Title "mod_perl_tuning 3" .TH mod_perl_tuning 3 "2007-03-30" "perl v5.8.8" "User Contributed Perl Documentation" .SH "NAME" mod_perl_tuning \- mod_perl performance tuning .SH "DESCRIPTION" .IX Header "DESCRIPTION" Described here are examples and hints on how to configure a mod_perl enabled Apache server, concentrating on tips for configuration for high-speed performance. The primary way to achieve maximal performance is to reduce the resources consumed by the mod_perl enabled \s-1HTTPD\s0 processes. .PP This document assumes familiarity with Apache configuration directives some familiarity with the mod_perl configuration directives, and that you have already built and installed a mod_perl enabled Apache server. Please also read the mod_perl documentation that comes with mod_perl for programming tips. Some configurations below use features from mod_perl version 1.03 which were not present in earlier versions. .PP These performance tuning hints are collected from my experiences in setting up and running servers for handling large promotional sites, such as The Weather Channel's \*(L"Blimp Site\-ings\*(R" game, the \s-1MSIE\s0 4.0 \&\*(L"Subscribe to Win\*(R" game, and the \s-1MSN\s0 Million Dollar Madness game. .SH "BASIC CONFIGURATION" .IX Header "BASIC CONFIGURATION" The basic configuration for mod_perl is as follows. In the \&\fIhttpd.conf\fR file, I add configuration parameters to make the \&\f(CW\*(C`http://www.domain.com/programs\*(C'\fR \s-1URL\s0 be the base location for all mod_perl programs. Thus, access to \&\f(CW\*(C`http://www.domain.com/programs/printenv\*(C'\fR will run the printenv script, as we'll see below. Also, any *.perl file will be interpreted as a mod_perl program just as if it were in the programs directory, and *.rperl will be mod_perl, but \fIwithout\fR any \s-1HTTP\s0 headers automatically sent; you must do this explicitly. If you don't want these last two, just leave it out of your configuration. .PP In the configuration files, I use \fI/var/www\fR as the \f(CW\*(C`ServerRoot\*(C'\fR directory, and \fI/var/www/docs\fR as the \f(CW\*(C`DocumentRoot\*(C'\fR. You will need to change it to match your particular setup. The network address below in the access to perl-status should also be changed to match yours. .PP Additions to \fIhttpd.conf\fR: .PP .Vb 10 \& # put mod_perl programs here \& # startup.perl loads all functions that we want to use within mod_perl \& Perlrequire /var/www/perllib/startup.perl \& \& AllowOverride None \& Options ExecCGI \& SetHandler perl-script \& PerlHandler Apache::Registry \& PerlSendHeader On \& .Ve .PP .Vb 8 \& # like above, but no PerlSendHeaders \& \& AllowOverride None \& Options ExecCGI \& SetHandler perl-script \& PerlHandler Apache::Registry \& PerlSendHeader Off \& .Ve .PP .Vb 7 \& # allow arbitrary *.perl files to be scattered throughout the site. \& \& SetHandler perl-script \& PerlHandler Apache::Registry \& PerlSendHeader On \& Options +ExecCGI \& .Ve .PP .Vb 7 \& # like *.perl, but do not send HTTP headers \& \& SetHandler perl-script \& PerlHandler Apache::Registry \& PerlSendHeader Off \& Options +ExecCGI \& .Ve .PP .Vb 7 \& \& SetHandler perl-script \& PerlHandler Apache::Status \& order deny,allow \& deny from all \& allow from 204.117.82. \& .Ve .PP Now, you'll notice that I use a \f(CW\*(C`PerlRequire\*(C'\fR directive to load in the file \fIstartup.perl\fR. In that file, I include all of the \f(CW\*(C`use\*(C'\fR statements that occur in any of my mod_perl programs (either from the programs directory, or the *.perl files). Here is an example: .PP .Vb 2 \& #! /usr/local/bin/perl \& use strict; .Ve .PP .Vb 3 \& # load up necessary perl function modules to be able to call from Perl-SSI \& # files. These objects are reloaded upon server restart (SIGHUP or SIGUSR1) \& # if PerlFreshRestart is "On" in httpd.conf (as of mod_perl 1.03). .Ve .PP .Vb 1 \& # only library-type routines should go in this directory. .Ve .PP .Vb 1 \& use lib "/var/www/perllib"; .Ve .PP .Vb 2 \& # make sure we are in a sane environment. \& $ENV{GATEWAY_INTERFACE} =~ /^CGI-Perl/ or die "GATEWAY_INTERFACE not Perl!"; .Ve .PP .Vb 1 \& use Apache::Registry (); # for things in the "/programs" URL .Ve .PP .Vb 6 \& # pull in things we will use in most requests so it is read and compiled \& # exactly once \& use CGI (); CGI->compile(':all'); \& use CGI::Carp (); \& use DBI (); \& use DBD::mysql (); .Ve .PP .Vb 1 \& 1; .Ve .PP What this does is pull in all of the code used by the programs (but does not \f(CW\*(C`import\*(C'\fR any of the module methods) into the main \s-1HTTPD\s0 process, which then creates the child processes with the code already in place. You can also put any new modules you like into the \&\fI/var/www/perllib\fR directory and simply \f(CW\*(C`use\*(C'\fR them in your programs. There is no need to put \f(CW\*(C`use lib "/var/www/perllib";\*(C'\fR in all of your programs. You do, however, still need to \f(CW\*(C`use\*(C'\fR the modules in your programs. Perl is smart enough to know it doesn't need to recompile the code, but it does need to \f(CW\*(C`import\*(C'\fR the module methods into your program's name space. .PP If you only have a few modules to load, you can use the PerlModule directive to pre-load them with the same effect. .PP The biggest benefit here is that the child process never needs to recompile the code, so it is faster to start, and the child process actually shares the same physical copy of the code in memory due to the way the virtual memory system in modern operating systems works. .PP You will want to replace the \f(CW\*(C`use\*(C'\fR lines above with modules you actually need. .Sh "Simple Test Program" .IX Subsection "Simple Test Program" Here's a sample script called \fIprintenv\fR that you can stick in the \&\fIprograms\fR directory to test the functionality of the configuration. .PP .Vb 3 \& #! /usr/local/bin/perl \& use strict; \& # print the environment in a mod_perl program under Apache::Registry .Ve .PP .Vb 1 \& print "Content-type: text/html\en\en"; .Ve .PP .Vb 1 \& print "Apache::Registry Environment\en"; .Ve .PP .Vb 3 \& print "
\en";
\& print map { "$_ = $ENV{$_}\en" } sort keys %ENV;
\& print "
\en"; .Ve .PP When you run this, check the value of the \s-1GATEWAY_INTERFACE\s0 variable to see that you are indeed running mod_perl. .SH "REDUCING MEMORY USE" .IX Header "REDUCING MEMORY USE" As a side effect of using mod_perl, your \s-1HTTPD\s0 processes will be larger than without it. There is just no way around it, as you have this extra code to support your added functionality. .PP On a very busy site, the number of \s-1HTTPD\s0 processes can grow to be quite large. For example, on one large site, the typical \s-1HTTPD\s0 was about 5Mb large. With 30 of these, all of \s-1RAM\s0 was exhausted, and we started to go to swap. With 60 of these, swapping turned into thrashing, and the whole machine slowed to a crawl. .PP To reduce thrashing, limiting the maximum number of \s-1HTTPD\s0 processes to a number that is just larger than what will fit into \s-1RAM\s0 (in this case, 45) is necessary. The drawback is that when the server is serving 45 requests, new requests will queue up and wait; however, if you let the maximum number of processes grow, the new requests will start to get served right away, \fIbut\fR they will take much longer to complete. .PP One way to reduce the amount of real memory taken up by each process is to pre-load commonly used modules into the primary \s-1HTTPD\s0 process so that the code is shared by all processes. This is accomplished by inserting the \f(CW\*(C`use Foo ();\*(C'\fR lines into the \fIstartup.perl\fR file for any \f(CW\*(C`use Foo;\*(C'\fR statement in any commonly used Registry program. The idea is that the operating system's \s-1VM\s0 subsystem will share the data across the processes. .PP You can also pre-load Apache::Registry programs using the \&\f(CW\*(C`Apache::RegistryLoader\*(C'\fR module so that the code for these programs is shared by all \s-1HTTPD\s0 processes as well. .PP \&\fB\s-1NOTE\s0\fR: When you pre-load modules in the startup script, you may need to kill and restart \s-1HTTPD\s0 for changes to take effect. A simple \&\f(CW\*(C`kill \-HUP\*(C'\fR or \f(CW\*(C`kill \-USR1\*(C'\fR will not reload that code unless you have set the \f(CW\*(C`PerlFreshRestart\*(C'\fR configuration parameter in \&\fIhttpd.conf\fR to be \*(L"On\*(R". .SH "REDUCING THE NUMBER OF LARGE PROCESSES" .IX Header "REDUCING THE NUMBER OF LARGE PROCESSES" Unfortunately, simply reducing the size of each \s-1HTTPD\s0 process is not enough on a very busy site. You also need to reduce the quantity of these processes. This reduces memory consumption even more, and results in fewer processes fighting for the attention of the \s-1CPU\s0. If you can reduce the quantity of processes to fit into \s-1RAM\s0, your response time is increased even more. .PP The idea of the techniques outlined below is to offload the normal document delivery (such as static \s-1HTML\s0 and \s-1GIF\s0 files) from the mod_perl \s-1HTTPD\s0, and let it only handle the mod_perl requests. This way, your large mod_perl \s-1HTTPD\s0 processes are not tied up delivering simple content when a smaller process could perform the same job more efficiently. .PP In the techniques below where there are two \s-1HTTPD\s0 configurations, the same httpd executable can be used for both configurations; there is no need to build \s-1HTTPD\s0 both with and without mod_perl compiled into it. With Apache 1.3 this can be done with the \s-1DSO\s0 configuration \*(-- just configure one httpd invocation to dynamically load mod_perl and the other not to do so. .PP These approaches work best when most of the requests are for static content rather than mod_perl programs. Log file analysis become a bit of a challenge when you have multiple servers running on the same host, since you must log to different files. .Sh "\s-1TWO\s0 \s-1MACHINES\s0" .IX Subsection "TWO MACHINES" The simplest way is to put all static content on one machine, and all mod_perl programs on another. The only trick is to make sure all links are properly coded to refer to the proper host. The static content will be served up by lots of small \s-1HTTPD\s0 processes (configured \&\fInot\fR to use mod_perl), and the relatively few mod_perl requests can be handled by the smaller number of large \s-1HTTPD\s0 processes on the other machine. .PP The drawback is that you must maintain two machines, and this can get expensive. For extremely large projects, this is the best way to go. .Sh "\s-1TWO\s0 \s-1IP\s0 \s-1ADDRESSES\s0" .IX Subsection "TWO IP ADDRESSES" Similar to above, but one \s-1HTTPD\s0 runs bound to one \s-1IP\s0 address, while the other runs bound to another \s-1IP\s0 address. The only difference is that one machine runs both servers. Total memory usage is reduced because the majority of files are served by the smaller \s-1HTTPD\s0 processes, so there are fewer large mod_perl \s-1HTTPD\s0 processes sitting around. .PP This is accomplished using the \fIhttpd.conf\fR directive \f(CW\*(C`BindAddress\*(C'\fR to make each \s-1HTTPD\s0 respond only to one \s-1IP\s0 address on this host. One will have mod_perl enabled, and the other will not. .Sh "\s-1TWO\s0 \s-1PORT\s0 \s-1NUMBERS\s0" .IX Subsection "TWO PORT NUMBERS" If you cannot get two \s-1IP\s0 addresses, you can also split the \s-1HTTPD\s0 processes as above by putting one on the standard port 80, and the other on some other port, such as 8042. The only configuration changes will be the \f(CW\*(C`Port\*(C'\fR and log file directives in the httpd.conf file (and also one of them does not have any mod_perl directives). .PP The major flaw with this scheme is that some firewalls will not allow access to the server running on the alternate port, so some people will not be able to access all of your pages. .PP If you use this approach or the one above with dual \s-1IP\s0 addresses, you probably do not want to have the *.perl and *.rperl sections from the sample configuration above, as this would require that your primary \&\s-1HTTPD\s0 server be mod_perl enabled as well. .PP Thanks to Gerd Knops for this idea. .Sh "\s-1USING\s0 ProxyPass \s-1WITH\s0 \s-1TWO\s0 \s-1SERVERS\s0" .IX Subsection "USING ProxyPass WITH TWO SERVERS" To overcome the limitation of the alternate port above, you can use dual Apache \s-1HTTPD\s0 servers with just slight difference in configuration. Essentially, you set up two servers just as you would with the two port on same \s-1IP\s0 address method above. However, in your primary \s-1HTTPD\s0 configuration you add a line like this: .PP .Vb 1 \& ProxyPass /programs http://localhost:8042/programs .Ve .PP Where your mod_perl enabled \s-1HTTPD\s0 is running on port 8042, and has only the directory \fIprograms\fR within its DocumentRoot. This assumes that you have included the mod_proxy module in your server when it was built. .PP Now, when you access http://www.domain.com/programs/printenv it will internally be passed through to your \s-1HTTPD\s0 running on port 8042 as the \&\s-1URL\s0 http://localhost:8042/programs/printenv and the result relayed back transparently. To the client, it all seems as if it is just one server running. This can also be used on the dual-host version to hide the second server from view if desired. .PP Thanks to Bowen Dwelle for this idea. .Sh "\s-1SQUID\s0 \s-1ACCELERATOR\s0" .IX Subsection "SQUID ACCELERATOR" Another approach to reducing the number of large \s-1HTTPD\s0 processes on one machine is to use an accelerator such as Squid (which can be found at http://squid.nlanr.net/Squid/ on the web) between the clients and your large mod_perl \s-1HTTPD\s0 processes. The idea here is that squid will handle the static objects from its cache while the \s-1HTTPD\s0 processes will handle mostly just the mod_perl requests once the cache is primed. This reduces the number of \s-1HTTPD\s0 processes and thus reduces the amount of memory used. .PP To set this up, just install the current version of Squid (at this writing, this is version 1.1.22) and use the RunAccel script to start it. You will need to reconfigure your \s-1HTTPD\s0 to use an alternate port, such as 8042, rather than its default port 80. To do this, you can either change the \fIhttpd.conf\fR line \f(CW\*(C`Port\*(C'\fR or add a \f(CW\*(C`Listen\*(C'\fR directive to match the port specified in the \fIsquid.conf\fR file. Your URLs do not need to change. The benefit of using the \f(CW\*(C`Listen\*(C'\fR directive is that redirected URLs will still use the default port 80 rather than your alternate port, which might reveal your real server location to the outside world and bypass the accelerator. .PP In the \fIsquid.conf\fR file, you will probably want to add \f(CW\*(C`programs\*(C'\fR and \f(CW\*(C`perl\*(C'\fR to the \f(CW\*(C`cache_stoplist\*(C'\fR parameter so that these are always passed through to the \s-1HTTPD\s0 server under the assumption that they always produce different results. .PP This is very similar to the two port, ProxyPass version above, but the Squid cache may be more flexible to fine tune for dynamic documents that do not change on every view. The Squid proxy server also seems to be more stable and robust than the Apache 1.2.4 proxy module. .PP One drawback to using this accelerator is that the logfiles will always report access from \s-1IP\s0 address 127.0.0.1, which is the local host loopback address. Also, any access permissions or other user tracking that requires the remote \s-1IP\s0 address will always see the local address. The following code uses a feature of recent mod_perl versions (tested with mod_perl 1.16 and Apache 1.3.3) to trick Apache into logging the real client address and giving that information to mod_perl programs for their purposes. .PP First, in your \fIstartup.perl\fR file add the following code: .PP .Vb 1 \& use Apache::Constants qw(OK); .Ve .PP .Vb 2 \& sub My::SquidRemoteAddr ($) { \& my $r = shift; .Ve .PP .Vb 3 \& if (my ($ip) = $r->header_in('X-Forwarded-For') =~ /([^,\es]+)$/) { \& $r->connection->remote_ip($ip); \& } .Ve .PP .Vb 2 \& return OK; \& } .Ve .PP Next, add this to your \fIhttpd.conf\fR file: .PP .Vb 1 \& PerlPostReadRequestHandler My::SquidRemoteAddr .Ve .PP This will cause every request to have its \f(CW\*(C`remote_ip\*(C'\fR address overridden by the value set in the \f(CW\*(C`X\-Forwarded\-For\*(C'\fR header added by Squid. Note that if you have multiple proxies between the client and the server, you want the \s-1IP\s0 address of the last machine before your accelerator. This will be the right-most address in the X\-Forwarded-For header (assuming the other proxies append their addresses to this same header, like Squid does.) .PP If you use apache with mod_proxy at your frontend, you can use Ask Bjørn Hansen's mod_proxy_add_forward module from ftp://ftp.netcetera.dk/pub/apache/ to make it insert the \&\f(CW\*(C`X\-Forwarded\-For\*(C'\fR header. .SH "SUMMARY" .IX Header "SUMMARY" To gain maximal performance of mod_perl on a busy site, one must reduce the amount of resources used by the \s-1HTTPD\s0 to fit within what the machine has available. The best way to do this is to reduce memory usage. If your mod_perl requests are fewer than your static page requests, then splitting the servers into mod_perl and non\-mod_perl versions further allows you to tune the amount of resources used by each type of request. Using the \f(CW\*(C`ProxyPass\*(C'\fR directive allows these multiple servers to appear as one to the users. Using the Squid accelerator also achieves this effect, but Squid takes care of deciding when to acccess the large server automatically. .PP If all of your requests require processing by mod_perl, then the only thing you can really do is throw a \fIlot\fR of memory on your machine and try to tweak the perl code to be as small and lean as possible, and to share the virtual memory pages by pre-loading the code. .SH "AUTHOR" .IX Header "AUTHOR" This document is written by Vivek Khera. If you need to contact me, just send email to the mod_perl mailing list. .PP This document is copyright (c) 1997\-1998 by Vivek Khera. .PP If you have contributions for this document, please post them to the mailing list. Perl \s-1POD\s0 format is best, but plain text will do, too. .PP If you need assistance, contact the mod_perl mailing list at modperl@perl.apache.org first (send 'subscribe' to modperl\-request@apache.org to subscribe). There are lots of people there that can help. Also, check the web pages http://perl.apache.org/ and http://www.apache.org/ for explanations of the configuration options. .PP $Revision: 177689 $ \&\f(CW$Date:\fR 2002\-03\-24 18:57:59 \-0800 (Sun, 24 Mar 2002) $