Subject: Performance Tuning on Apache 1.3.24 with mod_ssl 2.8.8

From: Patrick Dionisio <[email-masked]>
Date: Thu, 30 May 2002 13:38:23 -0700 (PDT)

Hi, I'd like to know what kind of tricks I can apply
to improve the performance of my apache server which
uses mod_ssl.  The OS I'm using is Linux 7.2.

Currently, I have a client script that generates n
number of requests to the apache server.  The page it
requests is a static page.  With SSL turned on, I'm
only able to get at most 7 to 8 requests per second. 
With SSL turned off, I am able to get 50+ requests per
second.

I've tried setting SSLMutex to use sem and
SSLSessionCache to
shm:/usr/local/apache/logs/ssl_gcache_data(512000),
but those changes didn't improve the results.

Any suggestions or ideas?  Thanks.

Patrick

__________________________________________________
Do You Yahoo!?
Yahoo! - Official partner of 2002 FIFA World Cup
http://fifaworldcup.yahoo.com
______________________________________________________________________
Apache Interface to OpenSSL (mod_ssl)                   www.modssl.org
User Support Mailing List                      [email-masked]
Automated List Manager                            [email-masked]

---

From: Cliff Woolley <[email-masked]>
Date: Thu, 30 May 2002 18:58:58 -0400 (EDT)

> With SSL turned off, I am able to get 50+ requests per

Wow, that's still incredibly slow.  What kind of CPU and how much RAM are
we talking about here?  With SSL turned off you should be able to pump out
way more RPS than that on a static page.  I suggest you tune that first
(you should be looking for a number in the hundreds of RPS at least), and
*then* focus on SSL.  See:

http://httpd.apache.org/docs/misc/perf-tuning.html

Upgrading to Apache 2.0.x might help, too.  :)

> shm:/usr/local/apache/logs/ssl_gcache_data(512000),

shmcb can perform better than shmht under stress (shm == shmht in 1.3, shm
== shmcb in 2.0, though you can explicitly specify either choice in both
versions)... that's probably worth looking into.  See the thread
http://marc.theaimsgroup.com/?l=apache-modssl&m=98529562629436&w=2 for an
explanation of the differences (though some of the information there is
out of date by now, eg shmcb is no longer experimental).

> but those changes didn't improve the results.

It should actually be a rather drastic improvement over other session
cache methods.  I definitely think you need to concentrate on the rest of
Apache first and then come back to looking at SSL tuning.

--Cliff

---

From: Geoff Thorpe <[email-masked]>
Date: Fri, 31 May 2002 14:05:33 -0400 (EDT)

Hi there,

> *then* focus on SSL.  See:

As a first tip - 50 requests per second is very slow already just for
http. I'd look at that first. Don't forget to bear in mind the size of the
page you're pulling down with your http request - multiply that by 50 and
check that you're not approaching any bandwidth limitation of your network
interfaces! :-)

Aside from that - there's a variety of settings in the default apache
config (at least this is true for 1.3.*) that although "generic" and
"helpful" are most certainly not "optimal". Just pulling down
http://<whatever-host>/ (ie. the "default page") can involve multiple file
I/O calls by apache just trying to figure out which HTML file to use (ie.
mime-magic, language support, etc). Numerous "Options" directives in
apache slow down generic operation so you may want to wade into the config
file pruning what you can. Likewise, turning off keepalives (which are
evil and should be amputated from all existing and future source) can be a
good idea - the little bit of one-browser-straight-line speed improvement
keepalives give a browser are more than compensated for by the
process-bloat and scalability hassles it gives your server (especially as
modern browsers launch multiple requests in parallel anyway when they want
to "speed up").

I found that I could eek quite a bit of speed improvement out of Apache
just by tweaking the config file and removing fancy (and almost never
used) modules and options.

Then you move onto the ssl-specific stuff ... disabling the "COMPAT" stuff
in mod_ssl is a good idea - last time I checked, the code that populates
environment variables with https-specifics was completely ass-about-face.
I measured something ridiculous like 20,000 strcmp() operations for a
single https handshake. Turning off "compat" support doesn't remove that
problem, but mitigates it somewhat by reducing (substantially) the number
of environment variables modssl tries to populate. Ie. this reduces the
number of iterations of the (slow) loop logic. You also get some mileage
by reducing the verbosity of log output - I'd recommend "Warn" as the
noisiest level you'd want if performance is important (for the regular
Apache LogLevel as well as the modssl-specific one).

> http://httpd.apache.org/docs/misc/perf-tuning.html

You might also want to check the README in the 'swamp' package (shameless
plug, http://www.geoffthorpe.net/crypto/swamp/) - apart from explaining
the usage of 'swamp' (which you may not care for) it does go into a
variety of considerations about client and/or server speeds and how to
meaningfully benchmark and interpret results. Just to start off with,
you've probably (with your https tests) fallen into the first gotcha - EDH
cipher suites. It wouldn't surprise me if your benchmarking program was
negotiating these much slower but higher-security cipher-suites. These
suites aren't actually supported by common browsers anwyay so the
usefulness of those numbers is questionable. OTOH: If you're only getting
50 ops/sec with plain http then it could also just be a hopelessly slow
web server. If it *is* EDH cipher-suites, then your numbers could go up by
a factor of 5 or much more if you test with non-ephemeral suites (eg.
RC4-SHA).

> out of date by now, eg shmcb is no longer experimental).

What my failed searches for benchmarking posts *did* turn up was a bit of
info on the 'shmcb' stuff. Eg. some misc. posts of mine that turned up in
that search that touch on session caching and testing (in no particular
order);

(a bit of a monster about 'shmcb')
http://marc.theaimsgroup.com/?l=apache-modssl&m=98531062704750&w=2

(a bit on swamp usage and session caching)
http://marc.theaimsgroup.com/?l=apache-modssl&m=98651105121737&w=2

(something else about problems with 'shmht')
http://marc.theaimsgroup.com/?l=apache-modssl&m=99997423802243&w=2

oh yeah, there's also that security problem with modssl that I mentioned
ages ago - AFAIK this still hasn't been changed in modssl and *may* not
yet have changed in apache 2.0 either. Ralf or David, please correct me
if I'm wrong;

http://marc.theaimsgroup.com/?l=apache-modssl&m=99717585106420&w=2

> Apache first and then come back to looking at SSL tuning.

Yes, if the move from dbm to shm (any kind of shm for that matter) didn't
show up on your benchmarks, then there's some other kind of sludge in your
setup that is large enough to obscure the benefits of tighter tuning at
the session caching level. Again, I'd recommend taking a look at 'swamp's
README (you can read it online if you don't want to download the package).

Regards,
Geoff

-- 
Geoff Thorpe, geoff(at)geoffthorpe(dot)net

2000 years on, it's a different empire but the same
zealots and the same attrocities.

---

From: Cliff Woolley <[email-masked]>
Date: Fri, 31 May 2002 15:12:42 -0400 (EDT)

> http://marc.theaimsgroup.com/?l=apache-modssl&m=99717585106420&w=2

This was fixed in 2.0 as of 2.0.25 but is not yet fixed in 1.3's modssl.

--Cliff

---

From: Geoff Thorpe <[email-masked]>
Date: Sat, 01 Jun 2002 11:56:28 -0400 (EDT)

Hi,

> This was fixed in 2.0 as of 2.0.25 but is not yet fixed in 1.3's modssl.

Ah, thanks for the update on that. I mentioned this problem a couple of
times *ages* ago, including private mail to Ralf, but it seemed very few
people seemed to regard it as "an issue". I'm glad Apache 2.0 has taken it
seriously. Ralf, would it be possible to get it similarly incorporated
into the 1.3.* tree? Please?

Cheers,
Geoff