How to harden Apache web server with mod_security and mod_evasive on CentOS

Web server security is a vast subject, and different people have different preferences and opinions as to what the best tools and techniques are to harden a particular web server. With Apache web server, a great majority of experts -if not all- agree that mod_security and mod_evasive are two very important modules that can protect an Apache web server against common threats.

In this article, we will discuss how to install and configure mod_security and mod_evasive, assuming that Apache HTTP web server is already up and running. We will perform a demo stress test to see how the web server reacts when it is under a denial-of-service (DOS) attack, and show how it fights back with these modules. We will be using CentOS platform in this tutorial.

Installing mod_security & mod_evasive

If you haven't enabled the EPEL repository in your CentOS/RHEL server, you need to do so before installing these packages.

# yum install mod_security
# yum install mod_evasive

After the installation is complete, you will find the main configuration files inside /etc/httpd/conf.d:

Now you need to make sure that Apache loads both modules when it starts. Look for the following lines (or add them if they are not present) in mod_security.conf and mod_evasive.conf, respectively:

LoadModule security2_module modules/mod_security2.so
LoadModule evasive20_module modules/mod_evasive20.so

In the two lines above:

  • The LoadModule directive tells Apache to link in an object file (*.so), and adds it to the list of active modules.
  • security2_module and evasive20_module are the names of the modules.
  • modules/mod_security2.so and modules/mod_evasive20.so are relative paths from the /etc/httpd directory to the source files of the modules. This can be verified (and changed, if necessary) by checking the contents of the /etc/httpd/modules directory.

Now restart Apache web server:

# service httpd restart

Configuring mod_security

In order to use mod_security, a Core Rule Set (CRS) must be installed first. Basically, a CRS provides a web server with a set of rules on how to behave under certain conditions. Trustwave's SpiderLabs (the firm behind mod_security) provides the OWASP (Open Web Application Security Project) ModSecurity CRS.

To download and install the latest OWASP CRS, use the following commands.

# mkdir /etc/httpd/crs
# cd /etc/httpd/crs
# wget https://github.com/SpiderLabs/owasp-modsecurity-crs/tarball/master
# tar xzf master
# mv SpiderLabs-owasp-modsecurity-crs-ebe8790 owasp-modsecurity-crs

Now navigate to the installed OWASP CRS directory.

# cd /etc/httpd/crs/owasp-modsecurity-crs

In the OWASP CRS directory, you will find a sample file with rules (modsecurity_crs_10_setup.conf.example).

We will copy its contents into a new file named (for convenience) modsecurity_crs_10_setup.conf.

# cp modsecurity_crs_10_setup.conf.example modsecurity_crs_10_setup.conf

To tell Apache to use this file for mod_security module, insert the following lines in the /etc/httpd/conf/httpd.conf file. The exact paths may be different depending on where you unpack the CRS tarball.

<IfModule security2_module>
    Include crs/owasp-modsecurity-crs/modsecurity_crs_10_setup.conf
    Include crs/owasp-modsecurity-crs/base_rules/*.conf
</IfModule>

Last, but not least, we will create our own configuration file within the modsecurity.d directory where we will include our chosen directives. We will name this configuration file xmodulo.conf in this example. It is highly encouraged that you do not edit the CRS files directly but rather place all necessary directives in this configuration file. This will allow for easier upgrading as newer CRSs are released.

# vi /etc/httpd/modsecurity.d/xmodulo.conf
<IfModule mod_security2.c>
	SecRuleEngine On
	SecRequestBodyAccess On
	SecResponseBodyAccess On 
	SecResponseBodyMimeType text/plain text/html text/xml application/octet-stream 
	SecDataDir /tmp
</IfModule>
  • SecRuleEngine On: Use the OWASP CRS to detect and block malicious attacks.
  • SecRequestBodyAccess On: Enable inspection of data transported request bodies (e.g., POST parameters).
  • SecResponseBodyAccess On: Buffer response bodies (only if the response MIME type matches the list configured with SecResponseBodyMimeType).
  • SecResponseBodyMimeType text/plain text/html text/xml application/octet-stream: Configures which MIME types are to be considered for response body buffering. If you are unfamiliar with MIME types or unsure about their names or usage, you can check the Internet Assigned Numbers Authority (IANA) web site.
  • SecDataDir /tmp: Path where persistent data (e.g., IP address data, session data, and so on) is to be stored. Here persistent means anything that is not stored in memory, but on hard disk.

You can refer to the SpiderLabs' ModSecurity GitHub repository for a complete guide of configuration directives.

Don't forget to restart Apache to apply changes.

Configuring mod_evasive

The mod_evasive module reads its configuration from /etc/httpd/conf.d/mod_evasive.conf. As opposed to mod_security, we don't need a separate configuration file because there are no rules to update during a system or package upgrade.

The default mod_evasive.conf file has the following directives enabled:

<IfModule mod_evasive20.c>
    DOSHashTableSize    3097
    DOSPageCount        2
    DOSSiteCount        50
    DOSPageInterval     1
    DOSSiteInterval     1
    DOSBlockingPeriod   10
</IfModule>
  • DOSHashTableSize: The size of the hash table that is used to keep track of activity on a per-IP address basis. Increasing this number will provide a faster look up of the sites that the client has visited in the past, but may impact overall performance if it is set too high.
  • DOSPageCount: The number of identical requests to a specific URI (for example, a file that is being served by Apache) a visitor can make over the DOSPageInterval interval.
  • DOSSiteCount: similar to DOSPageCount, but refers to how many overall requests can be made to the site over the DOSSiteInterval interval.
  • DOSBlockingPeriod: If a visitor exceeds the limits set by DOSSPageCount or DOSSiteCount, he/she will be blacklisted for the DOSBlockingPeriod amount of time. During this interval, any requests coming from him/her will return a 403 Forbidden error.

You may want to change these values according to the amount and type of traffic that your web server needs to handle. Please note that if these values are not set properly, you may end up blocking legitimate visitors.

Here are other useful directives for mod_evasive:

1) DOSEmailNotify: Sends an email to the address specified whenever an IP address becomes blacklisted. It needs a valid email address as argument. If SELinux status is set to enforcing, you will need to grant the user apache SELinux permission to send emails. That is, run this command as root:

# setsebool -P httpd_can_sendmail 1

Then add this directive in the mod_evasive.conf file:

DOSEmailNotify you@yourdomain.com

2. DOSSystemCommand: Executes a custom system command whenever an IP address becomes blacklisted. It may come in handy to add firewall rules to block offending IPs altogether.

DOSSystemCommand <command>

We will use this directive to add a firewall rule through the following script (/etc/httpd/scripts/ban_ip.sh):

#!/bin/sh
# Offending IP as detected by mod_evasive
IP=$1
# Path to iptables binary executed by user apache through sudo
IPTABLES="/sbin/iptables"
# mod_evasive lock directory
MOD_EVASIVE_LOGDIR=/tmp
# Add the following firewall rule (block IP)
$IPTABLES -I INPUT -s $IP -j DROP
# Unblock offending IP after 2 hours through the 'at' command; see 'man at' for further details
echo "$IPTABLES -D INPUT -s $IP -j DROP" | at now + 2 hours
# Remove lock file for future checks
rm -f "$MOD_EVASIVE_LOGDIR"/dos-"$IP"

Our DOSSystemCommand directive will then read as follows:

DOSSystemCommand "sudo /etc/httpd/scripts/ban_ip.sh %s" 

Don't forget to update sudo permissions to run our script as apache user:

# vi /etc/sudoers
apache ALL=NOPASSWD: /usr/local/bin/scripts/ban_ip.sh
Defaults:apache !requiretty

Simulating DoS Attacks

We will use three tools to stress test our Apache web server (running on CentOS 6.5 with 512 MB of RAM and a AMD Athlon II X2 250 Processor), with and without mod_security and mod_evasive enabled, and check how the web server behaves in each case.

Make sure you ONLY perform the following steps in your own test server and NOT against an external, production web site.

In the following examples, replace http://centos.gabrielcanepa.com.ar/index.php with your own domain and a file of your choosing.

Linux-based tools

1. Apache bench: Apache server benchmarking tool.

# ab -n1000 -c1000 http://centos.gabrielcanepa.com.ar/index.php
  • -n: Number of requests to perform for the benchmarking session.
  • -c: Number of multiple requests to perform at a time.

2. test.pl: a Perl script which comes with mod_evasive module.

#!/usr/bin/perl

# test.pl: small script to test mod_dosevasive's effectiveness

use IO::Socket;
use strict;

for(0..100) {
  my($response);
  my($SOCKET) = new IO::Socket::INET( Proto   => "tcp",
                                      PeerAddr=> "192.168.0.16:80");
  if (! defined $SOCKET) { die $!; }
  print $SOCKET "GET /?$_ HTTP/1.0\n\n";
  $response = <$SOCKET>;
  print $response;
  close($SOCKET);
}

Windows-based tools

1. Low Orbit Ion Cannon (LOIC): a network stress testing tool. To generate a workload, follow the order shown in the screenshot below and DO NOT touch anything else

Stress Test Results

With mod_security and mod_evasive enabled (and the three tools running at the same time), the CPU and RAM usage peak at a maximum of 60% and 50%, respectively for only 2 seconds before the source IPs are blacklisted, blocked by the firewall, and the attack is stopped.

On the other hand, if mod_security and mod_evasive are disabled, the three tools mentioned above knock down the server very fast (and keep it in that state throughout the duration of the attack), and of course, the offending IPs are not blacklisted.

Conclusion

We can see that mod_security and mod_evasive, when properly configured, are two important tools to harden an Apache web server against several threats (not limited to DoS attacks) and should be considered in deployments exposed on the Internet.

Subscribe to Xmodulo

Do you want to receive Linux FAQs, detailed tutorials and tips published at Xmodulo? Enter your email address below, and we will deliver our Linux posts straight to your email box, for free. Delivery powered by Google Feedburner.


Support Xmodulo

Did you find this tutorial helpful? Then please be generous and support Xmodulo!

The following two tabs change content below.
Gabriel Cánepa is a GNU/Linux sysadmin and web developer from Villa Mercedes, San Luis, Argentina. He works for a worldwide leading consumer product company and takes great pleasure in using FOSS tools to increase productivity in all areas of his daily work. When he's not typing commands or writing code or articles, he enjoys telling bedtime stories with his wife to his two little daughters and playing with them, the great pleasure of his life.

10 thoughts on “How to harden Apache web server with mod_security and mod_evasive on CentOS

  1. Hello Mr. Canepa,

    As usual, a very good tutorial. But some improvements could be made in my opinion:

    1. One error:

    In sudoers file you have:
    apache ALL=NOPASSWD: /usr/local/bin/scripts/ban_ip.sh

    .... but the script location is in "/etc/httpd/scripts/ban_ip.sh"

    ????

    2. The script /etc/httpd/scripts/ban_ip.sh who will be executed under apache user, MUST be with NO write access, so the "best of the best" for this is:

    chattr +i /etc/httpd/scripts/ban_ip.sh

    - "chattr +i" will prevent ANYONE(root included) to modifies this file, so apache process will not be able to change anything on this nice script;
    - when the root will need to make some changes on this script he must execute the reverse command(chattr -i /etc/httpd/scripts/ban_ip.sh), and after he finish, must re-execute chattr +i /etc/httpd/scripts/ban_ip.sh

    3. In the same script you have this:

    rm -f "$MOD_EVASIVE_LOGDIR"/dos-"$IP"

    ... but who will create this lock file? And what is the "future checks"?

    4. If the apache server will be under a lot of offending IPs which are distributed (DDoS), your iptables will need to parse a huge list of addresses. So the best solution will be to use ipset (yum install ipset) to make your block-list.

    Congratulation for this good tutorial.

  2. @Iulian Murgulet,
    Thanks for your comment! As last time, you provided some very valuable insights. Let me address them in order:
    1) You are correct - the mod_evasive.conf entry should read as follows:
    DOSSystemCommand /usr/local/bin/scripts/ban_ip.sh
    All I have to say in my defense is that the first location where I placed the script was in the /etc/httpd/scripts/ directory and when I changed it to /usr/local/bin/scripts/ (more appropriate) I forgot to update the reference. My bad.
    2) Great idea! I will contact the owner of the website to add that tip to the article.
    3) The lock file is created by Apache when mod_evasive detects an ongoing threat (at the same time, a rule is added to iptables in order to block the offending IP). "Future checks" refers to the fact that you may opt to delete the lock file in order for mod_evasive to check that IP again in the future. In other words, if an attack is performed from IP AAA.BBB.CCC.DDD, a lock file named dos-AAA.BBB.CCC.DDD will be created inside /tmp (which is the default directory where mod_evasive creates its lock files... yes, I know that writing a lock file to /tmp may not be the best idea. That's just the way it's set up by default in the source code of mod_evasive. You can change it using the DOSLogDir directive in the mod_evasive.conf file and use another directory with the proper permissions set.) and as long as that file exists, mod_evasive will not analyze that IP again. Note that you can omit the last 4 lines of the script if you want to block an offending IP for good.
    4) I must admit I didn't know about ipset. Sounds like a great tool but time did not allow to set up a lab in order to simulate a DDoS. That's why I focused on DoS exclusively in this article (attacks coming from only 3 sources). I will take a closer look at ipset in order to investigate further. I have to thank you for suggesting another topic for a future article.
    I hope I explained myself well. If not, don't hesitate to add another comment.
    Thanks again for your very well informative comments.

  3. Hello again,

    1. ..... anyone makes mistakes, me included.
    3. ..... then a better location would be /var/tmp ...!
    4. Ipset is the best tool which can deal with a huge list of IPs and with a very small footprint on server resources. And it is very fast even on lists with more than 4-5000 of different IPs running on an old server.

  4. Hello,

    I tried your provided steps and all went well except one.

    at command is unable to remove firewall rule. however i can see that the command has been inserted perfectly by ban_ip.sh script via at -l command.

    Can you please suggest what could be the issue?

    1. getenforce
    Disabled

  5. Hi. I have a WordPress website that I installed these security measures on. The site is still accessible but the layout of the site has changed. If i disable mod_security, or more specifically OWASP CRS the site looks like original.

    Any clues?

    Thanks

    Brian

  6. Great article so far.

    Two questions in regards to mod_security

    1. why not using yum install mod_security_crs instead of downloading and unpackaging the related files?
    2. where exactly should I look for
    LoadModule security2_module modules/mod_security2.so
    LoadModule evasive20_module modules/mod_evasive20.so

    OR where exactly should I add those lines?

  7. Hello

    I've installed it, but testing with ab does not seems to show mod_evasive working. The server take a big load.

    Modules are correctly loaded.

    How to debug it ?

  8. Hello I followed this guide (first half mod security). Everything seems to work okay when I browse the localhost, but when accessing from another PC to the website's host IP address, the apache 2 test page displays instead of the website (which worked before).

    Any tips would be helpful

    Thank you!

  9. @All,
    First off, I apologize for not replying sooner.
    Make sure that the following line:
    "Defaults requiretty"
    is commented out in the sudoers file.
    Keep in mind that as a default security policy, you can only run sudo in a terminal - but in this case you will need to use sudo without a tty.

Leave a comment

Your email address will not be published. Required fields are marked *