How to set up a transparent HTTPS filtering proxy on CentOS

HTTPS protocol is used more and more in today’s web. While this may be good for privacy, it leaves modern network administrator without any means to prevent questionable or adult contents from entering his/her network. Previously it was assumed that this problem does not have a decent solution. Our how-to guide will try to prove otherwise.

This guide will tell you how to set up Squid on CentOS / RedHat Linux for transparent filtering of HTTP and HTTPS traffic with help of Diladele Web Safety ICAP server, which is a commercial solution for Linux, BSD and MacOS. The Linux installer of Diladele Web Safety used in this tutorial contains fully featured keys which remain valid for 3 month period, so you can test its full features during this trial period.

Assumptions and Requirements

In this tutorial, I will assume the following. You have a network with IP addresses from 192.168.1.0 subnet, network mask is 255.255.255.0, and all workstations are set to use 192.168.1.1 as default gateway. On this default gateway, you have two NICs - one facing LAN with IP address 192.168.1.1, the other is plugged in into ISP network and gets its public Internet address through DHCP. It is also assumed your gateway has CentOS or RedHat Linux up and running.

Step 1. Update and Upgrade

Before going further, run the following script to upgrade your system to the most recent state.

#!/bin/bash
set -e

# update should be done as root
if [[ $EUID -ne 0 ]]; then
   echo "This script must be run as root" 1>&2
   exit 1
fi

# update and upgrade
yum update && yum upgrade 

# disable selinux
sed -i s/SELINUX=enforcing/SELINUX=disabled/g /etc/selinux/config

# and reboot
reboot

Step 2. Install Apache Web Server

Diladele Web Safety has sophisticated a web administrator console to easily manage filtering settings and policies. This Web UI is built using Python Django web framework, and requires Apache web server to function correctly. Run the following script to install them.

#!/bin/bash
set -e

# all web packages are installed as root
if [[ $EUID -ne 0 ]]; then
   echo "This script must be run as root" 1>&2
   exit 1
fi

# install python libs
yum install python-setuptools python-ldap

# install python django for web ui
easy_install django==1.5

# install apache web server to run web ui
yum install httpd php mod_wsgi

# make apache autostart on reboot
chkconfig httpd on

# this fixes some apache errors when working with python-django wsgi
echo "WSGISocketPrefix /var/run/wsgi" >> /etc/httpd/conf.d/wsgi.conf

# and restart apache
service httpd restart

Step 3. Install Diladele Web Safety

Download and install the latest version of Diladele Web Safety using the following script.

#!/bin/bash

# all packages are installed as root
if [[ $EUID -ne 0 ]]; then
   echo "This script must be run as root" 1>&2
   exit 1
fi

# detect current architecture (default assumes x86_64)
ARCH_1=`uname -m`
ARCH_2="amd64"
if [[ $ARCH_1 == 'i686' ]]; then
        ARCH_1="i386"
        ARCH_2="i386"
fi

# bail out on any error
set -e

# get latest qlproxy
curl http://updates.diladele.com/qlproxy/binaries/3.2.0.4CAF/$ARCH_2/release/centos6/qlproxy-3.2.0-4CAF.$ARCH_1.rpm > qlproxy-3.2.0-4CAF.$ARCH_1.rpm

# install it
yum -y --nogpgcheck localinstall qlproxy-3.2.0-4CAF.$ARCH_1.rpm
  
# qlproxy installed everything needed for apache, so just restart
service httpd restart

Step 4. Install Required Build Tools

To be able to perform HTTP/HTTPS transparent filtering, we need to get the latest version of Squid (the one that comes with CentOS / RedHat by default is too outdated), and rebuild it from source. The following script installs all build tools required.

#!/bin/bash

# install all build tools
if [[ $EUID -ne 0 ]]; then
   echo "This script must be run as root" 1>&2
   exit 1
fi

# install development packages required
yum install -y gcc-c++ pam-devel db4-devel expat-devel libxml2-devel libcap-devel libtool redhat-rpm-config rpm-build openldap-devel openssl-devel krb5-devel

# squid needs perl and needs additional perl modules not present by default in CentOS 6
curl http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm > epel-release-6-8.noarch.rpm
rpm -Uvh epel-release-6*.rpm
yum install -y perl-Crypt-OpenSSL-X509

Step 5. Build Squid from Source

Rebuild the Squid RPM by running the following script.

#!/bin/bash

# stop on any error
set -e

# rpm build MUST be run as normal user
if [[ $EUID -eq 0 ]]; then
   echo "This script must NOT be run as root" 1>&2
   exit 1
fi

# get squid sources
pushd rpmbuild/SOURCES
curl http://www.squid-cache.org/Versions/v3/3.4/squid-3.4.4.tar.xz > squid-3.4.4.tar.xz
curl http://www.squid-cache.org/Versions/v3/3.4/squid-3.4.4.tar.xz.asc > squid-3.4.4.tar.xz.asc
popd

# build the binaries RPMs out of sources
pushd rpmbuild/SPECS
rpmbuild -v -bb squid.spec
popd

Step 6. Install Squid

After build finishes, install Squid. It is advisable to uncomment the lines which generate your own root certification authority. Default installation of Diladele Web Safety does have its own ca, but trusting it may pose serious security risk if your devices are used by users outside of your network.

#!/bin/bash

# stop on every error
set -e

# install RPMs as root
if [[ $EUID -ne 0 ]]; then
   echo "This script must be run as root" 1>&2
   exit 1
fi

# detect current architecture (default assumes x86_64)
ARCH_1=`uname -m`
ARCH_2="amd64"
ARCH_3="lib64"

if [[ $ARCH_1 == 'i686' ]]; then
        ARCH_2="i386"
        ARCH_3="lib"
fi

pushd rpmbuild/RPMS/$ARCH_1
yum localinstall -y squid-3.4.4-0.el6.$ARCH_1.rpm
popd

# set up the ssl_crtd daemon
if [ -f /bin/ssl_crtd ]; then
	rm -f /bin/ssl_crtd
fi

ln -s /usr/$ARCH_3/squid/ssl_crtd /bin/ssl_crtd
/bin/ssl_crtd -c -s /var/spool/squid_ssldb
chown -R squid:squid /var/spool/squid_ssldb

# uncomment to regenerate certificates for SSL bumping if you do not like defaults
# openssl req -new -newkey rsa:1024 -days 1365 -nodes -x509 -keyout myca.pem  -out myca.pem
# openssl x509 -in myca.pem -outform DER -out myca.der
# then copy certificates 
# cp myca.pem /etc/opt/quintolabs/qlproxy/
# cp myca.der /etc/opt/quintolabs/qlproxy/

# make squid autostart after reboot
chkconfig squid on

Step 7. Integrate Squid with Diladele Web Safety

Integrate Squid and Diladele Web Safety by running the following script.

#!/bin/bash

# stop on any error
set -e

# integration should be done as root
if [[ $EUID -ne 0 ]]; then
   echo "This script must be run as root" 1>&2
   exit 1
fi

# allow web ui read-only access to squid configuration file
chmod o+r /etc/squid/squid.conf

# perform integration by replacing squid.conf file
mv /etc/squid/squid.conf /etc/squid/squid.conf.original && mv squid.conf /etc/squid/squid.conf

# parse the resulting config just to be sure
/usr/sbin/squid -k parse

# restart squid to load all config
/sbin/service squid restart

Step 8. Transparently Redirect HTTPS Traffic to Squid

Transparent filter for HTTP and HTTPS traffic will be implemented by redirecting traffic to ports 80 and 443 to Squid using iptables. This implies that the box with Squid acts as default gateway for your LAN. Please note this is only one way to implementing transparent filtering. Other possible solutions are described in Squid’s Wiki.

#!/bin/bash

# firewall setup should be done as root
if [[ $EUID -ne 0 ]]; then
   echo "This script must be run as root" 1>&2
   exit 1
fi

# check kernel forwarding is enabled
enabled=`cat /proc/sys/net/ipv4/ip_forward`
if [[ $enabled -ne 1 ]]; then
        echo "Kernel forwarding seems to be disabled, enable it in /etc/sysctl.conf, reboot and rerun this script" 1>&2
        exit 1
fi

# set the default policy to accept first (not to lock ourselves out from remote machine)
iptables -P INPUT ACCEPT

# flush all current rules from iptables
iptables -F

# allow pings from eth0 and eth1 for debugging purposes
iptables -A INPUT -p icmp -j ACCEPT

# allow access for localhost
iptables -A INPUT -i lo -j ACCEPT

# accept packets belonging to established and related connections
iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

# allow ssh connections to tcp port 22 from eth0 and eth1
iptables -A INPUT -p tcp --dport 22 -j ACCEPT

# allow connection from LAN to ports 3126, 3127 and 3128 squid is running on
iptables -A INPUT -i eth0 -p tcp --dport 3126 -j ACCEPT
iptables -A INPUT -i eth0 -p tcp --dport 3127 -j ACCEPT
iptables -A INPUT -i eth0 -p tcp --dport 3128 -j ACCEPT

# redirect all HTTP(tcp:80) traffic coming in through eth0 to 3126
iptables -t nat -A PREROUTING -i eth0 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 3126

# redirect all HTTPS(tcp:443) traffic coming in through eth0 to 3127
iptables -t nat -A PREROUTING -i eth0 -p tcp -m tcp --dport 443 -j REDIRECT --to-ports 3127

# configure forwarding rules
iptables -A FORWARD -i eth0 -o eth1 -p tcp --dport 22 -j ACCEPT
iptables -A FORWARD -i eth1 -o eth0 -p tcp --sport 22 -j ACCEPT
iptables -A FORWARD -p icmp -j ACCEPT
iptables -A FORWARD -i eth0 -o eth1 -p tcp --dport 80 -j ACCEPT
iptables -A FORWARD -i eth1 -o eth0 -p tcp --sport 80 -j ACCEPT
iptables -A FORWARD -i eth0 -o eth1 -p tcp --dport 53 -j ACCEPT
iptables -A FORWARD -i eth0 -o eth1 -p udp --dport 53 -j ACCEPT
iptables -A FORWARD -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A FORWARD -j REJECT --reject-with icmp-host-prohibited

# enable NAT for clients within LAN
iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADE

# set default policies for INPUT, FORWARD (drop) and OUTPUT (accept) chains
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -P OUTPUT ACCEPT

# list created rules
iptables -L -v

# save the rules so that after reboot they are automatically restored
/sbin/service iptables save

# enable the firewall
chkconfig iptables on

# and reboot machine
reboot

Check if HTTPS is Transparently Filtered

Please note, in order for HTTPS filtering to function correctly, we must install the proxy certificate from /etc/opt/quintolabs/qlproxy/myca.der into Trusted Root Certification on all workstations in our network. The following screenshots show that HTTPS requests were decrypted and filtered transparently.

Browsing to Google and searching for an adult term (e.g. NSFW), we get the HTTPS request filtered and blocked transparently.

Resume

We now have the default gateway in our network capable of transparently filtering HTTP and HTTPS traffic. All workstations in our network trust the root certificate from proxy, and thus get their HTTPS request decrypted and filtered. Browsing environment in our network became much safer.

Links

Subscribe to Xmodulo

Do you want to receive Linux FAQs, detailed tutorials and tips published at Xmodulo? Enter your email address below, and we will deliver our Linux posts straight to your email box, for free. Delivery powered by Google Feedburner.

The following two tabs change content below.
Diladele Web Safety for Squid Proxy Server is an ICAP server that integrates with existing Squid proxy server and provides rich content and web filtering functionality to sanitize Internet traffic passing into internal home/enterprise network. It may be used to block illegal or potentially malicious file downloads, remove annoying advertisements, prevent access to various categories of the web sites and block resources with explicit content.

Latest posts by Rafael Akchurin (see all)

9 thoughts on “How to set up a transparent HTTPS filtering proxy on CentOS

  1. Don't forget to create your own ssl certificate and install it on all your LAN hosts.
    If you have problem with your generated certificates try this tip: Rebuild ssldb
    sudo service squid3 stop
    sudo rm -R /var/spool/squid3_ssldb
    sudo mkdir /var/spool/squid3_ssldb
    sudo /bin/ssl_crtd -c -s /var/spool/squid3_ssldb
    sudo chown -R proxy:proxy /var/spool/squid3_ssldb
    sudo /usr/sbin/squid3 -k parse

  2. Is this just not a MITM attack?
    You will be giving "Diladele Web Safety" all your employee's web banking details too, along with all passwords, etc. etc.

    • The SSL bump is only appropriate in cases when you are the sole owner of a network (like home for example). In any case doing SSL bump may be fully illegal in some countries so consult your lawyer first when planning to implement it. This article describes only technical means of performing HTTPS filtering which may be required for some deployments (think schools, libraries etc). More info can be found on the Squid's web site (search for Squid SSL Bump Wiki).

    • MITM is definitely illegal in plenty of countries especially given that you will likely pick up banking details and plenty of passwords. The best you can legally do for HTTPS is to just block based on destination.

  3. Why should we trust you to know security setting appropriately if your page is littered with uncorrected HTML / XML entities?

    This line:
    echo "This script must be run as root" 1>&2

    Won't work very well if just blindly copied and pasted (as you seem to be suggesting users do)

    • Probably some "safe" parsing of wordpress on this site - there is downloadable archive attached to this article - it contains all the scripts for just running - no need to copy paste.

  4. Hi everybody,

    The post is great. But i've got something that's not OK.
    Transparent HTTP is OK but when it is HTTPS, squid crashed.
    i've got that logs:

    2014/04/27 11:55:26 kid1| helperOpenServers: Starting 5/32 'ssl_crtd' processes
    2014/04/27 11:55:26 kid1| WARNING: no_suid: setuid(0): (1) Operation not permitted
    2014/04/27 11:55:26 kid1| WARNING: no_suid: setuid(0): (1) Operation not permitted
    2014/04/27 11:55:26 kid1| WARNING: no_suid: setuid(0): (1) Operation not permitted
    2014/04/27 11:55:26 kid1| WARNING: no_suid: setuid(0): (1) Operation not permitted
    2014/04/27 11:55:26 kid1| WARNING: no_suid: setuid(0): (1) Operation not permitted
    2014/04/27 11:55:26 kid1| Logfile: opening log daemon:/opt/squid3/var/logs/access.log
    2014/04/27 11:55:26 kid1| Logfile Daemon: opening log /opt/squid3/var/logs/access.log
    2014/04/27 11:55:26 kid1| WARNING: no_suid: setuid(0): (1) Operation not permitted
    2014/04/27 11:55:26 kid1| Store logging disabled
    2014/04/27 11:55:26 kid1| Swap maxSize 0 + 262144 KB, estimated 20164 objects
    2014/04/27 11:55:26 kid1| Target number of buckets: 1008
    2014/04/27 11:55:26 kid1| Using 8192 Store buckets
    2014/04/27 11:55:26 kid1| Max Mem size: 262144 KB
    2014/04/27 11:55:26 kid1| Max Swap size: 0 KB
    2014/04/27 11:55:26 kid1| Using Least Load store dir selection
    2014/04/27 11:55:26 kid1| Set Current Directory to /opt/squid3/var/cache/squid
    2014/04/27 11:55:26 kid1| Finished loading MIME types and icons.
    2014/04/27 11:55:26 kid1| HTCP Disabled.
    2014/04/27 11:55:26 kid1| Squid plugin modules loaded: 0
    2014/04/27 11:55:26 kid1| Adaptation support is on
    2014/04/27 11:55:26 kid1| Accepting NAT intercepted HTTP Socket connections at local=192.168.1.254:3126 remote=[::] FD 21 flags=41
    2014/04/27 11:55:26 kid1| Accepting SSL bumped HTTP Socket connections at local=192.168.1.254:3128 remote=[::] FD 22 flags=9
    2014/04/27 11:55:26 kid1| Accepting NAT intercepted SSL bumped HTTPS Socket connections at local=192.168.1.254:3127 remote=[::] FD 23 flags=41
    2014/04/27 11:55:27 kid1| storeLateRelease: released 0 objects
    2014/04/27 11:55:28 kid1| assertion failed: comm.cc:769: "Comm::IsConnOpen(conn)"

    what am I doing wrong?
    I've got only one interface on my proxy.

Leave a comment

Your email address will not be published. Required fields are marked *

Current ye@r *