What is a secure web gateway?
A secure web gateway in it’s purest sense allows for the enforcement of policies and provides real-time threat protection for clients browsing the web.
Why should you the reader consider a secure web gateway?
There are a number of benefits a secure web gateway can provide, some of my favourites are the following:
- Real-time anti-virus scanning
- Corporate policy enforcement (Prevent users from viewing adult content, watching netflix, etc.)
- Logging of web activity
This tutorial will walk you through the creation of an open source secure web gateway using e2guardian, clamav, and debian 9 stretch. If any part of this tutorial is unclear or you are unable to follow the instructions please let me know in the comments and I will do my best to help. This tutorial is for someone with Linux experience, and a high level understanding of how the TCP, SSL, HTTP, and DNS protocols work.
1. Start by installing debian 9 to your preferences and configuring a static IP address. Our lab network already has a gateway so I used an IPv4 address of 192.168.8.2/24 on my debian host and left the gateway at 192.168.8.1.
2. Download the e2guardian deb package with the command below:
wget https://gitlab.com/fredbcode/e2guardian/raw/packages/v5.3.2/e2debian_V5.3.2_20190315.deb
3. Before we install the deb package we need to install some dependencies. Run the commands below to install them:
sudo apt update sudo apt install -y libevent-core-2.0-5 libevent-pthreads-2.0-5 libtommath1
4. Once the above commands are finished we can run the command below to install e2guardian:
sudo dpkg -i e2debian_V5.3.2_20190315.deb
5. Now to inspect HTTPS traffic there is some additional work we need to do. Start by running the commands below to create some additional directories we need:
sudo mkdir /etc/e2guardian/generatedcerts sudo mkdir /etc/e2guardian/ssl
6. Change you working directory with the command below:
sudo cd /etc/e2guardian/ssl
7. Run the command below to create a private key for use with the CA certificate we are going to create:
sudo openssl genrsa 4096 > private_e2cakey.pem
8. Use the command below to generate the CA certificate:
sudo openssl req -new -x509 -days 3650 -key private_e2cakey.pem -out e2_rootCA.pem
9. Use the command below if you would like to be able to import the CA into web browsers:
sudo openssl x509 -in e2_rootCA.pem -outform DER -out e2_rootCA.der
10. Lastly we need to generate one more private key to use for auto generated certificates:
sudo openssl genrsa 4096 > e2_cert_key.pem
11. Tighten up the permissions on the certificates we just created with the commands below:
sudo chown -R e2guardian:e2guardian /etc/e2guardian/generatedcerts sudo chown -R e2guardian:e2guardian /etc/e2guardian/ssl sudo chmod 600 /etc/e2guardian/ssl/*
12. Open e2guardian’s primary configuration file (/etc/e2guardian/e2guardian.conf) with your favourite text editor or use the following sed commands (Much faster and easier) to make the edits for you. Enable SSL support with the command below:
sudo sed -i "/enablessl =*/c\enablessl = on" /etc/e2guardian/e2guardian.conf
13. Run the commands below to tell e2guardian where its certificates and privates keys are located:
sudo sed -i "/cacertificatepath =/c\cacertificatepath = \'/etc/e2guardian/ssl/e2_rootCA.pem\'" /etc/e2guardian/e2guardian.conf sudo sed -i "/caprivatekeypath =/c\caprivatekeypath = \'/etc/e2guardian/ssl/private_e2cakey.pem\'" /etc/e2guardian/e2guardian.conf sudo sed -i "/certprivatekeypath =/c\certprivatekeypath = \'/etc/e2guardian/ssl/e2_cert_key.pem\'" /etc/e2guardian/e2guardian.conf
14. Tell e2guardian where to store auto generated certificates with the command below:
sudo sed -i "/generatedcertpath =/c\generatedcertpath = \'/etc/e2guardian/generatedcerts\'" /etc/e2guardian/e2guardian.conf
15. Now we haven’t installed it yet, but we will be using clamav for content scanning. Run the command below to configure the content scanner setting:
sudo sed -i "/clamdscan.conf/c\contentscanner = \'\/etc\/e2guardian\/contentscanners\/clamdscan.conf\'" /etc/e2guardian/e2guardian.conf
16. Now we need to modify the default group configuration file (/etc/e2guardian/e2guardianf1.conf ). Run the command below to instruct e2guardian to MITM (Man in the middle) SSL connections:
sudo sed -i "/sslmitm = off/c\sslmitm = on" /etc/e2guardian/e2guardianf1.conf
17. Modify the clamav content scanner configuration file (/etc/e2guardian/contentscanners/clamdscan.conf) and tell e2guardian where to find the clamav unix socket. Run the command below to accomplish this:
sudo sed -i "/clamdudsfile =/c\clamdudsfile = \'/var/run/clamav/clamd.ctl\'" /etc/e2guardian/contentscanners/clamdscan.conf
18. To make distributing our CA certificate less of a pain we will stand up a quick nginx server to help us out. Run the commands below to accomplish this:
sudo apt-get install -y nginx sudo rm /var/www/html/index.nginx-debian.html sudo sed -i '/listen 80 default_server;/c\\tlisten 8081 default_server;' /etc/nginx/sites-enabled/default sudo sed -i '/listen [::]:80 default_server;/c\\t# IPv6 config removed' /etc/nginx/sites-enabled/default sudo cp /etc/e2guardian/ssl/e2_rootCA.pem /var/www/html/e2_rootCA.pem sudo chmod +r /var/www/html/e2_rootCA.pem sudo service nginx start sudo systemctl enable nginx
🛈 If you have run the above commands your CA certificate will now be available to clients to download via <your_server_ip>:8081/e2_rootCA.pem. If you would like the der format or a pfx format to shared easily you can drop them in the /var/www/html directory. Just make sure you remember to set permissions so nginx can serve them to clients.
19. Run the command below to install the clamav and freshclam (keeps clamav signatures up to date):
sudo apt-get install -y clamav clamav-base clamav-daemon clamav-docs clamav-freshclam clamdscan
20. Use the command below to give clamav the ability to scan files created by e2guardian:
sudo usermod clamav -G e2guardian
21. Run the commands below to start the clamav and freshclam services and ensure they start at boot:
sudo service clamav-daemon start sudo service clamav-freshclam start sudo systemctl enable clamav-daemon sudo systemctl enable clamav-freshclam
22. Enable routing in the linux kernel with the command below:
sudo sysctl -w net.ipv4.ip_forward=1 sudo echo 1 > /proc/sys/net/ipv4/ip_forward
23. Now we need to use iptables to get everything working properly. Start by installing the iptables-persistent package with the command below:
sudo apt install -y iptables-persistent
24. Add the required iptables rules with the commands below:
sudo iptables -t nat -A PREROUTING -i ens18 -p tcp --dport 443 -j REDIRECT --to-port 8443 sudo iptables -t nat -A PREROUTING -i ens18 -p tcp --dport 80 -j REDIRECT --to-port 8080
25. Finally start the e2guardian service and configure it start on boot with the commands below:
sudo service e2guardian start sudo systemctl enable e2guardian
🛈 Once those services are started you should have a fully functional secure web gateway. Clients must either be configured to use this server as their gateway or in some cases you can selectively route HTTP and HTTPS traffic to this gateway. The commands below are an example of how to configure a debian 9 client to selectively route HTTP and HTTPS traffic to the e2guardian server.
sudo echo 200 proxy-route >> /etc/iproute2/rt_tables sudo ip route add default via 192.168.8.2 dev ens18 table proxy-route sudo ip rule add fwmark 0x1 table proxy-route sudo iptables -A OUTPUT -t mangle -o ens18 -p tcp --dport 443 -j MARK --set-mark 1 sudo iptables -A OUTPUT -t mangle -o ens18 -p tcp --dport 80 -j MARK --set-mark 1
26. Now some traffic we are not going to want to touch. This may be users browsing to their banking web site, video streaming like Netflix, and so on. To get around this we can use iptables, ipset, and a little bit of python (or any other scripting language you like). Start by running the command below to install ipset:
sudo apt install ipset -y
27. Create an ipset to use in a future iptables rule by running a command similar to the one below:
sudo ipset create test hash:ip
28. Use an iptables rule similar to the command below:
iptables -t nat -I PREROUTING -i ens18 -p tcp -m set --match-set test dst -j ACCEPT
🛈 Finally, here is some example code that will read a list of domains and add them to the ipset.
#!/usr/bin/python3
import socket
import os
listname = 'whitelist.txt'
# Get domains for whitelist
with open(listname) as list:
content = list.readlines()
addrs = []
for item in content:
print("Loaded domain to whitelist: {}".format(item))
query = socket.gethostbyname(item.strip())
print("IP Address is {}".format(query))
addrs.append(query)
for ip in addrs:
print("Adding IP: {}".format(ip))
os.system("ipset add test {}".format(ip))
Run that script or a similar script via a cron job and make sure that DNS server used by both the e2guardian server and clients is a local resolver. Having clients query a local DNS server will help overcome some of the challenges introduced by global server load balancing and ensure everything works as expected.
You will find this configuration works, but is likely too strict for your environment. Luckily it is very easy to fine tune the configuration both in the e2guardian group configuration /etc/e2guardian/e2guardianf1.conf and via editing the whitelists / blacklists used in the /etc/e2guardian/lists/ directory.
So what is this server actually doing?
For HTTP traffic, – unless bypassed by the ipset – the traffic is transparently redirected from port 80 to port 8080. The server looks at the content of the packet to figure out where it is going, performs security / policy checks and then proxies the request.
HTTPS traffic is a bit more complex. HTTPS traffic encrypts most of the data so the server cannot treat it the same as HTTP traffic. This is why the server must be the man-in-the-middle for SSL encrypted traffic. Unless bypassed by the ipset HTTPS traffic is transparently redirected from port 443 to port 8443. The server then looks at the destination IP address on packet to figure out where it is supposed to go, generates a certificate for that destination, signs the certificate with its CA certificate, and presents the certificate to the client. If the client accepts the certificate then the first half is accomplished and the server can finish setting up the SSL connection with the client. Once the HTTPS data is decrypted the server performs security / policy checks, and then makes the request to the destination website via HTTPS on behalf of the client.
How do I get better performance?
To get better performance you may want to reference the e2guardian performance wiki page. Also there is a significant performance hit when doing MITM on HTTPs due to the decryption and re-encryption. For this reason you may want to accept the risk of not performing checks on SSL traffic if performance is critical.