Linux-Tip.net - Howto install a content filtering and virus checking proxy (Part I)

Home

Howto install a content filtering and virus checking proxy (Part I)

Wednesday, 11 April 2007

All Internet content carries a risk. Content Filtering helps companies and home users to stop unwanted data. It is used as part of Internet Firewalls or Proxy Servers to screen the content of all incoming Internet traffic. Content filtering usually works by specifying character strings that, if matched, indicate undesirable content that is to be screened out. Content is typically screened for pornographic content and sometimes also for violence- or hate-oriented content.
To be effective you need a database or list of domains and URLs that should be blocked. Building and maintaining your own blacklist would require a huge investment in time. In the first part (Part I) of the workshop we will use a downloaded list which you can later update on the regular bases.
Part II of the workshop will explain how to add a virus scan engine to the existing configuration using ClamAV. StumbleUpon Toolbar

Stumble It!

The workshop setup could look like this:

Step 1 - Proxy Server - Basic Installation

We will use OpenSuse 10.2 and configure Squid, SquidGuard an ClamAV step by step. We really do not need a system with all bells and whistles, so use your Suse installation DVD or CD and install a very basic system without KDE or Gnome. As you can see in the workshop configuration above, our proxy server is acting as a router. For that reason we have to configure two network cards.
The installation should not be a problem for you. In the "Desktop Selection" part choose "Other" and click "Select". In our case "Text Mode" should work.

Step 2 - Network Configuration and Routing

After installing and rebooting the server you should be able to login as root and to configure the two network cards according to our network diagram above.

As already mentioned, the proxy is acting as a router. To get this part running, we will create the following small "naton" script and store it under /etc/mynat/:

#!/bin/bash
# /etc/mynat/naton
# switch on NAT
#

# Shell "debug" on
set -x

# define variables
ETHOUT=eth0
ETHIN=eth1

IP_EXT=82.12.20.42

# flash all firewall and NAT rules

iptables -t filter -F
iptables -t nat -F
iptables -t mangle -F
iptables -t raw -F

# delete user defined chains
iptables -X

# define default policies
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT

# allow all locally
iptables -t filter -A INPUT -i lo -j ACCEPT
iptables -t filter -A OUTPUT -o lo -j ACCEPT

# activate FORWADING
echo 1 > /proc/sys/net/ipv4/ip_forward

# activate SNAT
iptables -t nat -A POSTROUTING -o $ETHOUT -j SNAT --to $IP_EXT

Make sure to run this script after rebooting the server. You can later use this file to install some kind of firewall or to block unwanted traffic to and from your router.

Step 3 - Installing Squid and SquidGuard

Please use Yast again to install squid. Additionally you will need the following packages. This is a good time to install them:

* gcc
* make
* Berkeley DB

Unfortunately SquidGuard is not part of the distribution. You can download and install it using the familiar "configure, make, make install" routine. As an alternative, you should be able to download the rpm package from here:

http://rpm.pbone.net/

Install it like this:

rpm -i squidGuard-1.2.0-408.i586.rpm

After installing squidGuard using the rpm command, we should be able to configure squid and squidGuard. The following files are important for us:

/usr/sbin/squid -   squid binary
/etc/squid/squid.conf -   squid configuration file
/usr/sbin/squidGuard -   squidGard binary
/etc/squidGuard.conf -   squidGuard configuration file
/var/log/squidGuard/squidGuard.log   -   squidGuard log file
/var/lib/squidGuard/db -   directory for blacklist files
/var/lib/squidGuard/whitelist   -   file with whitelist domains
/var/log/squidGuard/blocked.log    -   log file for blocked content

Please configure squid for your needs using squid.conf. You can try to use your basic configuration and add additional requirements. If you have problems configuring it, please use the following web sites:

http://wiki.squid-cache.org/SquidFaq/ConfiguringSquid
http://www.squid-cache.org/

A basis squid.conf file could look like this:

visible_hostname your-server-name
acl our_networks src 192.168.1.0/24
http_access allow our_networks
http_access deny all

Squid has the ability to rewrite requested URLs. Implemented as an external process, Squid can be configured to pass every incoming URL through a redirector process that returns either a new URL or a blank line to indicate no change. To use this possibility, we have to add the following 3 lines to connect squidGuard to Squid. Please add it to /etc/squid/squid.conf

redirect_program /usr/sbin/squidGuard -c /etc/squidguard.conf
redirect_children 8
redirector_bypass on

Step 4 - Configuring black- and whitelists

In the next step we have to add blacklist with domais and URL you would like to block. As you can imagine, writing and adapting your own list would be very time-consuming work. Fortunately folks did already the work for us. We can download the blacklists and maintain it for our purpose. The Shalla list consists hundreds of thousands entries categorized by different subjects. The list is free for non-commercial use, but even available for commercial use without costs.

Download the list here:

http://squidguard.shalla.de/shallalist.html

Simply unpack the file and copy all content to the squidGuard directory:

tar xzf shallalist.tar.gz
mkdir /var/lib/squidGuard/db
cd BL
cp -R * /var/lib/squidGuard/db

Please take a look to the blacklist directory you have just created. You will find 29 subdirectories with plain text files for domains and URLs. To keep it simple, we will just use the categories "spyware" and "porn", but you can later step by step use more of this very good stuff until it meets your requirements. Additionally you should create our own blacklist with domains and URL not part of the provided blacklists. In this case create your own subdirectory "my_blacklist" and handle it accordingly.

As you can see at the configuration file, you just have to add the so-called "dest" block. It defines the domain list, url list and the location for the log file. In the "Access control lists" section we are redirecting all blocked traffic to a local apache web server running an index file with information for the users.

A basic sqiudGuard.conf file could look like this:

# Squid Guard config file
#-------------------------
logdir /var/log/squidGuard
dbhome /var/lib/squidGuard/db

# DESTINATIONS
dest spy {
        domainlist spyware/domains
        urllist spyware/urls
        log /var/log/squidGuard/blocked.log
}

dest porn {
        domainlist porn/domains
        urllist porn/urls
        log /var/log/squidGuard/blocked.log
}

# ACCESS CONTROL LISTS
acl {
        default {
                pass !porn !spy !in-addr all
                redirect http://192.168.1.1
        }
}

Unfortunately, squidGuard can not use the plain text files with domains and URLs. For that reason we need to convert it to Berkeley DB format. No worry, the following command will look in the squidGuard configuration file and will only convert our two categories "spyware" and "porn" to the necessary format. You will later find the new files with extension ".db" in the same subdirectories:

/usr/sbin/squidGuard -C all

If everything worked fine, you should see the following files:

/var/lib/squidGuard/db/porn/domains.db
/var/lib/squidGuard/db/porn/urls.db
/var/lib/squidGuard/db/spyware/domains.db
/var/lib/squidGuard/db/spyware/urls.db

Important Remark: Unfortunately squidGuard didn't work together with the blacklists using default file permissions. I had to change the blacklists permissions and ownership to get it running like this:

cd /var/lib/squidGuard/db
chown -R squid:root *
chmod -R 777 *

Additionally we have to create a list with domains that are basically excluded from the content filtering. Those Web pages can be added to the appropriate URL Whitelist in order to grant access. There are a couple of approaches possible, but we will configure it by adding the following lines to the squid.conf file.

acl white dstdomain "/var/lib/squidGuard/whitelist"
redirector_access deny white

The option "acl" must be defined before it is used in the redirector_access statement. So please put in the first part of the squid.conf file.
In the whitelist file you can later add domain names line by line. It could look like this:

www.domain.net
.domain.net
.net

To inform the user about content blocked by your proxy, you can run a web server.
Use YAST to install apache2.

The basis setup will work. You just have to create an index.html file with the following content in /srv/www/htdocs:

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1" http-equiv="content-type">
<title>Access Denied</title>
</head>

<body>

<big><big><big><big>Content
Blocked 

by
SquidGuard V 1.2</big></big></big></big>
</body>
</html>