How to analyze Squid logs with SARG log analyzer on CentOS

Squid provides many useful proxy features such as caching, access control, anti-virus, load balancing, authentication, rate-limiting, and so on. However, analyzing a raw Squid log file is not straightforward. For example, how could you analyze the time stamps and the number of hits in the following Squid log?

1404788984.429   1162 TCP_MISS/302 436 GET - DIRECT/ text/html
1404788985.046  12416 TCP_MISS/200 4169 CONNECT - DIRECT/ -
1404788986.124    174 TCP_MISS/200 955 POST - DIRECT/ application/ocsp-response
1404788989.738    342 TCP_MISS/200 3890 CONNECT - DIRECT/ -
1404788989.757    226 TCP_MISS/200 942 POST - DIRECT/ application/ocsp-response
1404788990.839   3939 TCP_MISS/200 78944 CONNECT - DIRECT/ -
1404788990.846   2148 TCP_MISS/200 118947 CONNECT - DIRECT/ -
1404788990.849   2151 TCP_MISS/200 76809 CONNECT - DIRECT/ -
1404788991.140    611 TCP_MISS/200 110073 CONNECT - DIRECT/ –

That is when SARG (or Squid Analysis Report Generator) comes in handy. In a nutshell, SARG is a web based tool that creates reports from Squid logs. SARG provides system admins with an easy-to-understand view of network traffic handled by Squid. What's more, it is very easy to set up and maintain.

In the following tutorial, we show how to set up SARG on CentOS platform.

We start the process by installing necessary dependencies using yum.

# yum install gcc make wget httpd crond

Necessary services are started and loaded at startup.

CentOS/RHEL 6:

# service httpd start
# service crond start
# chkconfig httpd on
# chkconfig crond on

CentOS/RHEL 7:

# systemctl start httpd.service
# systemctl start crond.service
# systemctl enable httpd.service
# systemctl enable crond.service

Now we download and extract SARG.

# wget
# tar zxvf sarg-2.3.8.tar.gz
# cd sarg-2.3.8

NOTE: For 64-bit Linux, the source code in log.c needs to be patched as follows.

< 			if (fprintf(ufile->file, "%s\t%s\t%s\t%s\t%"PRIi64"\t%s\t%ld\t%s\n",dia,hora,ip,url,nbytes,code,elap_time,smartfilter)<=0) {
> 			if (fprintf(ufile->file, "%s\t%s\t%s\t%s\t%"PRIi64"\t%s\t%ld\t%s\n",dia,hora,ip,url,(int64_t)nbytes,code,elap_time,smartfilter)<=0) {
< 				fprintf(fp_log, "%s\t%s\t%s\t%s\t%s\t%"PRIi64"\t%s\t%ld\t%s\n",dia,hora,user,ip,url,nbytes,code,elap_time,smartfilter);
> 				fprintf(fp_log, "%s\t%s\t%s\t%s\t%s\t%"PRIi64"\t%s\t%ld\t%s\n",dia,hora,user,ip,url,(int64_t)nbytes,code,elap_time,smartfilter);
< 				printf("LEN=\t%"PRIi64"\n",nbytes);
> 				printf("LEN=\t%"PRIi64"\n",(int64_t)nbytes);

Go ahead and build/install SARG as follows.

# ./configure
# make
# make install

After SARG is installed, the configuration file can be modified to match your requirements. The following is one example of SARG configuration.

# vim /usr/local/etc/sarg.conf
access_log /var/log/squid/access.log
temporary_dir /tmp
output_dir /var/www/html/squid-reports
date_format e     ## We use Europian DD-MM-YYYY format here ##
## we don’t want multiple reports for single day/week/month ##
overwrite_report yes

Now it's time for a test run. We run sarg command in debug mode to find whether there is any error in the configuration.

# sarg -x

If all goes well, sarg should analyze Squid logs, and create reports in /var/www/html/squid-reports. The reports should be visible in a web browser using the address http://<server-IP>/squid-reports/

SARG can be used to create daily, weekly and monthly reports. Time range can be specified using the "-d" parameter with possible values in the form of day-n, week-n or month-n, where n is the number of days/weeks/months to jump backward. For example, with week-1, SARG will generate a report for the previous week. With day-2, SARG will prepare reports for the previous two days.

As a demonstration, we will prepare a cron job to run SARG daily.

# vim /etc/cron.daily/sarg
/usr/local/bin/sarg -d day-1

The file needs a execution permission.

# chmod 755 /usr/local/bin/sarg

Now SARG should prepare daily reports about Squid-managed traffic. These reports can easily be accessed from the SARG web interface.

To sum up, SARG is a web based tool that analyzes Squid logs and presents the analysis in an informative way. System admins can leverage SARG to monitor what sites are being accessed, and to keep track of top visited sites and top users. This tutorial covers a working configuration for SARG. You can customize the configuration even further to match your requirements.

Hope this helps.­­­­

Subscribe to Xmodulo

Do you want to receive Linux FAQs, detailed tutorials and tips published at Xmodulo? Enter your email address below, and we will deliver our Linux posts straight to your email box, for free. Delivery powered by Google Feedburner.

Support Xmodulo

Did you find this tutorial helpful? Then please be generous and support Xmodulo!

The following two tabs change content below.
Sarmed Rahman is an IT professional based in Australia. He writes tutorial articles on technology every now and then from a belief that knowledge grows through sharing. During his free time, he loves gaming and spending time with his friends.

Leave a comment

Your email address will not be published. Required fields are marked *