squidguard and videocache working together
(squid with many redirectors)

Introduction

My school network have a squid with squidguard in a FreeBSD box for delivering contents. Our activity implies a lot of video delivering from the Internet. The same video is delivered many times, simultaneously or not. Our connectivity to the Internet is not very good, for geographical and economical reasons.

Squid doesn't cach videos from sites like www.youtube.com:

wiki.squid-cache.org/ConfigExamples/DynamicContent/YouTube

Videocache (cachevideos.com) is a redirector for squid that permits to catch videos from www.youtube.com and other similar sites.

But squid can't work with more than one redirector, unless you chain them. Fortunately, addzapper (adzapper.sourceforge.net) has a tool called zapchain to chain squid redirectors.

In this tutorial I will explain how I integrated videocache to my squid + squidguard FreeBSD box.

Initial scenario

My working squid box has the following relevant software:

Python installation from FreeBSD ports

proxy# cd /usr/ports
proxy# make update
proxy# pkgdb -F

proxy# cd lang/python26
proxy# make install

Confirm the default options at screen options for this port.

proxy# pkgdb -F


AddZapper installation from FreeBSD ports

proxy# cd usr/ports/www/adzap
proxy# make install
proxy# pkgdb -F

urlgrabber installation from source

proxy# cd /home/myuser
proxy# mkdir videocache
proxy# cd videocache
proxy# fetch http://linux.duke.edu/projects/urlgrabber/download/urlgrabber-3.1.0.tar.gz
proxy# tar -xzf urlgrabber-3.1.0.tar.gz
proxy# chown -R root:wheel urlgrabber-3.1.0
proxy# cd urlgrabber-3.1.0
proxy# python2.6 setup.py install
proxy# cd ..

iniparse installation from source

proxy# fetch http://iniparse.googlecode.com/files/iniparse-0.2.4.tar.gz
proxy# tar -xzf iniparse-0.2.4.tar.gz
proxy# chown -R root:wheel iniparse-0.2.4
proxy# cd iniparse-0.2.4
proxy# python2.6 setup.py install
proxy# cd ..

videocache installation from source

proxy# fetch http://cachevideos.com/sites/default/files/pub/videocache/videocache-1.6.tar.gz
proxy# tar -xzf videocache-1.6.tar.gz
proxy# chown -R root:wheel videocache-1.6
proxy# cd videocache-1.6
proxy# python2.6 setup.py install


videocache configuration

Backup and edit the initial configuration:

proxy# cd /etc
proxy# cp videocache.conf videocache.conf.dist
proxy# vi videocache.conf

Be careful with the following options:

# Better to use machine name than IP address.
# If you use in-addr for squidGuard, using IP address will block serving video from cache.
# Don't use localhost or 127.0.0.1 as cache_host to indicate your webserver for cache.
cache_host = proxy.domain.tld

# Port 3128 is the default port for squid, but many people (me, for example) use 8080.
# So, configure according your squid proxy server.
proxy = http://127.0.0.1:8080/

Read the file for the rest of options and configure it, if you need.

Apache configuration

videocache installation creates httpd folder at /etc with webserver options. These options make visible /var/log/spool/videocache for the webserver. They have to be integrated to /usr/local/etc/apache22/httpd.conf and restart the webserver.

I preferred another way:

proxy# cd /etc
proxy# rm -R httpd
proxy# cd /usr/local/www/apache22/data
proxy# ln -s /var/spool/videocache videocache

Now the cache folders should be visible in a workstation at http://proxy.domain.tld/videocache (you don't need to restart the webserver).

If configuring videocache you decided not to cache some of the sites you can erase their folders at /var/spool/videocache:

proxy# cd /var/spool/videocache
proxy# rm -R metacafe

Squid reconfiguration

Squid is complex to configure. The two more important things to put squidguard and videocache working together are:

• Ensure that localhost (127.0.0.1/32) can use the proxy.

• Chain squidguard and videocache redirectors using zapchain, one of the AddZapper scripts.

I only needed the following changes:

proxy# cd /usr/local/etc/squid
proxy# cp squid.conf squid.conf-yyyy-mm-dd
proxy# vi squid.conf

# videocache (127.0.0.1/32) needs squid access.
acl our_networks src 127.0.0.1/32 192.168.XXX.0/24 192.168.YYY.0/24

# Using zapchain to chain squid redirectors.
url_rewrite_program /usr/local/libexec/zapchain "/usr/local/bin/squidGuard -c /usr/local/etc/squid/squidGuard.conf" "/usr/local/bin/python2.6 /usr/share/videocache/videocache.py"

proxy# squid -k reconfigure

Testing

At any workstation using proxy.domain.tld as proxy server ...

If the content is not blocked by squidGuard (first redirector chained) ...

Looking a video (for example from www.youtube.com) should cause to write temporally the video to http://proxy.domain.tld/videocache/temp and finally store it at http://proxy.domain.tld/videocache/youtube for newer viewing.

It is useful to browse the log of videocache. For example:

proxy# cat /var/log/videocache/videocache.log | grep YOUTUBE | more

For more information please go to the videocache website, cachevideos.com.