CernVM-FS Configuration Examples

Installation

Please follow the instructions in the quickstart chapter of the CernVM-FS technical report.

Select Repositories

Let's assume you want to support ATLAS and LHCb software. Create or edit /etc/cvmfs/default.local and set

CVMFS_REPOSITORIES=atlas.cern.ch,atlas-condb.cern.ch,lhcb.cern.ch

The following table lists the available repositories and the repositories they rely on.
Note: For Grid sites deploying the Grid UI by other means, please ignore the CernVM-FS grid.cern.ch dependency.

Change Cache Quota and Location

Let's assume we want to have a 10GB cache for every repository except for ATLAS, for which we decide to have 20GB of cache space.  All the caches should be located in the existing directory /var/scratch/cvmfs.

If you're about to change a current cache location or if you decrease the quota, run /sbin/service cvmfs restartclean first in order to wipe out the current location. Create or edit /etc/cvmfs/default.local and set

CVMFS_CACHE_BASE=/var/scratch/cvmfs
CVMFS_QUOTA_LIMIT=10000

In order to overwrite the default 10GB limitation for ATLAS, create or edit /etc/cvmfs/config.d/atlas.cern.ch.local and set

CVMFS_QUOTA_LIMIT=20000

Make the changes effective by /sbin/service cvmfs restart.

In practice, 10-20GB should be more than enough for running jobs.  Please note that the cache quota is per repository: if you mount CMS and ATLAS software, both repositories may cache up to $CVMFS_QUOTA_LIMIT Megabytes. If you put the cache on it's own partition, set the soft limit to not more than 90% of the partition size.

Note: Please keep in mind that the cache location has to be on local storage.  Please take care that tmpwatch doesn't clean the cvmfs cache directory behind your back.

Note: A quota limit should be set.  The cvmfs process will abort if the partition containing the cache runs full.

Specify Local Site Proxies

Let's assume you have two Squid proxies (squid1 and squid2), both listening on port 3128, which you want to configure load-balanced and with fail-over in case one of them is offline.  Create or edit /etc/cvmfs/default.local and set

CVMFS_HTTP_PROXY="http://squid1:3128|http://squid2:3128"

followed by /sbin/service cvmfs reload.

Use the CernVM-FS Stratum 1 Mirror Servers

Create or edit /etc/cvmfs/domain.d/cern.ch.local (not /etc/cvmfs/default.local) and set

CVMFS_SERVER_URL="http://cvmfs-stratum-one.cern.ch:8000/opt/@org@;http://cernvmfs.gridpp.rl.ac.uk:8000/opt/@org@;http://cvmfs.racf.bnl.gov:8000/opt/@org@"

followed by /sbin/service cvmfs reload.  Switch the URLs in order to try the closest servers first, e.g. choose the RAL URL first if you are in the U.K.  In case of failure, CernVM-FS will fail-over to the next available host.  Specify the hosts using standard HTTP port 80 (like http://cvmfs-stratum-one.cern.ch/opt/@org@) if port 8000 is blocked on your site.

Setup a Local Squid Server

Squid is very powerful and has lots of configuration and tuning options. For CernVM- FS we require only the very basic static content caching. Starting from a standard Scientific Linux 5 Squid, all we have to do is to adjust the cache size. If you're using ACLs, don't forget to add ACLs allow rules for the Stratum 1 servers. Browse through your /etc/squid/squid.conf and make sure the following lines appear accordingly

collapsed_forwarding on
max_filedesc 8192
maximum_object_size 4096 MB
# 4 GB memory cache
cache_mem 4096 MB
maximum_object_size_in_memory 32 KB
# 50 GB disk cache
cache_dir ufs /var/spool/squid 50000 16 256

Check your Squid configuration with squid -k parse. Create the hard disk cache area with squid -z. In order to make the increased number of file descriptors effective for Squid, execute ulimit -n 8192 prior to starting the Squid service.