E-Planet 2013 Visit at UERJ
This page provides a summary of the visit program of CernVM at UERJ in the framework of the EU E-Planet program. The visit took place in Oktober/November 2013. Our main contacts at UERJ were Alberto Santoro and his group of Computing Scientists/Engineers, led by José Afonso Sanches and Eduardo Revoredo.
Program: HEP Data Processing in the Cloud
The program of lectures and exercises is focused on the effective use of cloud resources for processing HEP data. From a physicist's perspective, this includes instantiating, contextualizing, and coordinating a set of virtual machines (a "virtual analysis facility") that processes a given data set. For a virtual organization and the infrastructure service provider, hands-on sessions on CernVM, and CernVM-FS have are performed. The program builds on the previous work done together with CernVM project member Artem Harutyunyan.
Synopsis: Recap of grid constituents such as grid layer model, storage element, file catalog, task queue, scheduler, compute element, monitoring. Discussion of the cloud computing model such as types of clouds, IaaS cloud components, virtual appliances. Based on the tools and frameworks developed within WLCG, discussion of how virtualization and cloud computing can simplify distributed computing and harness new resources. Difficulties and challenges it will touch: virtual appliance management, contextualization interfaces, status of virtualization of co-processors (GPU, MIC) and low-power processors (e.g. Atom, ARM)
Synopsis: Details of the CernVM virtual machine internals and the build process. Being a very lightweight image, CernVM can eliminate the need for a virtual machine image store. The use of the CernVM-FS versioning features facilitates long-term data preservation since a previously defined data processing environment can be re-created.
Synopsis: Overview of the current status of the Test4Theory project. Overview of planned projects: LHC simulation production jobs, volunteer computing for Blue Brain. Running desktop grids and volunteer clouds detached from the BOINC platform.
Synopsis: Details of the use of content-addressable storage inside CernVM-FS. Details of versioning, replication, and storage management in CernVM-FS. Details of the implementation of a POSIX-compliant file system interface. Details of the CernVM-FS API for custom client application. Use of network resources and client-side fail-over algorithms. Status and limits of uses and scale.
Synopsis: Based on the CernVM and PROOF exercises for CERN School of Computing. A sample data analysis task is performed using a set of virtual machines on a cloud infrastructure (IaaS cloud).
Hands-on guidelines for local grid site adminstrators responsible for the CernVM-FS installation.