E-Planet 2013 Visit at UERJ

This page provides a summary of the visit program of CernVM at UERJ in the framework of the EU E-Planet program. The visit took place in Oktober/November 2013. Our main contacts at UERJ were Alberto Santoro and his group of Computing Scientists/Engineers, led by José Afonso Sanches and Eduardo Revoredo.

Program: HEP Data Processing in the Cloud

The program of lectures and exercises is focused on the effective use of cloud resources for processing HEP data. From a physicist's perspective, this includes instantiating, contextualizing, and coordinating a set of virtual machines (a "virtual analysis facility") that processes a given data set. For a virtual organization and the infrastructure service provider, hands-on sessions on CernVM, and CernVM-FS have are performed. The program builds on the previous work done together with CernVM project member Artem Harutyunyan. 

Topics

Lecture: Evolving from Grid Computing to Cloud Computing

Synopsis: Recap of grid constituents such as grid layer model, storage element, file catalog, task queue, scheduler, compute element, monitoring. Discussion of the cloud computing model such as types of clouds, IaaS cloud components, virtual appliances. Based on the tools and frameworks developed within WLCG, discussion of how virtualization and cloud computing can simplify distributed computing and harness new resources. Difficulties and challenges it will touch: virtual appliance management, contextualization interfaces, status of virtualization of co-processors (GPU, MIC) and low-power processors (e.g. Atom, ARM)

Lecture: Virtual Appliance Management with CernVM

Synopsis: Details of the CernVM virtual machine internals and the build process. Being a very lightweight image, CernVM can eliminate the need for a virtual machine image store. The use of the CernVM-FS versioning features facilitates long-term data preservation since a previously defined data processing environment can be re-created.

Lecture: Volunteer Computing with CernVM and Copilot

Synopsis: Overview of the current status of the Test4Theory project. Overview of planned projects: LHC simulation production jobs, volunteer computing for Blue Brain. Running desktop grids and volunteer clouds detached from the BOINC platform.

Lecture: Internals of the CernVM File System

Synopsis: Details of the use of content-addressable storage inside CernVM-FS. Details of versioning, replication, and storage management in CernVM-FS. Details of the implementation of a POSIX-compliant file system interface. Details of the CernVM-FS API for custom client application. Use of network resources and client-side fail-over algorithms. Status and limits of uses and scale.

Exercise: Distributed Data Analysis on the Cloud / with PROOF (part 1, part 2)

Synopsis: Based on the CernVM and PROOF exercises for CERN School of Computing. A sample data analysis task is performed using a set of virtual machines on a cloud infrastructure (IaaS cloud).

Hands-On: CernVM-FS configuration

Hands-on guidelines for local grid site adminstrators responsible for the CernVM-FS installation. 

You are here