67 research outputs found
Development of an Event Builder software based on InfiniBand network for the LHCb experiment
No full textLHCb è uno dei quattro esperimenti principali, attualmente in funzione, presso il Large Hadron Collider (LHC) del CERN. I suoi obiettivi sono quelli di eseguire misure precise dei parametri della violazione di CP e di studiare decadimenti rari degli adroni contenenti i quark b e c. L’esperimento LHCb subirà un importante aggiornamento durante il secondo spegnimento di LHC (2018-2019), da realizzare con l'obiettivo di acquisire una quantità di dati un'ordine di grandezza superiore rispetto alle possibilità del presente rivelatore. L'aggiornamento del detector prevede la realizzazione di un trigger completamente software, che permette di acquisire dati alla frequenza di bunch crossing di 40 MHz senza limitazioni. A tale scopo è stata progettata una nuova scheda di acquisizione, ad alte prestazioni, basata sullo standard PCIe-3. Questa scheda permetterà la costruzione di un sistema di acquisizione efficiente, ed economicamente vantaggioso, utilizzando tecnologie di rete ad alte prestazioni.
Lo scopo di questa tesi è lo studio, la progettazione e l'implementazione del software impiegato per 'Event Builder del futuro rivelatore di LHCb. L'Event Builder sarà costituito da circa 500 nodi di calcolo, ciascuno dei quali eseguirà due funzioni logiche distinte: la Readout Unit (RU) e la Builder Unit (BU). Ciascuna RU, a partire dai dati provenienti dal detector, creerà un frammento di evento e lo spedirà ad una specifica BU in un modello "molti a uno”. Ciascuna BU ha la funzione di riaggregare i frammenti per creare gli eventi completi. Gli eventi completi sono spediti al sistema di trigger software per successive elaborazioni.
Attualmente la rete InfiniBand è la tecnologia di connessione più utilizzata nei supercalcolatori e fornisce le prestazioni migliori tra le tecnologie basate su standard aperti. Per queste ragioni ho scelto InfiniBand come tecnologia di rete per l'implementazione di questo Event Builder. Per prima cosa ho sviluppato il Communication Layer, un software per il trasferimento dati tra differenti nodi utilizzando i "verbi" di InfiniBand (una libreria che offre delle API che permettono di sfruttare le capacità RDMA della scheda).
Sopra al Communication Layer ho creato il Logic Layer dell'Event Builder, un software che simula la logica dell'event-building in maniera realistica. Questo software implementa un approccio "push" con regole di spedizione dedicate, in modo tale da minimizzare il rischio di congestione del traffico. Ho inoltre evitato l'utilizzo di un supervisore centrale, preferendo una strategia predefinita di scheduling.
L'Event Builder è stato testato in differenti centri di calcolo in modo tale da provare le prestazioni e la scalabilità del software. Infine, ho effettuato studi preliminari sulla possibilità di utilizzare architetture a basso consumo per sistemi di acquisizione ad alte prestazioni, utilizzando l'Event Builder come caso d'uso.The LHCb experiment is one of the four main experiments operating at the Large Hadron Collider at CERN. Its main goals are to make precise measurements of CP violation and to study rare decays of b and c-quark hadrons. The LHCb experiment will undergo a major upgrade during the second long shutdown (2018 - 2019), aiming at collecting an order of magnitude more data the possible with the present detector. The upgraded detector foresees a full software trigger, running at the LHC bunch crossing frequency of 40 MHz. A new high-throughput PCIe Generation 3 based read-out board has been designed on this purpose. The read-out board will allow an efficient and cost-effective implementation of the Data Acquisition System by means of high-speed PC networks.
The aim this thesis is to study, design and implement an efficient Event Builder software for the LHCb upgrade. The Event Builder will consist of about 500 nodes. Each server of the EB relies on two distinct logical components: the Readout Unit (RU) and the Builder Unit (BU). The RU packs the event fragments from the detector, and ship it to a receiving BU in a many-to-one pattern. Each BU gathers the event fragments to assemble them in the full events, which is sent out afterwards for filtering to a specific trigger farm unit. For these reasons I choose InfiniBand as network technology to build up the Event Builder implementation. I developed the EB Communication Layer software, to be used for data transfer among different nodes of the EB exploiting InfiniBand verbs, which is a library that offers an user space API to access the RDMA capabilities of the network device. The InfiniBand network standard is currently the most commonly used interconnection technology in supercomputers. It provides the best performances among the open-standard I/O technologies.
On top of the Communication Layer I created the Logic Layer of the Event Builder, a software that performs the event-building in a realistic way. This software implements a push approach with custom dispatching policies, minimizing the risk of traffic congestion. I also avoided the adoption of a central scheduler, preferring a predefined scheduling strategy. The Event Builder software was tested on different clusters in order to prove its performance and scalability. Finally, I performed preliminary studies on the possibility to use low power architectures for high-throughput data acquisition purposes, using the Event Builder software as use case
A general purpose suite for Grid resources exploitation
No full textWe present a general-purpose software framework, which allows different multi-disciplinary communities to take advantage of a distributed computational infrastructure. The ultimate goal is to provide organizations that need to exploit resources with CPU-intensive loose-parallel tasks with a software service capable to offer a user-friendly, standard and highly customizable access to the Grid. The software suite we developed has been designed specifically for organizations that cannot afford the adoption costs of more specialized and complex frameworks, developed in High Energy Physics (HEP) environment, but that still require an easy-to-use interface to the Grid. Our framework heavily relies on a bookkeeping database, storing both application-specific and infrastructure meta-data, which is tightly coupled with a web-based user-interface. The first makes available to the users information on the execution status of jobs and their specific meaning and parameters, and contributes in orchestrating the submission mechanism. The latter provides job submission management, bookkeeping database interactions, basic monitoring functionality and eLog system. Multi-site submissions based on user-defined requests and fine grain parametric submission interfaces are available. The structure of framework services follow a centralized design: job management service and bookkeeping database are hosted in a European Grid Infrastructure (EGI) site. Jobs executed into remote sites transfer their output to predefined target site repository and update the bookkeeping database. In addition, the framework requires a proper configuration of the remote Grid sites on which the jobs will run. Results from a large production of Monte Carlo simulated events submitted to 15 Grid sites are reported, and a comparison in terms of features, scopes, and targets, with a broad spectrum of general-purpose solutions in the same field of application is presented as well
The Trigger and Data Acquisition System for the 8 tower subsystem of the KM3NeT detector
No full textKM3NeT is a deep-sea research infrastructure being constructed in the Mediterranean Sea. It will host a large Cherenkov neutrino telescope that will collect photons emitted along the path of the charged particles produced in neutrino interactions in the vicinity of the detector. The philosophy of the DAQ system of the detector foresees that all data are sent to shore after a proper sampling of the photomultiplier signals. No off-shore hardware trigger is implemented and a software selection of the data is performed with an on-line Trigger and Data Acquisition System (TriDAS) to reduce the large throughput due to the environmental light background. A first version of the TriDAS has been developed to operate a prototype detection unit deployed in March 2013 in the abyssal site of Capo Passero (Sicily, Italy), about 3500 m deep. A revised and improved version has been developed to meet the requirements of the final detector, using new tools and modern design solutions. First installation and scalability tests have been performed at the Bologna Common Infrastructure and results comparable to what expected have been observed
The Trigger and Data Acquisition System for the KM3NeT-Italia towers
No full textKM3NeT-Italia is an INFN project supported with Italian PON fundings for building the core of the Italian node of the KM3NeT neutrino telescope. The detector, made of 700 10′′ Optical Modules (OMs) lodged along 8 vertical structures called towers, will be deployed starting from fall 2015 at the KM3NeT-Italy site, about 80 km off Capo Passero, Italy, 3500 m deep. The all data to shore approach is used to reduce the complexity of the submarine detector, demanding for an on-line trigger integrated in the data acquisition system running in the shore station, called TriDAS. Due to the large optical background in the sea from 40K decays and bioluminescence, the throughput from the underwater detector can range up to 30 Gbps. This puts strong constraints on the design and performances of the TriDAS and of the related network infrastructure. In this contribution the technology behind the implementation of the TriDAS infrastructure is reviewed, focusing on the relationship between the various components and their performances. The modular design of the TriDAS, which allows for its scalability up to a larger detector than the 8-tower configuration is also discussed
Large-scale DAQ tests for the LHCb upgrade
No full textThe Data Acquisition (DAQ) of the LHCb experiment[1] will be upgraded in 2020 to a high-bandwidth trigger-less readout system. In the new DAQ event fragments will be forwarded to the to the Event Builder (EB) computing farm at 40 MHz. Therefore the front-end boards will be connected directly to the EB farm through optical links and PCI Express based interface cards. The EB is requested to provide a total network capacity of 32Tb/s, exploiting about 500 nodes. In order to get the required network capacity we are testing various technology and network protocols on large scale clusters. We developed on this purpose an Event Builder implementation designed for an InfiniBand interconnect infrastructure. We present the results of the measurements performed to evaluate throughput and scalability measurements on HPC scale facilities
Realizzazione di un’infrastruttura Cloud pilota basata su OpenStack
No full textIl contributo si sviluppa all’interno del progetto Marche Cloud, che prevede lo sviluppo di un’infrastrut- tura cloud basata su software open source per la Regione Marche. In questo lavoro si presenta la realizzazione del prototipo di tipo IaaS (Infrastructure as a Service) di tale infrastruttura, inizialmente installato al CNAF e successi- vamente portato presso il data center della Regione Marche ad Ancona. L’infrastruttura è basata sul software Open- Stack, installato e con gurato nelle componenti di identity service, image repository, compute node, object storage e dashboard. Nel progetto sono supportati diversi sistemi operativi e formati di immagini per le VM. Il le system distribuito GlusterFS è stato utilizzato per abilitare la funzionalità di live migration e al ne di ottenere ridondanza, performance e alta af dabilità di alcune componenti dell’infrastruttura stessa. È stato sviluppato un sistema essibile di monitoring e allarmistica sfruttando l’integrazione in OpenStack di framework esterni, speci catamente Ganglia e Nagios
High throughput data acquisition with InfiniBand on x86 low-power architectures for the LHCb upgrade
No full textThe LHCb Collaboration is preparing a major upgrade of the detector and the Data Acquisition (DAQ) to be installed during the LHC-LS2. The new Event Builder computing farm for the DAQ requires about 500 nodes, and have to be capable of transporting order of 32 Tbps. The requested performance can possibly be achieved using high-bandwidth data-centre switches and commodity hardware. Several studies are ongoing to evaluate and compare network and hardware technologies, with the aim of optimising the performance and also the purchase and maintenance costs of the system. We are investigating if x86 low-power architectures can achieve equivalent performance as traditional servers when used for multi gigabit DAQ. In this talk we introduce an Event Builder implementation based on InfiniBand network and show preliminary tests with this network technology on x86 low-power architectures, such as Intel Atom C2750 and Intel Xeon D-1540, comparing measured bandwidth and power consumption
The 40 MHz trigger-less DAQ for the LHCb Upgrade
No full textThe LHCb experiment will undergo a major upgrade during the second long shutdown (2018–2019), aiming to let LHCb collect an order of magnitude more data with respect to Run 1 and Run 2. The maximum readout rate of 1 MHz is the main limitation of the present LHCb trigger. The upgraded detector, apart from major detector upgrades, foresees a full read-out, running at the LHC bunch crossing frequency of 40 MHz, using an entirely software based trigger
An integrated infrastructure in support of software development
No full textThis paper describes the design and the current state of implementation of an infrastructure made available to software developers within the Italian National Institute for Nuclear Physics (INFN) to support and facilitate their daily activity. The infrastructure integrates several tools, each providing a well-identified function: project management, version control system, continuous integration, dynamic provisioning of virtual machines, efficiency improvement, knowledge base. When applicable, access to the services is based on the INFN-wide Authentication and Authorization Infrastructure. The system is being installed and progressively made available to INFN users belonging to tens of sites and laboratories and will represent a solid foundation for the software development efforts of the many experiments and projects that see the involvement of the Institute. The infrastructure will be beneficial especially for small- and medium-size collaborations, which often cannot afford the resources, in particular in terms of know-how, needed to set up such services. © Published under licence by IOP Publishing Ltd
- …
