1,720,994 research outputs found
Apprendimento Continuo mediante Metodi Rehearsal
No full textLe reti neurali artificiali (Artificial Neural Networks - ANN) hanno acquisito un ruolo di massimo rilievo nel contesto delle applicazioni contemporanee di Intelligenza Artificiale, portando ad un incremento costante delle potenzialità dei programmi informatici grazie alla loro efficacia e versatilità. Benché eccellenti nella loro capacità di generalizzazione, queste richiedono strettamente che il loro addestramento sfrutti dati indipendenti e identicamente distribuiti. Mentre l'intelligenza umana permette naturalmente di acquisire nuovi concetti in maniera incrementale, le ANN dimenticano la conoscenza pregressa in modo catastrofico ogniqualvolta intervenga una variazione nella distribuzione dei dati di addestramento. Questa limitazione fondamentale impedisce lo sviluppo di sistemi intelligenti capaci di adattarsi rapidamente al contesto in cui operano e vincola l'aggiornamento dei modelli a onerose procedure di riaddestramento.
L'apprendimento continuo (Continual Learning - CL) è una branca in rapido sviluppo del machine learning che si prefigge come obiettivo lo sviluppo di architetture volte a compensare la dimenticanza catastrofica nelle ANN. Tra le soluzioni proposte, un ruolo di primaria importanza è rivestito dai metodi rehearsal (Rehearsal-Based Methods - RBM), che evitano la necessità di riaddestramento mediante l'immagazzinamento e il riutilizzo una modica quantità di dati pregressi, individuando così un compromesso ottimale tra efficacia e efficienza.
Questa tesi raccoglie i contributi scientifici nell'ambito del CL prodotti dal candidato nel corso delle sue attività di dottorato. Inizialmente, si presenta un esame della letteratura recente, evidenziando la rilevanza degli RBM e mostrando che il noto approccio Experience Replay - proposto per la prima volta negli anni '90 - resta competitivo rispetto allo stato dell'arte quando si assumono opportuni accorgimenti operativi. Successivamente, il lavoro si focalizza sulla proposta di nuovi RBM che sfruttano i principi di distillazione di conoscenza ([X-]DER), adattamento dinamico implicito della capacità del modello (LiDER) e regolarizzazione geometrica dello spazio latente del modello (CaSpeR). Gli approcci proposti sono convalidati mediante estese analisi sperimentali, volte anche a mettere in risalto le specifiche proprietà da essi conferite al modello.
La parte finale di questa tesi presenta analisi dell'applicabilità di RBM a scenari che superano il tipico assetto sperimentale di classificazione incrementale: un nuovo esperimento volto a perseguire una modellazione più realistica dei cambi di distribuzione nei dati di ingresso, uno studio sulla applicabilità di CL in regime di supervisione limitata e una analisi sull'interazione tra CL e il pre-addestramento. Questi studi portano allo sviluppo di architetture e prassi operative volte a colmare il divario tra la letteratura e la applicazione di sistemi CL ad applicazioni realistiche.Artificial Neural Networks (ANNs) have been established as the centrepiece of contemporary Artificial Intelligence, steadily raising the bar for what can be accomplished by computer programs thanks to their effectiveness and versatility. While they shine especially for their capability for generalisation, these systems impose the strict requirement that their training procedure should insist on independent and identically distributed data. In contrast with human intelligence - which seamlessly allows us to acquire knowledge continuously - ANNs forget previously acquired knowledge catastrophically whenever their training data distribution changes over time. Such a fundamental limitation prevents the development of intelligent systems capable of quick adaptation, crucially tying model updates to a cumbersome offline retraining procedure.
Continual Learning (CL) is a rapidly growing area of machine learning whose aim is counteracting the catastrophic forgetting phenomenon in ANNs through purposefully designed approaches. Among these, a prominent role is played by Rehearsal-Based Methods (RBM), which operate by storing few pieces of previously encountered data for later re-use, thus striking a favourable balance between efficacy and efficiency.
This thesis encompasses the contributions to CL made by the candidate during his doctoral studies. Starting from a review of recent literature, it highlights the relevance of RBMs and shows that the decades-old Experience Replay baseline is competitive with current state-of-the-art approaches when carefully trained. Subsequently, this manuscript focuses on the proposal of novel RBMs, which expand on the basic replay formula by leveraging knowledge distillation ([X-]DER), implicit dynamic adaptation of network capacity (LiDER) and geometric regularisation of the model's latent space (CaSpeR). Extensive experimental analyses highlight the merits of the proposed approaches, shedding light on the specific properties they confer on the in-training model.
Finally, this thesis investigates the applicability of RBMs beyond the typical incremental classification setting. Namely, a novel CL experimental scenario is introduced to provide more realistic evaluations w.r.t. common benchmarks in literature, an investigation is presented concerning the viability of CL when limited supervision is available, a thorough study is conducted on the interplay between pre-training and CL. As a result, architectures and best practices are introduced that bridge the gap between standard CL evaluations and real-world applications
Continual Semi-Supervised Learning through Contrastive Interpolation Consistency
Full text linkContinual Learning (CL) investigates how to train Deep Networks on a stream
of tasks without incurring forgetting. CL settings proposed in literature
assume that every incoming example is paired with ground-truth annotations.
However, this clashes with many real-world applications: gathering labeled
data, which is in itself tedious and expensive, becomes infeasible when data
flow as a stream. This work explores Continual Semi-Supervised Learning (CSSL):
here, only a small fraction of labeled input examples are shown to the learner.
We assess how current CL methods (e.g.: EWC, LwF, iCaRL, ER, GDumb, DER)
perform in this novel and challenging scenario, where overfitting entangles
forgetting. Subsequently, we design a novel CSSL method that exploits metric
learning and consistency regularization to leverage unlabeled examples while
learning. We show that our proposal exhibits higher resilience to diminishing
supervision and, even more surprisingly, relying only on 25% supervision
suffices to outperform SOTA methods trained under full supervision.Comment: 7 pages, 2 figures, to appear in Pattern Recognition Letters, Volume
162, October 2022, Pages 9-1
Are questionnaires a reliable method to measure food waste? A pilot study on Italian households
No full textPurpose: The purpose of this paper is to assess the reliability of questionnaires as a method of quantifying household food waste (FW), thus providing context regarding the validity of existing Italian estimates. Design/methodology/approach: A total of 30 households were involved in a diary study that was conducted for one week. The participating households were first asked about their FW quantities in a questionnaire. Half of the households who filled their diaries properly were then audited through waste sorting analysis performed on their garbage. Non-parametric tests were used to test for differences in FW estimates between audited and non-audited households, as well as differences among estimates obtained through different quantification methodologies. Findings: Edible FW was estimated to be 489 grams per week based on questionnaires, and 1,035 grams per week based on diaries. In the audited sub-sample of households, FW estimates were 334 grams per week based on questionnaires, 818 grams per week based on diaries and 1,058 grams per week based on waste sorting analysis. Research limitations/implications: Given the small sample size in the present study, future studies can utilize larger samples to assess whether the differences identified in estimates can be replicated. Future studies can also inquire into the behavioral biases that led consumers to underestimate their FW. Practical implications: Results of the present study point against the use of questionnaires to quantify household FW, hence raising some doubt on the reliability of existent Italian estimates. Where waste sorting is unfeasible, the use of adjustment methods or diaries is suggested to better inform policies. Originality/value: This study is one of the first on FW quantification that tests three different methodologies on the same sample, and is the first to do so in Italy, where estimates are still very poor
Class-Incremental Continual Learning into the eXtended DER-verse
Full text linkThe staple of human intelligence is the capability of acquiring knowledge in
a continuous fashion. In stark contrast, Deep Networks forget catastrophically
and, for this reason, the sub-field of Class-Incremental Continual Learning
fosters methods that learn a sequence of tasks incrementally, blending
sequentially-gained knowledge into a comprehensive prediction.
This work aims at assessing and overcoming the pitfalls of our previous
proposal Dark Experience Replay (DER), a simple and effective approach that
combines rehearsal and Knowledge Distillation. Inspired by the way our minds
constantly rewrite past recollections and set expectations for the future, we
endow our model with the abilities to i) revise its replay memory to welcome
novel information regarding past data ii) pave the way for learning yet unseen
classes.
We show that the application of these strategies leads to remarkable
improvements; indeed, the resulting method - termed eXtended-DER (X-DER) -
outperforms the state of the art on both standard benchmarks (such as CIFAR-100
and miniImagenet) and a novel one here introduced. To gain a better
understanding, we further provide extensive ablation studies that corroborate
and extend the findings of our previous research (e.g. the value of Knowledge
Distillation and flatter minima in continual learning setups).Comment: 23 pages, 22 figures. To appear in IEEE TPAM
Improving the reliability of 3D people tracking system by means of deep-learning
No full textPeople tracking is a crucial task in most computer vision applications aimed at analyzing specific behaviors in the sensed area. Practical applications include vision analytics, people counting, etc. In order to properly follow the actions of a single subject, a people tracking framework needs to robustly recognize it from the rest of the surrounding environment, thus allowing proper management of changing positions, occlusions and so on. The recent widespread diffusion of deep learning techniques on almost any kind of computer vision application provides a powerful methodology to address recognition. On the other hand, a large amount of data is required to train state-of-the-art Convolutional Neural Networks (CNN) and this problem is solved, when possible, by means of transfer learning. In this paper, we propose a novel dataset made of nearly 26 thousand samples acquired with a custom stereo camera providing depth according to a fast and accurate stereo algorithm. The dataset includes sequences acquired in different environments with more than 20 different people moving across the sensed area. Once labeled the 26 K images and depth maps of the dataset, we train a head detection module based on state-of-the-art deep network on a portion of the dataset and validate it a different sequence. Finally, we include the head detection module within an existing 3D tracking framework showing that the proposed approach notably improves people detection and tracking accuracy
Preliminary assessment of a methodology for determining food waste in primary school canteens
No full textReducing food losses and waste is increasingly seen as a main way to improve sustainability of food systems, both in itself and as a way to question and improve the efficiency of resource use. Numerous studies have stressed the need to improve data collection and analysis of main causes of food losses and waste particularly in the last parts of the food chain. The project REDUCE, financed by the Italian Ministry of Environment and Protection of Land and Sea, aims to improve data collection on waste in the last stages of food chains and to identify innovative solutions to reduce it. This paper presents the first results of a study developed as part of this project. The objective of this study is to devise an innovative methodology to assess food waste in school canteens that is at the same time accurate, easy to transpose, does not require external support, provides all the useful data on quantity and nutritional quality of food waste (to enable comparison of food intake in children with dietary recommendations such as the Dietary Guidelines for Italians) and involves all concerned actors: kitchen employees and teachers, as well as the pupils themselves, so that monitoring becomes an instrument of active learning
Preliminary results of a methodology for determining food waste in primary school canteens
No full textReducing food waste (FW) is seen as a way to improve
sustainability of food systems, both in itself and as a way
to improve the efficiency of resource use. A first step is
to improve data collection of FW.
The paper presents the results of a test conducted in a
primary school located in the Bologna province. The aim
of this study is to define a new methodology to assess
FW in school canteens that can be applied in large-scale
studies involving all stakeholders.
The results show that a methodology for data gathering
on FW in school canteens involving all the concerned
actors can be implemented. However for the success of
the monitoring it is necessary the involvement of teachers
that remain the key to success, but also it is necessary
to adapt the methodology to the capabilities of pupils
Latent Spectral Regularization for Continual Learning
Full text linkWhile biological intelligence grows organically as new knowledge is gathered throughout life, Artificial Neural Networks forget catastrophically whenever they face a changing training data distribution. Rehearsal-based Continual Learning (CL) approaches have been established as a versatile and reliable solution to overcome this limitation; however, sudden input disruptions and memory constraints are known to alter the consistency of their predictions. We study this phenomenon by investigating the geometric characteristics of the learner\u27s latent space and find that replayed data points of different classes increasingly mix up, interfering with classification. Hence, we propose a geometric regularizer that enforces weak requirements on the Laplacian spectrum of the latent space, promoting a partitioning behavior. Our proposal, called Continual Spectral Regularizer for Incremental Learning (CaSpeR-IL), can be easily combined with any rehearsal-based CL approach and improves the performance of SOTA methods on standard benchmarks.14 pages, 4 figures, , to appear in Pattern Recognition Letters, Volume 184, August 2024, Pages 119-12
Going Beyond Counting First Authors in Author Co-citation Analysis
Full text linkThe present study examines one of the fundamental aspects of author co-citation analysis (ACA) - the way co-citation
counts are defined. Co-citation counting provides the data on which all subsequent statistical analyses and mappings
are based, and we compare ACA results based on two different types of co-citation counting - the traditional type that
only counts the first one among a cited work's authors on the one hand and a non-traditional type that takes into
account the first 5 authors of a cited work on the other hand. Results indicate that the picture produced through this non-traditional author co-citation counting contains more coherent author groups and is therefore considerably clearer. However, this picture represents fewer specialties in the research field being studied than that produced through the traditional first-author co-citation counting when the same number of top-ranked authors is selected and analyzed. Reasons for these effects are discussed
- …
