165 research outputs found
Accelerazione basata sull’indice: join in tempo reale e query ibride
No full textL’analisi dei dati in tempo reale `e diventata sempre pi`u importante con la crescita di sistemi interconnessi. Un’applicazione comune `e il mon- l’elaborazione dei dati energetici. Questi dati sono costantemente generati dai sensori installato su diversi dispositivi che producono e consumano energia. Di nuova generazione I dati devono essere elaborati frequentemente per offrire informazioni significative subito. L’approccio tipico alla lavorazione coinvolge produttore e consumatore modelli computazionali. Sono stati utilizzati numerosi quadri di elaborazione dei dati proposto di consumare flussi di dati (dati in tempo reale) da vari input eseguire calcoli distribuiti, combinare risultati individuali e Fornisci approfondimenti. Questi framework utilizzano in genere il parallelismo della pipeline sui dati in entrata ed effettuare varie operazioni online come l’adesione, aggregazione e filtraggio.
I dati in streaming sono gestiti all'interno di finestre (scorrevoli, a cascata, di sessione, ecc.), dove le tuple vengono continuamente aggiunte e quelle scadute rimosse. L'unione di flussi è fondamentale per i dati in tempo reale, ma presenta sfide computazionali maggiori rispetto all'unione di batch tradizionali, a causa della continua ricerca, aggiunta ed eliminazione di dati. Gli operatori di unione comuni includono unioni di uguaglianza e disuguaglianza (theta), con le unioni di disuguaglianza che risultano particolarmente intensive. Per affrontare queste sfide, si propongono due intuizioni chiave: 1) identificare distribuzioni di dati distorte in tempo reale e implementare strutture di indicizzazione dedicate per ridurre i costi di aggiornamento; 2) sfruttare strutture di dati ottimizzate, con strutture mutabili efficienti per l'inserimento e immutabili per la ricerca, per ottimizzare il processo di unione dei flussi.
In questo lavoro di dottorato propongo nuove soluzioni per l’elaborazione di join di flussi distribuiti. Uno dei contributi chiave `e un metodo di indicizzazione che utilizza un filtro dedicato efficiente in termini di spazio per monitorare la frequenza delle chiavi di input in tempo reale. Questo metodo, chiamato STA-Join, adatta la logica di elaborazione dei dati in base all’asimmetria dei dati. Inoltre, ho ampiamente confrontato questa tecnica con gli approcci esistenti. Inoltre, ho anche introdotto una struttura dati a due stadi per gestire ed elaborare efficacemente elementi della finestra scorrevole (contenuti streaming delimitati) con operatori di disuguaglianza complessi. Questo approccio, denominato SPO-Join, divide la finestra scorrevole in strutture dati mutabili (efficienti per l’inserimento) e immutabili (efficienti per la ricerca). Nonostante le sfide affrontate, come la gestione dello stato per l’elaborazione distribuita, le garanzie di elaborazione e i meccanismi di concorrenza efficienti, i risultati sperimentali dei sistemi di elaborazione di flussi distribuiti dimostrano che le soluzioni proposte superano i metodi all’avanguardia esistenti.
Allo stesso modo, man mano che i modelli di intelligenza artificiale generativa si diffondono in vari settori, tra cui quello energetico, i database vettoriali vengono sempre più utilizzati per archiviare dati industriali multidimensionali e fornire suggerimenti efficaci a questi modelli. Le prestazioni e l'accuratezza del I modelli dipendono in gran parte dalla qualità dei suggerimenti. Tuttavia, il recupero efficiente di vettori rilevanti, in particolare per le query ibride con un elevato richiamo, è un'attività complessa. Propongo una soluzione sensibile alla frequenza per una struttura di dati indice per affrontare questo problema.Real-time data analysis has become increasingly important with the growth of interconnected systems. One common application is the continuous monitoring of energy data. This data is constantly generated by the sensors installed on different energy-producing and consuming devices. Newly generated data need to be processed frequently to offer meaningful insights promptly. The typical processing approach involves producer and consumer computational patterns. Numerous data processing frameworks have been proposed to consume streaming (real-time) data from various input devices, perform distributed computation, combine individual results, and provide insights. These frameworks commonly employ pipeline parallelism on incoming data and carry out various online operations such as joining, aggregation, and filtering.
Streaming data is confined to windows (sliding, tumbling, session, etc.), where newly arriving tuples are continually inserted and expired tuples are removed frequently. Stream join is an essential operation for handling real-time data, however, it comes with additional computational challenges compared to traditional batch join, due to the continuous look-up, add, and delete data points from streaming windows. Common join operators include equality or inequality (theta) joins. The stream inequality join is particularly computationally intensive because it requires additional overhead to hold the contents of the streaming window using index data structures. To tackle this challenge, we identify two key insights: 1) identifying skewed data distributions in real-time and implementing dedicated indexing structures for skewed keys to reduce index update costs; 2) leveraging optimized data structures, including insert-efficient mutable and search-efficient immutable structures to optimize the search stream join process.
In this Ph.D. work, I propose novel solutions for distributed stream join processing. One of the key contributions is an indexing method that uses a space-efficient dedicated filter to monitor the frequency of input keys in real-time. This method, called STA-Join, adapts the data processing logic based on the skewness of the data. Additionally, I have extensively compared this technique with existing approaches. Moreover, I have also introduced a two-stage data structure for handling and processing sliding window items (bounded streaming contents) with complex inequality operators. This approach, named SPO-Join, divides the sliding window into mutable (insert-efficient) and immutable (search-efficient) data structures. Despite facing challenges such as state management for distributed processing, processing guarantees, and efficient concurrency mechanisms, experimental results from distributed stream processing systems demonstrate that the proposed solutions outperform existing state-of-the-art methods.
Similarly, as generative AI models become more widespread in various industries, including the energy sector, vector databases are increasingly being used to store multidimensional industry data and provide effective prompts to these models. The performance and accuracy of the models depend largely on the quality of the prompts. However, efficiently retrieving relevant vectors, especially for hybrid queries (vectors and predicate conditions) with high recall, is a challenging task. I propose a frequency-aware solution for an index-data structure to address this issue to facilitate approximate nearest neighbor (ANN) searches in high-dimensional spaces, especially for hybrid queries. I have extensively compared this solution with state-of-the-art vector indexing approaches for various types of queries (point, range, and mixed), and the results show that it performs better than the alternatives
SPO-Join: Efficient Stream Inequality Join.
No full textStream inequality join aims to combine tuples coming from differ-
ent streams based on inequality conditions and is a fundamental
operator in distributed data stream processing. It is known to
be computationally expensive as indexing data structures for
determining matching tuples must be continuously updated.
To significantly alleviate this problem, we propose SPO-Join,
a novel solution that combines a mutable B+-tree for efficient
insertions and an immutable sorted-array-based data structure
for efficient searching. Furthermore, our proposed method is
designed to be efficiently executed with distributed stream pro-
cessing engines. Our experiments on real-world and synthesized
datasets suggest that the proposed SPO-Join exhibits superior
performance compared to state-of-the-art index-based stream
inequality join solutions
Big Data Integration for Data-Centric AI
No full textBig data integration represents one of the main challenges for the use of techniques and tools based on Artificial Intelligence (AI) in several crucial areas: eHealth, energy management, enterprise data, etc. In this context, Data-Centric AI plays a primary role in guaranteeing the quality of the data on which these tools and techniques operate. Thus, the activities of the Database Research Group (DBGroup) of the “Enzo Ferrari” Engineering Department of the University of Modena and Reggio Emilia are moving in this direction. Therefore, we present the main research projects of the DBGroup, which are part of collaborations in various application sectors
Folio
No full textNisar Ahmad-Essay-The Role of Stereotypes in the Development of the Female Personality. pp. 1-2; M. Moazzam Zubair-Essay-By Love Serve One Another. pp. 3; Jehanzeb Anwar-Essay-A Great Escape. pp. 4-5; Ahmed Ilyas Butt-Essay-War: A Solution for Peace. pp. 6-7; Fatima Zahra-Essay-Proliferation of Electronic Media and Youth. pp. 8; M. Imran-Essay-Environmental Pollution and Our Responsibility. pp. 9; Muiz Junaid Khan-Essay-Intelligence. pp. 10; Safa Aleem-Essay-A Wake-up Call. pp. 11; Fareeha Tahir-Essay-Karo Kari: The Cruelest Reality in Pakistan. pp. 12-13; Adnan Farooqui-Essay-Democracy. pp. 14; Riaz Akbar-Essay-Politics: a Dirty Game or a Human Necessity? pp. 15-16; Mujtaba Chaudhry-Essay-Emancipation of Women. pp. 17; Adeel Riaz-Essay-The Unheard Miseries of Bonded Laborers. pp. 18-19; Nazeef Ishtiaq-Essay-Pakistan Today. pp. 20; Muhammad Adeel-Short Story-Broken Threads. pp. 21-23; Tehreem Fatima-Short Story-But Still. pp. 24; Naima Fatima-Short Story-Once Upon a Time. pp. 25-26; Syed Irfan Haider Shah-Short Story-By The Riverside, I Sat and Wept! pp. 27-28; Faiqa Javed-Short Story-Ghosts. pp. 29; M. Bilal Aslam-Short Story-A Mysterious Night. pp. 30-31; Sabrina Asim-Short Story-A Dismal Encounter. pp. 32; Umair Vahidy-Short Story-Uncertain Ambiguities. pp. 33-36; Jahanzaib Aslam-Interview-Jamsheed Marker. pp. 37-43; U. Vahidy, H. Aslam-Interview-Cecil Chaudhry's Interview. pp. 44-48; N. Ahmad, K. Shah-Interview-Muhammad Junaid. pp. 49-51; N. Ishtriaq, U. Vahidy-Interview-Qazi Laeeque Ahmed. pp. 52-56; S. Aleem, S. Ahmad-Interview-Bilal Bajwa. pp. 57-58; M. Mesam Ismail-Reflections-Loneliness. pp. 59; Haya Fatima-Reflections-I Love to Fantasize. pp. 60; Jahanzeb Anwar-Reflections-A Faith for the Faithless. pp. 61; Fizza Ali Shah-Reflections-Where Are We Heading To. pp. 62; Rabia Shad-Reflections-Need of Revolution. pp. 63; Mariam Iqbal-Reflections-An Extract from a Mother�s Diary. pp. 64; Ali Abbas-Reflections-Sense of Responsibility. pp. 65; Sabrina Asim-Reflections-Painting in Words. pp. 66; Dr. Waseem Anwar-Poetry-Reading Between Silences. pp. 67; Muhammad Adeel-Poetry-The Hand. pp. 67; Nauman Ahmad-Poetry-Fragrance, Piercing Through My Heart. pp. 68; Shumyila Imam-Poetry-Human Right. pp. 68; M. Y. Sandhu-Poetry-To the Mausoleum. pp. 69; Mumtaz Hussain Kherani-Poetry-The Real Inventor. pp. 69; Shakeel Fiaz-Poetry-God Almighty. pp. 70; Jahanzaib-Poetry-My Mother. pp. 70; Ahmed Ilyas Butt-Poetry-A Walk in the Park. pp. 70; Tajwar Ali Buber-Poetry-My Craze. pp. 70; Samra Zafarullah-Poetry-How can we Forget? pp. 71; Tanzeel Ahmad Khan Niazy-Poetry-My Daddy. pp. 71; Toqeer Ahamad Wazir Gilgity-Poetry-Heart and Mind. pp. 71; Faisal Nizami-Poetry-I am... pp. 71; Basit Zafar-Poetry-Lord! pp. 72; Nauman Ahmad-Poetry-I Try Reaching You. pp. 72; Muiz Khan-Poetry-Untitled. pp. 72; Warda Tahseen-Poetry-I am Not a Perfect Girl. pp. 72; Nisar Ahmed-Poetry-Chaos. pp. 73; Furqan Farukh-Poetry-I'll Die Another Day. pp. 73; Nisar Ahmed-Poetry-Secret Joy. pp. 74; Jahangir Jan Khokhar-Poetry-I Want To. pp. 74; Arman Ahmed-Poetry-On the Edge of Dreaming. pp. 74; Professor Arif Qureshi-Poetry-Mother, O' Dear Mother! pp. 74; Furqan Farrukh-Poetry-Love at First Sight. pp. 75; Faisal Karim Nomali-Poetry-Hazrat Muhammad (P.B.U.H.). pp. 75; Saad Akmal-Poetry-Laid Forgotten. pp. 75; Zamzam Rizvi-Poetry-A Lonely Island. pp. 76; Jahanzaib Aslam-Poetry-O My Beloved! pp. 76; Society Reports. pp. 77-80; [Urdu]. 80 p.Mr Jamsheed Marker. before page 37; Mr Cecil Chaudhry. after page 48; Qazi Laeeque Ahmed. after page 56; Mr Bilal Bajwa. before page 57; Presidents 2009-2010. after page 76; FCC Dramatic Club. before page 77; 20 pages covering different activities at FC, i.e. Alumni Reunion, Commencement, Honors Convocation, Drama, Class of 2010, Sports, Debates and Societies. after page 80; Professor Dr Agha Sohail. before page 7 Urdu section; Professor Dr Ehson Raza Khan. before page 15 Urdu sectio
Big Data Integration for Data-Centric AI
No full textBig data integration represents one of the main challenges for the use of techniques and tools based on Artificial Intelligence (AI) in several crucial areas: eHealth, energy management, enterprise data, etc. In this context, Data-Centric AI plays a primary role in guaranteeing the quality of the data on which these tools and techniques operate. Thus, the activities of the Database Research Group (DBGroup) of the “Enzo Ferrari” Engineering Department of the University of Modena and Reggio Emilia are moving in this direction. Therefore, we present the main research projects of the DBGroup, which are part of collaborations in various application sectors
Progressive Entity Resolution with Node Embeddings
No full textEntity Resolution (ER) is the task of finding records that refer to the same real-world entity, which are called matches. ER is a fundamental pre-processing step when dealing with dirty and/or heterogeneous datasets; however, it can be very time-consuming when employing complex machine learning models to detect matches, as state-of-the-art ER methods do. Thus, when time is a critical component and having a partial ER result is better than having no result at all, progressive ER methods are employed to try to maximize the number of detected matches as a function of time.
In this paper, we study how to perform progressive ER by exploiting graph embeddings. The basic idea is to represent candidate matches in a graph: each node is a record and each edge is a possible comparison to check—we build that on top of a well-known, established graph-based ER framework. We experimentally show that our method performs better than existing state-of-the-art progressive ER methods on real-world benchmark datasets
Progressive Entity Resolution with Node Embeddings
No full textEntity Resolution (ER) is the task of finding records that refer to the same real-world entity, which are called matches. ER is a fundamental pre-processing step when dealing with dirty and/or heterogeneous datasets; however, it can be very time-consuming when employing complex machine learning models to detect matches, as state-of-the-art ER methods do. Thus, when time is a critical component and having a partial ER result is better than having no result at all, progressive ER methods are employed to try to maximize the number of detected matches as a function of time.
In this paper, we study how to perform progressive ER by exploiting graph embeddings. The basic idea is to represent candidate matches in a graph: each node is a record and each edge is a possible comparison to check—we build that on top of a well-known, established graph-based ER framework. We experimentally show that our method performs better than existing state-of-the-art progressive ER methods on real-world benchmark datasets
- …
