18 research outputs found
Rock pingos in northern Ungava Peninsula, Quebec, Canada
No full text The author reports the first pingos found in Ungava Peninsula, Quebec. The pingos, 15–30 m high, are open-system rock pingos built of very fractured schist and hornblende gabbro. At the time of observation, water was draining out of their slopes. Peat from the top of the smaller pingo was dated at 2880 ± 100 years BP. </jats:p
Modern Data Storage Architectures for Managing Big Data: The Role of Semantically Enrichment Mechanisms in Data Management and Security
No full textΑυτή η διδακτορική διατριβή κινείται στο ευρύτερο πεδίο της Έξυπνης Επεξεργασίας Δεδομένων (Smart Data Processing - SDP) και των Συστημάτων Βαθέων Ενοράσεων (Systems of Deep Insights - SDI), εστιάζοντας στην αποθήκευση και διαχείριση Μεγάλων Δεδομένων (Big Data), αντιμετωπίζοντας σημαντικές προκλήσεις όπως η βελτιστοποίηση της πρόσβασης, της ασφάλειας και της ανάκτησης των δεδομένων. Εξετάζει σύγχρονες προσεγγίσεις για την αποδοτική διαχείριση πηγών δεδομένων, την οργάνωση και αποθήκευσή τους, εξασφαλίζοντας απρόσκοπτη πρόσβαση και ανάκτηση, ενώ ταυτόχρονα διαχειρίζεται ζητήματα που σχετίζονται με την ακεραιότητα των δεδομένων, την ιδιωτικότητα και τον έλεγχο πρόσβασης. Μια βασική συνεισφορά της έρευνας αυτής είναι η ανάπτυξη ενός εννοιολογικά εμπλουτισμένου πλαισίου Data Lake, το οποίο ενισχύει τη δομή, την προσβασιμότητα και τη διακυβέρνηση των δεδομένων μέσω της αξιοποίησης εννοιολογικών προτύπων δεδομένων καθοδηγούμενων από μεταδεδομένα (Semantic Data Blueprints - SDB), υποστηρίζοντας παράλληλα και την εξόρυξη διαδικασιών (process mining). Τα εμπειρικά ευρήματα δείχνουν ότι οι αρχιτεκτονικές Data Mesh υπερτερούν σημαντικά των παραδοσιακών Data Lakes, προσφέροντας αυξημένη επεκτασιμότητα, ευελιξία και ευκινησία στη λήψη αποφάσεων. Η διατριβή αποδεικνύει ότι η μετάβαση από κεντρικοποιημένα Data Lakes σε αποκεντρωμένες, εννοιολογικά εμπλουτισμένες αρχιτεκτονικές Data Mesh επιτρέπει την ενίσχυση της ανακαλυψιμότητας των δεδομένων, της άντλησης πληροφοριών σε πραγματικό χρόνο και της ασφαλούς διαλειτουργικότητας μεταξύ οργανισμών. Η εφαρμογή των παραπάνω εννοιών σε ένα περιβάλλον έξυπνης παραγωγής καταδεικνύει πώς τα Data Mesh που καθοδηγούνται από μεταδεδομένα μπορούν να βελτιώσουν την επιχειρησιακή αποδοτικότητα, την ιχνηλασιμότητα των δεδομένων και να υποστηρίξουν αποκεντρωμένους μηχανισμούς ελέγχου πρόσβασης. Η ενσωμάτωση τεχνολογίας Blockchain και Μη Ανταλλάξιμων Διακριτικών (Non-Fungible Tokens - NFTs) ενισχύει περαιτέρω την ιδιοκτησία, την ακεραιότητα και την ασφαλή διαχείριση πρόσβασης στα Data Lakes και τα Data Meshes. Μέσα από πειραματική αξιολόγηση με τη χρήση πραγματικών βιομηχανικών δεδομένων, η έρευνα αναδεικνύει την αποτελεσματικότητα του προτεινόμενου πλαισίου στη βελτιστοποίηση των ροών δεδομένων, στη μείωση καθυστερήσεων επεξεργασίας και στην ενίσχυση της ασφάλειας. Η παρούσα διατριβή παρέχει πολύτιμες μεθοδολογίες για επιχειρήσεις που επιδιώκουν να αξιοποιήσουν τη δύναμη των Μεγάλων Δεδομένων, προάγοντας ένα πιο έξυπνο, ασφαλές και προσαρμοστικό πρότυπο διαχείρισης δεδομένων.This PhD thesis moves in the broader area of Smart Data Processing (SDP) and Systems of Deep Insights (SDI) and focuses on Big Data storage and management, addressing significant challenges such as optimizing data access, security, and retrieval. It explores current approaches for efficiently managing data sources, their organization, and storage for seamless access and retrieval while addressing challenges related to data integrity, privacy, and access control. A key contribution of this research is the development of a semantically enriched Data Lake framework, which enhances data structuring, accessibility, and governance by leveraging metadata-driven semantic data blueprints (SDB) supporting also process mining. Empirical findings demonstrate that Data Mesh architectures significantly outperform traditional Data Lakes, offering improved scalability, flexibility, and decision-making agility. The thesis demonstrates how transitioning from centralized Data Lakes to decentralized, semantically enriched Data Meshes enables enhanced data discoverability, real-time insights, and secure cross-organizational collaboration. The application of the aforementioned concepts in a smart manufacturing environment showcases how metadata-driven Data Meshes streamline operational efficiency, improve data traceability, and facilitate decentralized access control mechanisms. The integration of Blockchain technology and Non-Fungible Tokens (NFTs) further strengthens data ownership, integrity, and secures access management in Data Lakes and Data Meshes. Through experimental evaluation using real-world industrial data, research conducted highlights the effectiveness of the proposed framework in optimizing data workflows, reducing processing delays and enhancing security. This research provides valuable methodologies for enterprises seeking to harness the power of Big Data, fostering a more intelligent, secure, and adaptive data management paradigm.Dr. Andreas S. Andreou, Professor
Dr. Herodotos Herodotou, Associate Professor
Dr. Willem-Jan van den Heuvel, ProfessorComplete
Using Knowledge Graphs for Record Linkage: Challenges and Opportunities
No full textIn this paper, we explore how Knowledge Graphs (KGs) can potentially benefit Record Linkage (RL). RL is the process of identifying and resolving duplicate records across different data sources, including structured, semi-structured, and unstructured data (e.g., in data lakes). RL is a critical task for information systems that rely on data to make decisions and is used in a wide variety of fields such as healthcare, finance, government and marketing. Due to recent advances in machine learning, there has been a significant progress in building automated RL methods. However, when dealing with vertical applications, featuring specialized domains such as a particular hospital or industry, human experts are still required to enter domain-specific knowledge, making RL prohibitively expensive. Despite KGs can be powerful tools to represent and derive domain-specific knowledge, their application to RL has been overlooked. Inspired by a healthcare case study in the Republic of Cyprus, we aim at filling this gap by identifying challenges and opportunities of using KGs to reduce the effort of solving RL in vertical applications
Discovering Data Domains and Products in Data Meshes Using Semantic Blueprints
No full textNowadays, one of the greatest challenges in data meshes revolves around detecting and creating data domains and data products for providing the ability to adapt easily and quickly to changing business needs. This requires a disciplined approach to identify, differentiate and prioritize distinct data sources according to their content and diversity. The current paper tackles this highly complicated issue and suggests a standardized approach that integrates the concept of data blueprints with data meshes. In essence, a novel standardization framework is proposed that creates data products using a metadata semantic enrichment mechanism, the latter also offering data domain readiness and alignment. The approach is demonstrated using real-world data produced by multiple sources in a poultry meat production factory. A set of functional attributes is used to qualitatively compare the proposed approach to existing data structures utilized in storage architectures, with quite promising results. Finally, experimentation with different scenarios varying in data product complexity and granularity suggests a successful performance
Exploiting Metadata Semantics in Data Lakes Using Blueprints
No full textSmart processing of Big Data has been recently emerged as a field that provides quite a few challenges related to how multiple heterogeneous data sources that produce massive amounts of structured, semi-structured and unstructured data may be handled. One solution to this problem is manage this fusion of disparate data sources through Data Lakes. The latter, though, suffers from the lack of a disciplined approach to collect, store and retrieve data to support predictive and prescriptive analytics. This chapter tackles this challenge by introducing a novel standardization framework for managing data in Data Lakes that combines mainly the 5Vs Big Data characteristics and blueprint ontologies. It organizes a Data Lake using a ponds architecture and describes a metadata semantic enrichment mechanism that enables fast storing to and efficient retrieval. The mechanism supports Visual Querying and offers increased security via Blockchain and Non-Fungible Tokens. The proposed approach is compared against other known metadata systems utilizing a set of functional properties with very encouraging results
A Data Lake Metadata Enrichment Mechanism via Semantic Blueprints
No full textOne of the greatest challenges in Smart Big Data Processing nowadays revolves around handling multiple heterogeneous data sources that produce massive amounts of structured, semi-structured and unstructured data through Data Lakes. The latter requires a disciplined approach to collect, store and retrieve/analyse data to enable efficient predictive and prescriptive modelling, as well as the development of other advanced analytics applications on top of it. The present paper addresses this highly complex problem and proposes a novel standardization framework that combines mainly the 5Vs Big Data characteristics, blueprint ontologies and Data Lakes with ponds architecture, to offer a metadata semantic enrichment mechanism that enables fast storing to and efficient retrieval from a Data Lake. The proposed mechanism is compared qualitatively against existing metadata systems using a set of functional characteristics or properties, with the results indicating that it is indeed a promising approach
DLMetaChain: An IoT Data Lake Architecture Based on the Blockchain
No full textNowadays, the IoT ecosystem is evolving rapidly, with multiple heterogeneous sources producing high volumes of data and processes transforming this data into meaningful or “smart” information. These volumes of data, including IoT data, need to be stored in repositories that can host raw, unprocessed, relational and non-relational types of data, such as Data Lakes. Due to the weakness of metadata management, security & access control is one of the main challenges of Big Data storage architectures as Data Lakes can be replaced without oversight of the contents. Recently, the Blockchain technology has been introduced as an effective solution to build trust between different entities, where trust is either nonexistent or unproven, and to address security and privacy concerns. In this paper we introduce DLMetaChain, an extended Data Lake metadata mechanism that consists of data from heterogeneous data sources which interact with IoT data. The extended mechanism mainly focuses on developing an architecture to ensure that the data in the Data Lake is not modified or altered by taking into advantage the capabilities of the Blockchain
Security and Ownership in User-Defined Data Meshes
No full textData meshes are an approach to data architecture and organization that treats data as a product and focuses on decentralizing data ownership and access. It has recently emerged as a field that presents quite a few challenges related to data ownership, governance, security, monitoring, and observability. To address these challenges, this paper introduces an innovative algorithmic framework leveraging data blueprints to enable the dynamic creation of data meshes and data products in response to user requests, ensuring that stakeholders have access to specific portions of the data mesh as needed. Ownership and governance concerns are addressed through a unique mechanism involving Blockchain and Non-Fungible Tokens (NFTs). This facilitates the secure and transparent transfer of data ownership, with the ability to mint time-based NFTs. By combining these advancements with the fundamental tenets of data meshes, this research offers a comprehensive solution to the challenges surrounding data ownership and governance. It empowers stakeholders to navigate the complexities of data management within a decentralized architecture, ensuring a secure, efficient, and user-centric approach to data utilization. The proposed framework is demonstrated using real-world data from a poultry meat production factory
Transforming Data Lakes to Data Meshes Using Semantic Data Blueprints
No full textIn the continuously evolving and growing landscape of Big Data, a key challenge lies in the transformation of a Data Lake into a Data Mesh structure. Unveiling a transformative approach through semantic data blueprints enables organizations to align with changing business needs swiftly and effortlessly. This paper delves into the intricacies of detecting and shaping Data Domains and Data Products within Data Lakes and proposes a standardized methodology that combines the principles of Data Blueprints with Data Meshes. Essentially, this work introduces an innovative standardization framework dedicated to generating Data Products through a mechanism of semantic enrichment of data residing in Data Lakes. This mechanism not only enables the creation readiness and business alignment of Data Domains, but also facilitates the extraction of actionable insights from software products and processes. The proposed approach is qualitatively assessed using a set of functional attributes and is compared against established data structures within storage architectures yielding very promising results
Enhancing Interaction with Data Lakes Using Digital Twins and Semantic Blueprints
No full textAdvanced analytical techniques and sophisticated decision-making strategies are imperative for handling extensive volumes of data. As the quantity, diversity, and speed of data increase, there is a growing lack of confidence in the analytics process and resulting decisions. Despite recent advancements, such as metadata mechanisms in Big Data Processing and Systems of Deep Insight, effectively managing the vast and varied data from diverse sources remains a complex and unresolved challenge. Aiming to enhance interaction with Data Lakes, this paper introduces a framework based on a specialized semantic enrichment mechanism centred around data blueprints. The proposed framework takes into account unique characteristics of the data, guiding the process of locating sources and retrieving data from Data Lakes. More importantly, it facilitates end-user interaction without the need for programming skills or database management techniques. This is performed using Digital Twin functionality which offers model-based simulations and data-driven decision support
