1,377 research outputs found
Bridging the gap between expressivity and efficiency in stream reasoning: a structural caching approach for IoT streams
Get PDFIn today’s data landscape, data streams are well represented. This is mainly due to the rise of data-intensive domains such as the Internet of Things (IoT), Smart Industries, Pervasive Health, and Social Media. To extract meaningful insights from these streams, they should be processed in real time, while solving an integration problem as these streams need to be combined with more static data and their domain knowledge. Ontologies are ideal for modeling this domain knowledge and facilitate the integration of heterogeneous data within data-intensive domains such as the IoT. Expressive reasoning techniques, such as OWL2 DL reasoning, are needed to completely interpret the domain knowledge and for the extraction of meaningful decisions. Expressive reasoning techniques have mainly focused on static data environments, as it tends to become slow with growing datasets. There is thus a mismatch between expressive reasoning and the real-time requirements of data-intensive domains. In this paper, we take a first step towards bridging the gap between expressivity and efficiency while reasoning over high-velocity IoT data streams for the task of event enrichment. We present a structural caching technique that eliminates reoccurring reasoning steps by exploiting the characteristics of most IoT streams, i.e., streams typically produce events that are similar in structure and size. Our caching technique speeds up reasoning time up to thousands of times for fully fledged OWL2 DL reasoners and even tenths and hundreds of times for less expressive OWL2 RL and OWL2 EL reasoners
Autonomous resource-aware scheduling of large-scale media workflows
Get PDFThe media processing and distribution industry generally requires considerable resources to be able to execute the various tasks and workflows that constitute their business processes. The latter processes are often tied to critical constraints such as strict deadlines. A key issue herein is how to efficiently use the available computational, storage and network resources to be able to cope with the high work load. Optimizing resource usage is not only vital to scalability, but also to the level of QoS (e.g. responsiveness or prioritization) that can be provided. We designed an autonomous platform for scheduling and workflow-to-resource assignment, taking into account the different requirements and constraints. This paper presents the workflow scheduling algorithms, which consider the state and characteristics of the resources (computational, network and storage). The performance of these algorithms is presented in detail in the context of a European media processing and distribution use-case
Autonomic quality of experience management of multimedia networks
No full textThe proliferation of multimedia services over access networks (e. g., IPTV or network-based Personal Video Recording) has introduced important new revenue potential for network and service providers but has also complicated the management burden. As a result, today's management of multimedia networks is often too static to cope with the increasing quality requirements of multimedia services. A key point in these quality requirements is the quality as perceived by the end users, denoted as the Quality of Experience (QoE). In the thesis, we have introduced an autonomic management layer that optimizes the QoE of multimedia networks. We have studied several QoE optimizing techniques with respect to traffic adaptation, admission control and video rate adaptation. All these QoE optimizing techniques exhibit autonomic behavior as they continuously monitor the network to optimize their configuration and consequently optimize the QoE. Furthermore, we have investigated the coordinated deployment of these QoE optimizing techniques by focusing on the exchange of context between entities in the distributed autonomic management layer. Through extensive evaluation using both simulation and emulation on a large-scale testbed, we have shown that the proposed QoE optimizing techniques can successfully optimize the QoE of multimedia services. This QoE optimization was characterized in terms of metrics such as the number of admitted sessions and video quality
Self-organizing fog support services for responsive edge computing
Get PDFRecent years have seen fog and edge computing emerge as new paradigms to provide more responsive software services. While both these concepts have numerous advantages in terms of efficiency and user experience by moving computational tasks closer to where they are needed, effective service scheduling requires a different approach in the geographically widespread fog than it does in the cloud. Additionally, fog and edge networks are volatile, and of such a scale that gathering all the required data for a centralized scheduler results in prohibitively high memory use and network traffic. Since the fog is a geographically distributed computational substrate, a suitable solution is to use a decentralized service scheduler, deployed on all nodes, which can monitor and deploy services in its neighbourhood without having to know the entire service topology. This article presents a fully decentralized service scheduler, labeled "SoSwirly", for fog and edge networks containing hundreds of thousands of devices. It scales service instances as required by the edge, based on available resources and flexibly defined distance metrics. A mathematical model of fog networks is presented, along with a theoretical analysis and an empirical evaluation which indicate that under the right conditions, SoSwirly is highly scalable. It is also explained how to achieve these conditions by carefully selecting configuration parameters. Concretely, only 15 MiB of memory is required on each node, and network traffic in the evaluations is less than 4 Kbps on edge nodes, while 4-6% more service instances are created than by a centralized algorithm
Diktyo: Network-Aware Scheduling in Container-based Clouds
Get PDFContainers have revolutionized application deployment and life-cycle management in current cloud platforms. Applications have evolved from single monoliths to complex graphs of loosely-coupled microservices. However, the efficient allocation of microservice-based applications is challenging due to their complex inter-dependencies. Further, recent applications are becoming even more delay-sensitive, demanding lower latency between dependent microservices. Scheduling policies in popular container orchestration platforms mainly aim to increase the resource efficiency of the infrastructure, insufficient for latency-sensitive applications. Application domains such as the Internet of Things and multi-tier Web services would benefit from network-aware policies that consider network latency and bandwidth in the scheduling process. Previous works have studied network-aware scheduling via theoretical formulations or heuristic-based methods evaluated via simulations or small testbeds, making their full applicability in popular platforms difficult. This paper proposes a novel network-aware framework for the popular Kubernetes (K8s) platform named Diktyo that determines the placement of dependent microservices in long-running applications focused on reducing the application's end-to-end latency and guaranteeing bandwidth reservations. Simulations show that Diktyo can significantly reduce the network latency for various applications across different infrastructure topologies compared to default K8s scheduling plugins. Also, experiments in a K8s cluster with microservice benchmark applications show that Diktyo can increase database throughput by 22% and reduce application response time by 45%
- …
