1,721,083 research outputs found
Aggregates are all you need (to bridge stream processing and Complex Event Recognition)
No full textEmerging as an alternative to databases for continuous data processing, stream processing has evolved significantly since its inception in the early 2000s, leading to the emergence of numerous Stream Processing Engines (SPEs). Two main approaches exist to define streaming applications: to explicitly define graphs of common operators (Filters, Maps, Joins, and Aggregates) as the Dataflow model prescribes, or to express patterns of interest based on observations of low-level events within the domain under analysis, known as Complex Event Recognition (CER). Motivated by SPEs' semantic overlap, recent research has shown Aggregates suffice for an SPE to be as semantically expressive as other SPEs. However, a question remains open: Do Aggregates possess the semantic expressiveness required to cover CER too? We address this question formally demonstrating they indeed hold such semantic expressiveness
Search-Based Software Engineering - 6th International Symposium, SSBSE 2014, Proceedings
No full textUser-intensive software, such asWeb andmobile applications, heavily depends on the interactions with large and unknown populations of users. Knowing the preferences and behaviors of these populations is crucial for the success of this class of systems. A/B testing is an increasingly popular technique that supports the iterative development of userintensive software based on controlled experiments performed on live users. However, as currently performed, A/B testing is a time consuming, error prone and costly manual activity. In this paper, we investigate a novel approach to automate A/B testing. More specifically, we rephrase A/B testing as a search-based software engineering problem and we propose an initial approach that supports automated A/B testing through aspect-oriented programming and genetic algorithms
Low Latency Complex Event Processing on Parallel Hardware
No full textMost complex information systems are event-driven: each part of the system reacts to the events happening in the other parts, potentially generating new events. Complex Event Processing (CEP) engines in charge of interpreting, filtering, and combining primitive events to identify higher level composite events
according to a set of rules, are the new breed of Message Oriented Middleware, which is being proposed today to better support event-driven interactions.
A key requirement for CEP engines is low latency processing, even in presence of complex rules and large numbers of incoming events. In this paper we investigate how parallel hardware may speed up CEP processing. In particular, we consider the most common operators oered by existing rule languages (i.e., sequences, parameters, and aggregates); we consider different algorithms to process rules built using such operators; and we discuss how they can be implemented on a multi-core CPU and on CUDA, a widespread architecture for general purpose programming on GPUs.
Our analysis shows that the use of GPUs can bring impressive speedups in presence of complex rules. On the other hand, it shows that multi-core CPUs scale better with the number of rules. Our conclusion is that an advanced CEP engine should leverage a multi-core CPU for processing the simplest rules, using the GPU as a coprocessor devoted to process the most complex ones
Complex event processing with T-REX
No full textSeveral application domains involve detecting complex situations and reacting to them. This asks for a Complex Event Processing (CEP) middleware specifically designed to timely process large amounts of event notifications as they flow from the peripheral to the center of the system, to identify the composite events relevant for the application. To answer this need we designed T-Rex, a new CEP middleware that combines expressiveness and efficiency. On the one hand, it adopts a language (TESLA) explicitly conceived to easily and naturally describe composite events. On the other hand, it provides an efficient event detection algorithm based on automata to interpret TESLA rules. Our evaluation shows that the T-Rex engine can process a large number of complex rules with a reduced overhead, even in the presence of challenging workloads
High-Performance Publish-Subscribe Matching Using Parallel Hardware
Full text linkMatching incoming event notifications against received subscriptions are a fundamental part of every publish-subscribe infrastructure. In the case of content-based systems this is a fairly complex and time consuming task, whose performance impacts that of the entire system. In the past, several algorithms have been proposed for efficient content-based event matching. While they differ in most aspects, they have in common the fact of being conceived to run on conventional, sequential hardware. On the other hand, parallel hardware is becoming available off-the-shelf: the number of cores inside CPUs is constantly increasing, and CUDA makes it possible to access the power of GPU hardware for general purpose computing. In this paper, we describe a new publish-subscribe content-based matching algorithm designed to run efficiently both on multicore CPUs and CUDA GPUs. A detailed comparison with two state-of-the-art sequential matching algorithms demonstrates how the use of parallel hardware can bring impressive speedups in content-based matching. At the same time, our analysis identifies the characteristic aspects of multicore and CUDA programming that mostly impact performance
Semi-Automated Design of Data-Intensive Architectures
Full text linkToday, data guides the decision-making process of most companies. Effectively analyzing and manipulating data at scale to extract and exploit relevant knowledge is a challenging task, due to data characteristics such as its size, the rate at which it changes, and the heterogeneity of formats. To address this challenge, software architects resort to build complex data-intensive architectures that integrate highly heterogeneous software systems, each offering vertically specialized functionalities. Designing a suitable architecture for the application at hand is crucial to enable high quality of service and efficient exploitation of resources. However, the design process entails a series of decisions that demand technical expertise and in-depth knowledge of individual systems and their synergies, To assist software architects in this task, this paper introduces a development methodology for data-intensive architectures, which guides architects in (i) designing a suitable architecture for their specific application scenario, and (ii) selecting an appropriate set of concrete systems to implement the application. To do so, the methodology grounds on (1) a language to precisely define an application scenario in terms of characteristics of data and requirements of stakeholders; (2) an architecture description language for data-intensive architectures; (3) a classification of systems based on the functionalities they offer and their performance trade-offs. We show that the description languages we adopt can capture the key aspects of data-intensive architectures proposed by researchers and practitioners, and we validate our methodology by applying it to real-world case studies documented in literature
Deployment Strategies for Distributed Complex Event Processing
No full textSeveral complex event processing (CEP) middleware solutions have been proposed in the past. They act by processing primitive events generated by sources, extracting new knowledge in the form of composite events, and delivering them to interested sinks. Event-based applications often involve a large number of sources and sinks, possibly dispersed over a wide geographical area. To better support these scenarios, the CEP middleware can be internally built around several, distributed processors, which cooperate to provide the processing and routing service. This paper introduces and compares different deployment strategies for a CEP middleware, which define (i) how the processing load is distributed over different processors and (ii) how these processors interact to produce the required results and to deliver them to sinks. Our evaluation compares the presented solutions and shows their benefits with respect to a centralized deployment, both in terms of network traffic and in terms of forwarding delay
DEBS '21: The 15th ACM International Conference on Distributed and Event-based Systems, Virtual Event, Italy, June 28 - July 2, 2021
No full textThis conference is the fifteenth in a series that spans 20 years of history, with 14 past editions as a conference and five editions as a workshop co-located with major conferences. The ACM International Conference on Distributed and Event - Based Systems (DEBS) has become the premier venue for cutting-edge research in event processing, distributed computing, and the integration of distributed and event-based systems in relevant domains such as Big Data, AI, ML, IoT and Blockchain. The objectives of the ACM International Conference on Distributed and Event - Based Systems (DEBS) are to provide a forum dedicated to the dissemination of original research, the discussion of practical insights, and the reporting of experiences relevant to distributed systems and event-based computing. The conference provides a forum for academia and industry to exchange ideas through its tutorials, research papers, and the grand challenge
On the Semantic Overlap of Operators in Stream Processing Engines
Full text linkStream Processing Engines (SPEs) extract value from data streams in the Edge-to-Cloud continuum through graphs of operators that progressively transform data. State-of-the-art SPEs are bridged into shared models based on their overlapping APIs. The overlap in their semantic expressiveness, though, goes beyond their APIs and can be formally assessed by distilling the semantics they support into minimal sets of operators, and by checking whether such sets overlap. As we show, stream Aggregates suffice to enforce the semantics of other common operators. Moreover, compositions of Aggregates can match the performance of other operators in state-of-the-art SPEs, and micro-SPEs building on a single Aggregate operator can even surpass other SPEs’ performance while holding the same semantic expressiveness with a minimal code footprint. Our approach lays down new analytical findings with practical implications in minimizing the operational effort to use SPEs, especially at the edge, while seamlessly benefiting existing distribution/parallelization techniques
- …
