1,720,997 research outputs found
Dependency-Aware Reordering for Parallelizing Query Optimization in Multi-Core CPUs
No full textThe state of the art commercial query optimizers employ cost-based optimization and exploit dynamic programming (DP) to find the optimal query execution plan (QEP) without evaluating redundant sub-plans. The number of alternative QEPs enumerated by the DP query optimizer can increase exponentially, as the number of joins in the query increases. Recently, by exploiting the coming wave of multi-core processor architectures, a state of the art parallel optimization algorithm [14], referred to as PDPsva, has been proposed to parallelize the "time-consuming" DP query optimization process itself. While PDPsva significantly extends the practical use of DP to queries having up to 20-25 tables, it has several limitations: I) supporting only the size-driven DP enumerator, 2) statically allocating search space, and 3) not fully exploiting parallelism. In this paper, we propose the first generic solution for parallelizing any type of bottom-up optimizer, including the graph-traversal driven type, and for supporting dynamic search allocation and full parallelism. This is a challenging problem, since recently developed, state of art DP optimizers such as DPcpp [21] and DPhyp [22] are very difficult to parallelize due to tangled dependencies in the join pairs they generate. Unless the solution is very carefully devised, a lot of synchronization conflicts are bound to occur. By viewing a serial bottom-up optimizer as one which generates a totally ordered sequence of join pairs in a streaming fashion, we propose a novel concept of dependency-aware reordering, which minimizes waiting time caused by dependencies of join pairs. To maximize parallelism, we also introduce a series of novel performance optimization techniques: 1) pipelining of join pair generation and plan generation; 2) the synchronization-free global MEMO; and 3) threading across dependencies. Through extensive experiments with various query topologies, we show that our solution supports any type of bottom up optimization, achieving linear speedup for each type. Despite the fact that our solution is generic, due to sophisticated optimization techniques, our generic parallel optimizer outperforms PDPsva tailored to size-driven enumeration. Experimental results also show that our solution is much more robust than PDPsva with respect to search space allocation.11211sciescopu
Fast and Robust Parallel SGD Matrix Factorization
No full textMatrix factorization is one of the fundamental techniques for analyzing latent relationship between two entities. Especially, it is used for recommendation for its high accuracy. Efficient parallel SGD matrix factorization algorithms have been developed for large matrices to speed up the convergence of factorization. However, most of them are designed for a shared-memory environment thus fail to factorize a large matrix that is too big to fit in memory, and their performances are also unreliable when the matrix is skewed. This paper proposes a fast and robust parallel SGD matrix factorization algorithm, called MLGF-MF, which is robust to skewed matrices and runs efficiently on block-storage devices (e.g., SSD disks) as well as shared-memory. MLGF-MF uses Multi-Level Grid File (MLGF) for partitioning the matrix and minimizes the cost for scheduling parallel SGD updates on the partitioned regions by exploiting partial match queries processing. Thereby, MLGF-MF produces reliable results efficiently even on skewed matrices. MLGF-MF is designed with asynchronous I/O permeated in the algorithm such that CPU keeps executing without waiting for I/O to complete. Thereby, MLGF-MF overlaps the CPU and I/O processing, which eventually offsets the I/O cost and maximizes the CPU utility. Recent flash SSD disks support high performance parallel I/O, thus are appropriate for executing the asynchronous I/O. From our extensive evaluations, MLGF-MF significantly outperforms (or converges faster than) the state-of-the-art algorithms in both shared-memory and block-storage environments. In addition, the outputs of MLGF-MF is significantly more robust to skewed matrices. Our implementation of MLGF-MF is available at http ://dm.postech.ac.kr/MLGF-MF as executable files.118sciescopu
General Match: A Subsequence Matching Method in Time-Series Databases Based on Generalized Windows
No full text1
TurboISO: Towards UltraFast and Robust Subgraph Isomorphism Search in Large Graph Databases
No full text1167scopu
A New Approach for Processing Ranked Subsequence Matching Based on Ranked Union
No full text1118scopu
2D-CHI: A Tunable Two-Dimensional Class Hierarchy Index for Object- Oriented Databases
No full textThis paper presents a tunable two-dimensional class hierarchy indexing technique (2D-CHI) for object-oriented databases. We nse a two-dimensional file organization as the index structure. 2D-CHI deals with the problem of clustering objects in a two-dimensional domain space consisting of the key attribute domain and the class identifier domain. In conventional class indexing techniques using one-dimensional index structures such as the Bt-tree, the clustering property! is exclusively owned by one attribute. These indexing techniques do not efficiently handle the queries that address both the attribute keys and the class identifiers. 2D-CHI enhances query performance by adjusting the degree of clustering between the key value domain and the class identifier domain based on the precollected usage pattern. For performance evaluation, we first compare 2D-CHI with the conventional class indexing techniques using no analytic cost model based on the assumption of uniform object distribution, and then, verify the cost model through experiments using the multilevel grid file as the two-dimensional index. We further perform experiments with nonuniform object distributions. Our experiments show that our proposed method does indeed build optimal class index structures regardless of query types and object distributions. We strongly believe that our paper significantly contributes to building a self-tunable database system by supporting automatically tunable index structure.1100sciescopu
Prefetching Based on the Type-Level Access Pattern in Object-Relational DBMSs
No full textPrefetching is an effective method for minimizing the number of round-trips between the client and the server in database management systems. In this paper, we propose new notions of the type-level access locality and the type-level access pattern. We also formally define the notions of capturing and prefetching to help understand the underlying mechanisms. We then develop an efficient prefetching policy based on these notions and the framework. The type-level access locality is a pheonomenon that repetitive patterns exist in the attributes referenced. The type-level access pattern is a pattern of attributes that are referenced in accessing the objects. Existing prefetching methods are based on object-level or page-level access patterns, which consist of object-ids or page-ids of the objects accessed. However the drawback of these methods is that they work only when exactly the same objects or pages are accessed repeatedly. In contrast, even though the same objects are not accessed repeatedly, our technique effectively prefetches objects if the same attributes are referenced repeatedly, i.e., if there is type-level access locality. Many navigational applications in Object-Relational Database Management Systems(ORDBMSs) have type-level access locality Therefore, our technique can be employed in ORDBMSs to effectively reduce the number of round-trips, thereby significantly enhancing the performance. We have conducted extensive experiments in a prototype ORDBMS to show the effectiveness of our algorithm. Experimental results using the OO7 benchmark and a real GIS application show that our technique provides orders of magnitude improvements in round-trips and several factors of improvements in overall performance over on-demand fetching and context-based prefetching, which is a state-of-the-art prefetching method. These results indicate that our approach provides a new paradigm in prefetching that improves performance of navigational applications significantly and is a practical method that can be implemented in commercial ORDBMSs.11sci
- …
