209 research outputs found
Neural Architecture Search Benchmarks: Insights and Survey
No full textNeural Architecture Search (NAS), a promising and fast-moving research field, aims to automate the architectural design of Deep Neural Networks (DNNs) to achieve better performance on the given task and dataset. NAS methods have been very successful in discovering efficient models for various Computer Vision, Natural Language Processing, etc. The major obstacles to the advancement of NAS techniques are the demand for large computation resources and fair evaluation of various search methods. The differences in training pipeline and setting make it challenging to compare the efficiency of two NAS algorithms. A large number of NAS Benchmarks to simulate the architecture evaluation in seconds have been released over the last few years to ease the computation burden of training neural networks and can aid in the unbiased assessment of different search methods. This paper provides an extensive review of several publicly available NAS Benchmarks in the literature. We provide technical details and a deeper understanding of each benchmark and point out future directions.This article is published as Chitty-Venkata, Krishna Teja, Murali Emani, Venkatram Vishwanath, and Arun K. Somani. "Neural Architecture Search Benchmarks: Insights and Survey." IEEE Access 11 (2023): 25217 - 25236.
DOI: 10.1109/ACCESS.2023.3253818.
Copyright 2023 The Author(s).
Attribution 4.0 International (CC BY 4.0).
Posted with permission
Efficient Design Space Exploration for Sparse Mixed Precision Neural Architectures
No full textPruning and Quantization are two effective Deep Neural Network (DNN) compression methods for efficient inference on various hardware platforms. Pruning refers to removing unimportant weights or nodes, whereas Quantization converts the floating-point parameters to low-bit fixed integer representation. The pruned and low precision models result in smaller and faster inference models on hardware platforms with almost the same accuracy as the unoptimized network. Tensor Cores in Nvidia Ampere 100 (A100) GPU supports (1) 2:4 fine-grained sparse pruning where 2 out of every 4 elements are pruned, and (2) traditional dense multiplication to achieve a good accuracy and performance trade-off. The A100 Tensor Core also takes advantage of 1-bit, 4-bit, and 8-bit multiplication to speed up the inference of a model. Hence, finding the right matrix type (dense or 2:4 sparse) along with the precision for each layer becomes a combinatorial problem. Neural Architecture Search (NAS) can alleviate such problems by automating the architecture design process instead of a brute-force search. In this paper, we propose (i) Mixed Sparse and Precision Search (MSPS), a NAS framework to search for efficient sparse and mixed-precision quantized model within the predefined search space and fixed backbone neural network (Eg. ResNet50), and (ii) Architecture, Sparse and Precision Search (ASPS) to jointly search for kernel size and number of filters, and sparse-precision combination of each layer. We illustrate the effectiveness of our methods targeting A100 Tensor Core on Nvidia GPUs by searching efficient sparse-mixed precision networks on ResNet50 and achieving better accuracy-latency trade-off models compared to the manually designed Uniform Sparse Int8 networks.This proceeding is published as Chitty-Venkata, Krishna Teja, Murali Emani, Venkatram Vishwanath, and Arun K. Somani. "Efficient Design Space Exploration for Sparse Mixed Precision Neural Architectures." In Proceedings of the 31st International Symposium on High-Performance Parallel and Distributed Computing, pp. 265-276. 2022.
DOI: 10.1145/3502181.3531463.
Copyright 2022 The Author(s).
Attribution 4.0 International (CC BY 4.0).
Posted with permission
A Survey of Techniques for Optimizing Transformer Inference
No full textRecent years have seen a phenomenal rise in performance and applications of transformer neural networks. The family of transformer networks, including Bidirectional Encoder Representations from Transformer (BERT), Generative Pretrained Transformer (GPT) and Vision Transformer (ViT), have shown their effectiveness across Natural Language Processing (NLP) and Computer Vision (CV) domains. Transformer-based networks such as ChatGPT have impacted the lives of common men. However, the quest for high predictive performance has led to an exponential increase in transformers' memory and compute footprint. Researchers have proposed techniques to optimize transformer inference at all levels of abstraction. This paper presents a comprehensive survey of techniques for optimizing the inference phase of transformer networks. We survey techniques such as knowledge distillation, pruning, quantization, neural architecture search and lightweight network design at the algorithmic level. We further review hardware-level optimization techniques and the design of novel hardware accelerators for transformers. We summarize the quantitative results on the number of parameters/FLOPs and accuracy of several models/techniques to showcase the tradeoff exercised by them. We also outline future directions in this rapidly evolving field of research. We believe that this survey will educate both novice and seasoned researchers and also spark a plethora of research efforts in this field.This preprint of Chitty-Venkata, K.T., Mittal, S., Emani, M., Vishwanath, V., Somani, A.K., A Survey of Techniques for Optimizing Transformer Inference. Is available at ArXiv (arXiv:2307.07982) https://doi.org/10.48550/arXiv.2307.07982. Posted with permission. CC BY-NC-ND 4.0 DEED Attribution-NonCommercial-NoDerivs 4.0 International. Published as Chitty-Venkata, Krishna Teja, Sparsh Mittal, Murali Emani, Venkatram Vishwanath, and Arun K. Somani. "A survey of techniques for optimizing transformer inference." Journal of Systems Architecture (2023): 102990.
doi: https://doi.org/10.1016/j.sysarc.2023.10299
Neural Architecture Search for Transformers: A Survey
No full textTransformer-based Deep Neural Network architectures have gained tremendous interest due to their effectiveness in various applications across Natural Language Processing (NLP) and Computer Vision (CV) domains. These models are the de facto choice in several language tasks, such as Sentiment Analysis and Text Summarization, replacing Long Short Term Memory (LSTM) model. Vision Transformers (ViTs) have shown better model performance than traditional Convolutional Neural Networks (CNNs) in vision applications while requiring significantly fewer parameters and training time. The design pipeline of a neural architecture for a given task and dataset is extremely challenging as it requires expertise in several interdisciplinary areas such as signal processing, image processing, optimization and allied fields. Neural Architecture Search (NAS) is a promising technique to automate the architectural design process of a Neural Network in a data-driven way using Machine Learning (ML) methods. The search method explores several architectures without requiring significant human effort, and the searched models outperform the manually built networks. In this paper, we review Neural Architecture Search techniques, targeting the Transformer model and its family of architectures such as Bidirectional Encoder Representations from Transformers (BERT) and Vision Transformers. We provide an in-depth literature review of approximately 50 state-of-the-art Neural Architecture Search methods and explore future directions in this fast-evolving class of problems.This article is published as Chitty-Venkata, Krishna Teja, Murali Emani, Venkatram Vishwanath, and Arun K. Somani. "Neural Architecture Search for Transformers: A Survey." IEEE Access 10 (2022): 108374-108412.
DOI: 10.1109/ACCESS.2022.3212767.
Copyright 2022 IEEE. Attribution 4.0 International (CC BY 4.0).
Posted with permission
Review of the book Critiquing Brahmanism: A collection of essays, by K. Murali (Ajith)
No full textDr. Devin Zane Shaw (Douglas College) reviews the book Critiquing Brahmanism: A collection of essays, by K. Murali (Ajith) (2020).Final article published
Differentiable Neural Architecture, Mixed Precision and Accelerator Co-Search
No full textQuantization, effective Neural Network architecture, and efficient accelerator hardware are three important design paradigms to maximize accuracy and efficiency. Mixed Precision Quantization is a process of assigning different precision to different Neural Network layers for optimized inference. Neural Architecture Search (NAS) is a process of automatically designing the neural network for a task and can also be extended to search for the precision of each weight and activation matrix. In this paper, we develop the following three methods: (i) Fast Differentiable Hardware-aware Mixed Precision Quantization Search method to find optimal precision, (ii) Joint Differentiable hardware-aware Architecture and Mixed Precision Quantization Co-search, (iii) Joint Accelerator, Architecture, and Precision triple co-search to find best possibilities in all the three worlds. We demonstrate the effectiveness of our proposed methods targeting Bitfusion accelerator by searching mixed precision models on MobilenetV2. We achieve better accuracy-latency trade-off models than the manually designed and previously proposed search methods.This article is published as Chitty-Venkata, Krishna Teja, Yiming Bian, Murali Emani, Venkatram Vishwanath, and Arun K. Somani. "Differentiable Neural Architecture, Mixed Precision and Accelerator Co-search." IEEE Access 11 (2023):106670-106687. doi: https://doi.org/10.1109/ACCESS.2023.3320133. This is an open access article distributed under a Creative Commons Attribution 4.0 License
Adaptive parallelism mapping in dynamic environments using machine learning
No full textModern day hardware platforms are parallel and diverse, ranging from mobiles to
data centers. Mainstream parallel applications execute in the same system competing
for resources. This resource contention may lead to a drastic degradation in a program’s
performance. In addition, the execution environment composed of workloads
and hardware resources, is dynamic and unpredictable. Efficient matching of program
parallelism to machine parallelism under uncertainty is hard. The mapping policies
that determine the optimal allocation of work to threads should anticipate these variations.
This thesis proposes solutions to the mapping of parallel programs in dynamic environments.
It employs predictive modelling techniques to determine the best degree of
parallelism. Firstly, this thesis proposes a machine learning-based model to determine
the optimal thread number for a target program co-executing with varying workloads.
For this purpose, this offline trained model uses static code features and dynamic runtime
information as input.
Next, this thesis proposes a novel solution to monitor the proposed offline model
and adjust its decisions in response to the environment changes. It develops a second
predictive model for determining how the future environment should be, if the current
thread prediction was optimal. Depending on how close this prediction was to the
actual environment, the predicted thread numbers are adjusted.
Furthermore, considering the multitude of potential execution scenarios where no
single policy is best suited in all cases, this work proposes an approach based on the
idea of mixture of experts. It considers a number of offline experts or mapping policies,
each specialized for a given scenario, and learns online the best expert that is optimal
for the current execution. When evaluated on highly dynamic executions, these solutions
are proven to surpass default, state-of-art adaptive and analytic approaches
Performance Analysis of Process Parameters on Machining Titanium (Ti-6Al-4V) Alloy Using Abrasive Water Jet Machining Process
No full textAbstractOwing to its light weight and corrosive resistant, Titanium (Ti-6Al-4V) alloy is mainly utilized in fabricating medical device applications. Since it has high strength, it is very difficult to machine alloy using conventional machining. In the present study, an endeavor has been made to machine titanium alloy using AWJM process. Since the process involves with less heat affect zone and higher material removal, it is possible to enhance machinability of workpiece. It has attempted to find the influence of process parameters on surface roughness and topography for enhancing the process. It has been observed that the abrasive flow rate and standoff distance has the most significant role on determining surface quality
Performance evaluation of transmit diversity techniques in the CDMA 2000 standard
No full textThesis (M. Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2004.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Includes bibliographical references (p. 123).This thesis evaluates the performance of two forward-link transmit diversity techniques in the CDMA2000 standard: Space-Time Spreading (STS) and Phase-Sweep Transmit Diversity (PSTD). For each technique, the evaluation consists of conducting 9.6 kbps Markov calls in the field and measuring the mean forward-link fundamental-channel (F-FCH) transmit power required to achieve a 1% frame error-rate (FER) at the mobile receiver. The required transmit power is used to compute an estimate of cell capacity as measured by the number of supported users, assuming a fixed total transmit power at the base station. It is observed that enabling STS increases capacity by up to 80% if all mobiles support STS, but capacity is reduced by up to 20% when fewer than 35% of the mobiles support the technique. The capacity loss results from interference of the diversity-antenna signal on mobiles that do not support STS; such interference causes an F-FCH transmit power increase of up to 1.5 dB in multipath Rayleigh-faded channels, as observed in lab experiments. PSTD, which does not require mobile-specific support, was found to improve cell capacity by 12% according to the field experiments.by Murali S. Vajapeyam.M.Eng
- …
