1,720,996 research outputs found
Recommended from our members
Mixed Signal Neurocomputing Based on Floating-gate Memories
Get PDFNervous systems inspired neurocomputing has shown its great advantage in object detection, speech recognition and a lot of other machine-learning technology-driven applications from speed and power efficiency. Among handful neurocomputing implementation approaches, analog nanoelectronic circuits are very appealing because they may far overcome digital circuits of the same functionality in circuit density, speed and energy efficiency. Device density is one of the most essential metrics for designing large-scale neural networks, allowing for high connectivity between neurons. Thanks to the high-density nature of traditional memory applications, building artificial neural networks with hybrid complementary metal oxide semiconductor (CMOS)/memory devices would enable the high parallelism as well as achieve the performance advantages.Synapses, the most numerous elements of neural networks, are efficiently implemented by memory devices. This application, however, imposes a number of requirements, such as the continuous change of the memory resistance state, creating the need for novel engineering approaches. Here we report such engineering approaches for advancedcommercial 180-nm ESF1 and 55-nm ESF3 NOR flash memory, facilitating fabrication and successful test of high performance analog vector-by-matrix multiplication which is the key operation performed at signal propagation through any neuromorphic network. Furthermore, we discuss the recent progress toward neuromorphic computing implementations based on nonvolatile floating-gate devices, in particular the experimental results for a prototype 28×28-binary-input, 10-output, 3-layer neuromorphic network based on arrays of highly optimized embedded nonvolatile floating-gate cells. The fabricated neuromorphic network’s active components, including 101,780 floating-gate cells, have a total area below 1 mm2. The network has shown a 94.7% classification fidelity on the common MNIST benchmark, close to the 96.2% obtained in simulation. The classification of one pattern takes sub-1 μs time and sub-20 nJ energy – both numbers much better than for the best reported digital implementations of the same task. Estimates show that a straightforward optimization of the hardware, and its transfer to the already available 55-nm technology may increase this advantage to more than 100X in speed and 10000X in energy efficiency.As pure analog circuits cannot address the noise accumulation problem, a practical solution would also require inclusion of analog-to-digital and digital-to-analog stages for signal restoration. High energy-efficient and compact data converters are therefore expected to play an important role in future computing platforms. We perform an experimental demonstration of 6-bit digital-to-analog (DAC) and 4-bit analog-to-digital conversion (ADC) operations implemented with a hybrid circuit consisting of Pt/TiO2-x /Pt resistive switching devices (also known as ReRAMs or memristors) and a CMOS operational amplifier (opamp). In particular, ADC is implemented with a Hopfield neural network circuit
Recommended from our members
Energy-Efficient Neuromorphic Computing with CMOS-Integrated Memristive Crossbars
Get PDFThe von Neumann architecture has been broadly adopted in modern computing systems in which the central processor unit (CPU) is separated from the memory unit. During data processing, it is necessary to transfer data between the memory and CPU. For data-intensive applications such as deep neural networks, as the size of data increases, data movement between memory and CPU becomes a significant bottleneck for high throughput and energy-efficient implementation. In-memory computing is a paradigm that tackles this challenge by allowing computation within the memory, i.e., where data are stored. Hence, in-memory computing is a promising solution for implementing energy-efficient neuromorphic systems since it minimizes data transportation between memory and the processing units. The major component in developing neuromorphic circuits is a nanoscale memory device, which is responsible for weight storage and analog computation. Resistive Random-Access Memory (RRAM) is one of the most promising memory candidates due to its long-term retention, analog storage, low-power operation, and compact nanoscale footprint.The first part of this thesis explores the nonidealities of RRAM technology, such as temperature dependency, stuck-at-fault, and tunning error, and their impact on the accuracy of neuromorphic hardware implementation. We show that these imperfections may significantly degrade the inference accuracy of neuromorphic circuits. To mitigate them, we have proposed a holistic approach based on hardware-aware training in which modifications are done in tunning, circuit, and training phase (ex-situ) of hardware development. The proposed method significantly decreases the accuracy drop across the 25–100 °C temperature range, allows 2.5× to 9× improvement in energy consumption of the memory arrays during inference, and improves the defect tolerance by >100×.
In the second part of this thesis, we also study the impact of device uniformity in passive memristive circuits and the tradeoffs between computing accuracy, crossbar size, switching threshold variations, and target precision. Nonidealities are investigated in two representative deep neural networks, and several solutions, including hardware-aware training, improved tuning algorithm, and switching threshold modification, are proposed to enhance the performance. These techniques allow us to implement advanced deep neural networks (DNNs) with almost no accuracy drop, using state-of-the-art analog 0T1R technology.
In the last part, we focus on integrating passive and active RRAM with CMOS circuits for implementing efficient demos for various applications such as neural networks. First, focusing on passive technology, we show the building block circuit that facilitates the forming, programing, reading, inference, and monitoring of RRAM circuits. We discuss several neuromorphic networks and prototype demos with integrated analog passive RRAM and CMOS. The designs are fabricated in two wafer-scale tapeout runs in 180 nm CMOS technology, and preliminary encouraging experimental results are obtained. Second, we demonstrate a massive DNN accelerator fabricated in a standard 65 nm CMOS process with integrated active analog RRAM devices. The main focus is on novelties in the design of the VMM and tuning circuits, which reduced the impact of IR drop, improved the area efficiency, and allowed massive parallel programming features in this chip
Recommended from our members
Abusing Hardware Race Conditions for High Throughput Energy Efficient Computation
Get PDFWe propose a novel computing approach, called “Race Logic”, which utilizes a new data representation to accelerate a broad class of optimization problems, such as those solved by dynamic programming algorithms. The core idea of Race Logic is to deliberately engineer race conditions in a circuit to perform useful computation. In Race Logic, information, instead of being represented as logic levels (as is done in conventional logic), is represented as a timing delay. Computations can then be performed by observing the relative propagation times of signals injected into a configurable circuit (i.e. the outcome of races through the circuit).In this dissertation I will introduce Race Based computation and talk about multiple VLSI implementations. We first begin by considering a synchronous approach, which uses simple clocked delay elements. Though this synchronous implementation outperforms highly optimized conventional implementations of the well-studied, DNA sequence alignment problem, its third order energy scaling with problem size and limited dynamic range of timing delays are its major pitfalls. Next, in the search for energy efficiency, we study asynchronous designs in order to understand the performance trade-offs and applicability of this new architecture. Finally, I will present the results of a prototype asynchronous Race Logic chip and demonstrate that Race-Based computations can align up to 10 million 50 symbol long DNA sequences per second, about 2-3 orders of magnitude faster than the state of the art general purpose computing systems
Recommended from our members
On Design and Machine Learning Resiliency of Memristor- and eFlash-Memory-Based Strong Physical Unclonable Functions
Get PDFThe emergence of the Internet of Things (IoT) has enabled an unprecedented expansion of interconnected networks and devices over which a huge amount of personal and/or sensitive data is carried. As a result, privacy and security issues are among the most significant challenges in designing IoT devices. These challenges can hardly be addressed using conventional cryptographic approaches because they rely on storing secret keys in memories, which not only are vulnerable to physical and side-channel attacks but also consume huge area and vast amounts of power.Hardware-based security approaches such as physical unclonable functions (PUFs) have attracted considerable attention as replacements for conventional methods. PUFs are well suited to a wide spectrum of security applications including key generation and authentication because they generate secure keys on the fly (rather than explicitly storing any security-critical information). This is achieved by utilizing electronic devices that entail inherent sources of randomness, which in turn help create unique keys for different physical entities.Recently, a variety of emerging nano-scale non-volatile memories are being explored for use in the design of PUFs including memristors and embedded flash (eFlash) memories. The highly non-linear current-voltage characteristics and the inherent process variations of these memory devices make them promising candidates for designing PUFs. Additionally, the ultra-low power consumption and low computation time of these devices enable their use in applications with stringent requirements on energy efficiency and throughput.This dissertation presents memristor- and eFash-memory-based PUF designs that show promising security characteristics such as near-to-ideal uniformity, diffuseness, robustness, and reliability. The robustness is verified by demonstrating the high output randomness with the test suits of the National Institute of Standards and Technology and by studying various machine learning attacks. The specific contributions of this dissertation is that investigates several unexplored areas in crossbar-memory-design PUFs, e.g., finding optimal design for maximizing robustness characteristics, studying the impact of the capacity of machine learning models on robustness, and the impact of environmental change and thermal noise on reliability
Recommended from our members
Novel Circuits and Systems with Analog-Grade Memories
Get PDFThe neural computation field had finally delivered on its promises in 2013 when the University of Toronto group reported a deep neural network that outperformed other machine learning approaches in image classification accuracy. That breakthrough was not due to algorithmic advances but rather the availability of high-performance graphical processors that enabled large-scale neural network modeling. Since then, the biologically-inspired neural network algorithms have become state-of-the-art approaches in many artificial intelligence tasks, and the future progress in this field hinges on even more powerful hardware. Such hardware, however, is unlikely to be implemented with the conventional digital circuit technology, whose performance seems to be saturating due to the faltering Moore’s law. On the other hand, further opportunities are presented by neuromorphic hardware that mimics critical features of biological neural networks, most importantly analog in-memory computing, in an attempt to match their energy-efficiency. Most importantly, neuromorphic hardware takes advantage of the physical-level analog implementation of vector-by-matrix multiplication (VMM), which is the most frequent operation in any neural network. The key component of such a circuit is a nanodevice with adjustable conductance —essentially an analog nonvolatile memory—used at each crosspoint of a crossbar array and mimicking the biological synapse. Prior work showed that analog VMM circuits based on redesigned eFlash memories and metal-oxide memristors, the most promising analog memory device technologies for neuromorphic computing, are much more energy-efficient as compared to the digital counterpart implemented in similar process node and performing a similar function.
The main goal of this dissertation is to advance neuromorphic circuits based on memristors and eFlash memories on several fronts. The first part of the thesis is devoted to improving functional and physical performance of analog-domain vector-by-matrix multiplication with a specific focus on neuromorphic inference applications, including the development of novel programming algorithms, mitigation approaches for various device and circuit non-idealities, and design of efficient peripheral circuits. For example, we use novel programming algorithms to experimentally demonstrate <4% relative tuning error in a 64x64 passively integrated crossbar circuit despite significant variations, with 25% normalized standard deviation in device I-V characteristics. The developed post-fabrication methods for mitigating IR drops, I-V static nonlinearity, and device variations enable software-equivalent accuracy for the large-scale neural networks for the studied memristor technology. The efficacy of novel peripheral circuits is verified via SPICE modeling, which shows, e.g., POp/J-scale energy-efficiency for current-mode 55-nm NOR-flash memory circuits. The section is concluded with the discussion of our ongoing work on the design and fabrication of several large-scale neuromorphic chips.
The second part of this thesis extends the work on analog VMM circuits to enable the implementation of more advanced probabilistic neuromorphic hardware, which is especially effective in solving combinatorial optimization problems. By operating the previously developed analog VMM circuit in a lower signal-to-noise-ratio regime, we achieve stochastic VMM functionality and utilize such circuits to prototype small-scale restricted Boltzmann machine and Hopfield neural network with runtime-controlled effective temperature. Furthermore, we suggest several novel hardware-friendly annealing approaches and successfully verify them by solving experimentally typical combinatorial optimization problems.
The last part of this dissertation is devoted to hardware security primitives, such as physically unclonable functions and true random number generators. At the core of our idea are analog circuits based on metal-oxide memristors and eFlash memories, which are very similar to analog VMMs developed for neuromorphic computing. The main difference is that memory device non-idealities, e.g., randomness in tuning and memory I-V variations, are utilized as a rich source of static entropy, which is essential for implementing hardware security primitives. We developed three architectures - RX-PUF and VR-PUF that avoid the need for conductance tuning procedure in previously proposed memristor-based PUFs, and ChipSecure, which exploits variations in leakage current, subthreshold slope, nonlinearity, and stochastic tuning error in eFlash memory arrays to create a unique digital fingerprint. The key novelties of the proposed designs include enormous challenge-response pairs to enable strong PUF properties and a low-overhead key-booking scheme to dramatically improve the PUF reliability across a wide temperature range of operation. The analysis of the measured data in all our PUF demonstrations shows strong resilience against machine learning attacks
Recommended from our members
3D Integrated Memristive Devices for Memory and Computing
Get PDFThe von Neumann bottleneck has been growing narrower over the years, as CPU speed and memory have been increasing much faster than the bandwidth between them can accommodate. One promising approach to circumvent this problem is logic-in-memory computing, where computation is performed in the memory itself, significantly reducing traffic between the CPU and the memory subsystem. The most practical implementation of logic-in-memory utilizes electronic devices that can perform both storage and logic while being monolithically integrated into existing CMOS technology. A very prominent example of such a device is the memristor a two-terminal memory devices with high endurance, low power consumption, and proven scalability down to 10 nm. Memristors are now actively investigated for non-volatile memory applications and energy-efficient hardware implementations of artificial neural networks. Recently, a novel logic-in-memory approach implementing material implication logic with memristors was proposed by Hewlett Packard, providing a potential new way forward for opening the von Neumann bottleneck. The objective of this dissertation is to advance of the state of the art for material implication logic through three research goals. Our first goal was to develop a fabrication pathway for monolithical vertical integration of memristors in order to implement 3D memories. This allows us to experimentally test logic-in-memory systems. Our second goal was to determine memristor device and circuit constraints for implementing material implication logic and explore circuit and device level solutions to increase robustness of operation. Our final goal was to combine these two efforts together and demonstrate reliable material implication logic in vertically stacked memristors. To this end, we fabricated and successfully tested monolithically stacked memristive structures implemented with TiO2-based memristors. We also developed an optimized circuit configuration able to perform material implication with maximum tolerance to device variations. This allowed us to demonstrate, for the first time, hundreds of successful three-dimensional data manipulation cycles using material implication. An inter-layer NAND gate with the inputs and output in different device layers was implemented with 94% yield. This high yield demonstrates the potential for using the inter-layer stateful logic gates in larger circuits for in-memory logic. This implementation also opens the way through aggressive scaling to achieve one of the Feynman Grand Challenges - the construction of a functional nano-scale 8-bit adder in 50x50x50nm for which a circuit implementation is proposed
Recommended from our members
Toward Efficient Mixed-Signal Neural Processors Using Non-Volatile Memory Devices
Get PDFRecently, availability of big data and enormous processing power along with maturing of the applied algorithms, especially for deep learning, have led to a breakthrough in the performance of machine learning algorithms for various applications such as image classification, speech recognition, natural language processing, and so on. The rapidly growing range of machine learning applications, especially in IoT/mobile devices (smart phones, self-driving cars, virtual reality gadgets, medical devices, etc.), call for a specialized efficient neural processing platform. Of that, there is much more demand for fast, low-precision neural inference accelerators than for higher-precision systems for network training. Though custom-designed digital accelerators significantly outperform their conventional counterparts, their performance is limited due to inherent separation of storage and computing elements and their large footprint. Analog-domain in-memory computing using nonvolatile memory (NVM) devices appears to be a promising bio-inspired solution to break the performance barrier of today's neural accelerators. Small footprint of the storage cells, and their ability to operate as multiplier-and-accumulators enable compact, fast, and energy-efficient implementation of vector-by-matrix multipliers (VMMs) as the core processing element in the neural accelerators. In this work, we develop several compact, energy-efficient time-domain VMM approaches based on various NVM devices such as 1T1R resistive cells, 2D-NOR floating-gate devices, and 3D-NAND flash memories. In these approaches, the computation is solely performed via charge transfer while no static power is consumed in the peripheral circuitry. Moreover, the digital nature of the peripheral circuitry in these approaches significantly relaxes the technology node scaling limitation which is a serious challenge for analog-domain computing circuits. One of the key advantages of our time-domain approach is its full compatibility with complex and compact structure of the commercial 3D-NAND memory block enabling the implementation of ultra-compact mixed-signal neural operators based on this NVM platform. In the second phase of this work, we propose “aCortex”, an extremely energy efficient, fast, compact, and versatile neuromorphic processor architecture suitable for acceleration of a wide range of neural network inference models. The most important feature of our processor is a configurable mixed-signal computing array of vector-by-matrix multiplier (VMM) blocks utilizing embedded NVM arrays for storing weight matrices. In this architecture, power-hungry analog peripheral circuitry for data integration and conversion is shared among a very large array of VMM blocks enabling efficient instant analog/time-domain VMM operation for different neural layer types with a wide range of layer specifications. This approach also maximizes the processor’s area efficiency through sharing the area-hungry high-voltage programming switching circuitry as well as the analog peripheries among a large 2D array of NVM blocks. Such compact implementation further boosts the energy efficiency via lowering the digital data transfer cost. Other unique features of aCortex include configurable chain of buffers and data buses, a simple and efficient Instruction Set Architecture (ISA) and its corresponding multi-agent controller, and a customized refresh-free embedded DRAM memory. Using aCortex estimator, we perform rigorous system-level analysis targeting various NVM devices (1T1R ReRAM, and 2D-NOR/3D-NAND flash) as well as different computing approaches (current/time mode), and propose a roadmap for the future efforts in this domain
Going Beyond Counting First Authors in Author Co-citation Analysis
Get PDFThe present study examines one of the fundamental aspects of author co-citation analysis (ACA) - the way co-citation
counts are defined. Co-citation counting provides the data on which all subsequent statistical analyses and mappings
are based, and we compare ACA results based on two different types of co-citation counting - the traditional type that
only counts the first one among a cited work's authors on the one hand and a non-traditional type that takes into
account the first 5 authors of a cited work on the other hand. Results indicate that the picture produced through this non-traditional author co-citation counting contains more coherent author groups and is therefore considerably clearer. However, this picture represents fewer specialties in the research field being studied than that produced through the traditional first-author co-citation counting when the same number of top-ranked authors is selected and analyzed. Reasons for these effects are discussed
- …
