46 research outputs found
Investigating the Indirect Object Identification circuit in Mamba
No full textHow well will current interpretability techniques generalize to future models? A relevant case study is Mamba, a recent recurrent architecture with scaling comparable to Transformers. We adapt pre-Mamba techniques to Mamba and partially reverse-engineer the circuit responsible for the Indirect Object Identification (IOI) task. Our techniques provide evidence that 1) Layer 39 is a key bottleneck, 2) Convolutions in layer 39 shift names one position forward, and 3) The name entities are stored linearly in Layer 39\u27s SSM. Finally, we adapt an automatic circuit discovery tool, positional Edge Attribution Patching, to identify a Mamba IOI circuit. Our contributions provide initial evidence that circuit-based mechanistic interpretability tools work well for the Mamba architecture
INSTRUCTION SET ARCHITECTURE OF MAMBA, A NEW VIRTUAL MACHINE FOR PYTHON
No full textPython programs are implemented by compiling them into code for
a virtual machine. Mamba is a new virtual machine for Python whose design
is minimalist and register-based. In contrast, the current Python virtual
machine is stack-based and contains almost six times as many instructions as
Mamba. We anticipate that our smaller instruction set will greatly simplify
optimization implementation. In this paper we detail Mamba's instruction
set and execution model.We are currently acquiring citations for the work deposited into this collection. We recognize the distribution rights of this item may have been assigned to another entity, other than the author(s) of the work.If you can provide the citation for this work or you think you own the distribution rights to this work please contact the Institutional Repository Administrator at [email protected]
MARCA: Mamba Accelerator with ReConfigurable Architecture
No full textWe propose a Mamba accelerator with reconfigurable architecture, MARCA.We propose three novel approaches in this paper. (1) Reduction alternative PE array architecture for both linear and element-wise operations. For linear operations, the reduction tree connected to PE arrays is enabled and executes the reduction operation. For element-wise operations, the reduction tree is disabled and the output bypasses. (2) Reusable nonlinear function unit based on the reconfigurable PE. We decompose the exponential function into element-wise operations and a shift operation by a fast biased exponential algorithm, and the activation function (SiLU) into a range detection and element-wise operations by a piecewise approximation algorithm. Thus, the reconfigurable PEs are reused to execute nonlinear functions with negligible accuracy loss.(3) Intra-operation and inter-operation buffer management strategy. We propose intra-operation buffer management strategy to maximize input data sharing for linear operations within operations, and inter-operation strategy for element-wise operations between operations. We conduct extensive experiments on Mamba model families with different sizes.MARCA achieves up to 463.22/11.66 speedup and up to 9761.42/242.52 energy efficiency compared to Intel Xeon 8358P CPU and NVIDIA Tesla A100 GPU implementations, respectively.9 pages, 10 figures, accepted by ICCAD 2024. arXiv admin note: text overlap with arXiv:2001.02514 by other author
MTMamba: Enhancing Multi-task Dense Scene Understanding by Mamba-Based Decoders
No full textMulti-task dense scene understanding, which learns a model for multiple dense prediction tasks, has a wide range of application scenarios. Modeling long-range dependency and enhancing cross-task interactions are crucial to multi-task dense prediction. In this paper, we propose MTMamba, a novel Mamba-based architecture for multi-task scene understanding. It contains two types of core blocks: self-task Mamba (STM) block and cross-task Mamba (CTM) block. STM handles long-range dependency by leveraging Mamba, while CTM explicitly models task interactions to facilitate information exchange across tasks. Experiments on NYUDv2 and PASCAL-Context datasets demonstrate the superior performance of MTMamba over Transformer-based and CNN-based methods. Notably, on the PASCAL-Context dataset, MTMamba achieves improvements of +2.08, +5.01, and +4.90 over the previous best methods in the tasks of semantic segmentation, human parsing, and object boundary detection, respectively. The code is available at https://github.com/EnVision-Research/MTMamba. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025
HMT-UNet: A hybird Mamba-Transformer Vision UNet for Medical Image Segmentation
No full textIn the field of medical image segmentation, models based on both CNN and Transformer have been thoroughly investigated. However, CNNs have limited modeling capabilities for long-range dependencies, making it challenging to exploit the semantic information within images fully. On the other hand, the quadratic computational complexity poses a challenge for Transformers. State Space Models (SSMs), such as Mamba, have been recognized as a promising method. They not only demonstrate superior performance in modeling long-range interactions, but also preserve a linear computational complexity. The hybrid mechanism of SSM (State Space Model) and Transformer, after meticulous design, can enhance its capability for efficient modeling of visual features. Extensive experiments have demonstrated that integrating the self-attention mechanism into the hybrid part behind the layers of Mamba\u27s architecture can greatly improve the modeling capacity to capture long-range spatial dependencies. In this paper, leveraging the hybrid mechanism of SSM, we propose a U-shape architecture model for medical image segmentation, named Hybird Transformer vision Mamba UNet (HTM-UNet). We conduct comprehensive experiments on the ISIC17, ISIC18, CVC-300, CVC-ClinicDB, Kvasir, CVC-ColonDB, ETIS-Larib PolypDB public datasets and ZD-LCI-GIM private dataset. The results indicate that HTM-UNet exhibits competitive performance in medical image segmentation tasks. Our code is available at https://github.com/simzhangbest/HMT-Unet.arXiv admin note: text overlap with arXiv:2403.09157; text overlap with arXiv:2407.08083 by other author
Writing Across Languages and Cultures. Nadifa Mohamed's Black Mamba Boy
No full textWith her debut novel Black Mamba Boy, published in 2010 to much acclaim, Nadifa Mohamed has articulated in the increasingly diverse and thriving landscape of contemporary multicultural English fiction a distinct British Somali voice. Her work situates within an expanding body of literary works that reflects the multiplicity of the non-native cultures within multicultural Britain and has emerged out of the particular predicament of both first and second generation immigrant English writers.This contribution discussesthe ways in which the author weaves a tapestry of languages, cultures, personal and communal histories, aiming to challenge an enduring dynamic of marginal, peripheral stories disappearing from history. In Black Mamba boy Mohamed revisits the western tradition of the Bildungsroman novel and incorporates strategies of the African oral narrative, as she chronicles the coming of age process of her protagonist Jama. The initiation novel, concerned with the education, development, and maturing of a young protagonist has often been favoured by postcolonial and bicultural writers to tell their stories of personal growth, within a context of migration and displacement
Mamba - A waterproof snake robot with tactile sensing
No full text- author postprintThis paper presents the snake robot Mamba, which is a modular, reconfigurable, and waterproof experimental platform developed to support the ongoing research on snake robot locomotion, including underwater locomotion. A novel contribution of the snake robot is its ability to measure environment contact forces acting along its body, which is achieved by isolating the actuator inside each joint module with a custom-designed force/torque sensor. The paper describes the design of this sensor and presents experimental results which illustrate its performance.(c) 2014 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works
Model-based assessment of replicability for genome-wide association meta-analysis
No full textGenome-wide association meta-analysis (GWAMA) is an effective approach to enlarge sample sizes and empower the discovery of novel associations between genotype and phenotype. Independent replication has been used as a gold-standard for validating genetic associations. However, as current GWAMA often seeks to aggregate all available datasets, it becomes impossible to find a large enough independent dataset to replicate new discoveries. Here we introduce a method, MAMBA (Meta-Analysis Model-based Assessment of replicability), for assessing the “posterior-probability-of-replicability” for identified associations by leveraging the strength and consistency of association signals between contributing studies. We demonstrate using simulations that MAMBA is more powerful and robust than existing methods, and produces more accurate genetic effects estimates. We apply MAMBA to a large-scale meta-analysis of addiction phenotypes with 1.2 million individuals. In addition to accurately identifying replicable common variant associations, MAMBA also pinpoints novel replicable rare variant associations from imputation-based GWAMA and hence greatly expands the set of analyzable variants.</p
