1,720,978 research outputs found
Recommended from our members
The Topology of Neural Population Activity in the Songbird Auditory System
Get PDFPerception depends upon the coordinated activity of populations of neurons. Howneural populations represent structure in the outside world through their activity is an
important open question. One approach to this problem is the concept of the receptive field,
which quantifies how external stimulation modulates the activity of individual neurons. While
receptive fields are a powerful concept for the experimenter, the brain itself does not have
access to its own receptive fields. This dissertation applies methods from the field of
algebraic topology to characterize the structure of neural population activity in the secondary
auditory region NCM of the European starling. Chapter 2 demonstrates that the simplicial
complex associated to population activity in NCM carries behaviorally-relevant information
xiiabout learned categories. Along the way, a new similarity measure for population activity is
developed, called the Simplicial Laplacian Spectral Entropy. It is shown that this measure
quantifies the similarity of simplicial complexes associated with neural activity in a way that
depends upon their global topological structure. Chapter 3 explores the connection between
neural topology and classical receptive fields, by showing that the intrinsic geometry of the
population activity matches the geometry of the receptive fields. This shows that the temporal
coativity structure of the population contains a direct representation of the stimulus structure
without requiring explicit computation of receptive fields. This validates a previously
described theoretical mechanism in a sensory system for the first time in-vivo, and provides a
new understanding for population activity in sensory regions. This chapter also introduces a
technique for reconstructing acoustic spectrograms of complex, natural vocal signals from
neural activity, which will be a powerful technique for exploring population level
representations in future studies. The final chapter of this dissertation describes a new
mathematical description of the phenomenon of polychronization in spiking neural networks.
This serves as a bridge between the experimental results of this dissertation and a theoretical
understanding of the emergence of spatiotemporal structure in the activity of neural
populations
Recommended from our members
Temporal organization in vocal communication: sequential structure, perceptual integration, and neural foundations
Get PDFOur interactions with the world unfold over time. Whether it's speaking, where one word follows the next, or walking, where each step follows another, the organization of our behaviors in time tends to follow a predictable pattern. Those patterns are dictated by a multitude of underlying factors, influenced both by endogenous physiological factors like the rhythmic nature of our gait as well as by exogenous factors, like the social dynamics underlying turn-taking while speaking.
Despite decades of research studying the temporal organization of behavior, dating back to the work of influential biologists like Tinbergen, Lashley, and Dawkins, little is known about the physiological substrates that underlie either the production of the sequential organization of most aspects of behavior.
Despite widespread acknowledgment that physiological motor programs and many non-linguistic behaviors are hierarchical, for example, few physiological investigations into the dynamics of behavior extend beyond low-order (Markovian) transition statistics. In this thesis, I build onto the emerging field of computational neuroethology to further our understanding of what structure underlies the sequential organization of behavior, what physiological mechanisms might be involved in producing, perceiving, and representing sequential behavioral organization, and how sequential behavioral organization might have emerged developmentally and evolutionarily. Throughout the thesis, I draw primarily upon birdsong and human speech, developing methods to analyze the acoustic and temporal structure in vocal signals and then behaviorally and physiologically probing the underpinnings of sequential organization in the songbird. This work advances the field of computational neuroethology in several ways.I uncover novel acoustic structure in vocal signals separating avian and mammalian vocalizations along a spectrum of vocal stereotypy.
I observe that both human speech and birdsong are characterized by a combination of long and short-range temporal patterning.
I find that the long-range temporal patterning characterizing human speech, believed to be underlied by hierarchical linguistic organization, is present at the earliest developmental stages of human speech, well before complex syntax is produced.
I find that the perceptual integration of birdsong syllable sequences can be well explained by Bayesian models of probabilistic perceptual decision-making.
Finally, I find that sensory neural representations of syllable sequences are modulated by sequential context and that this modulation reflects the animals underlying perceptual behavior.
In the following paragraphs, I give a brief overview of the methods and major results of the chapters comprising this thesis.In Chapter \ref{chapter:review} I give an introduction to the emerging field of vocal computational neuroethology. This introduction contextualizes the following chapters in a review of current work. I emphasize current tools, challenges, and future directions in vocal neuroethology. I start with a discussion of low-level bioacoustics challenges and build up to a discussion of behavioral organization and physiology. I first discuss challenges in signal processing such as dealing with noise and signals and representing vocal signals as time-frequency representations. I then discuss machine learning approaches used to identify, segment, and label vocalizations. Next, I discuss how to extract relational structure between vocalizations, and cluster latent projections of vocalizations. I then give an overview of methods for capturing temporal relationships in vocal sequences, outlining traditional Markovian descriptions of vocal structure, and new tools for capturing long-range structure, enabled by large datasets. I then move on to machine learning tools that can be used to systematically control and synthesize vocal signals from learned vocal spaces. Finally, I discuss how these techniques are being utilized in several active areas of neuroethology research. In Chapter \ref{chapter:avgn} I develop a set of methods to visualize and quantify relational structure in vocalizations, which enable the analyses and experiments performed in the following chapters. I use graph-based dimensionality reduction to uncover local structure in vocal communication signals and apply that technique to 19 datasets consisting of vocalizations from 29 species, including songbirds, primates, cetaceans, rodents, and bats. I observe that these methods uncover novel structure in animal vocal signals, including vocal dialects, acoustic units, behaviorally relevant signal information, and sub-syllabic structure. In Chapter \ref{chapter:parametric_umap}, I extend the methods from Chapter \ref{chapter:avgn} by introducing Parametric UMAP, a graph-based dimensionality reduction algorithm that parametrically learns the relationship between data (here vocal signals) and latent embeddings. Parametric UMAP enables the methods from Chapter \ref{chapter:avgn} to be applied in real-time closed-looped settings over larger datasets due to the learned parametric embeddings. I show that this algorithm has applications in semi-supervised settings, and provides additional control over the trade-off between capturing global and local structure in embeddings. In Chapter \ref{chapter:parallels} I explore the long and short-range temporal patterning of vocal sequences in birdsong and human speech. I use an information-theoretic framework to analyze statistical dependencies as a function of the distance between elements in vocal sequences. I find that both birdsong and human speech exhibit two forms of structure: short-range relationships captured by Markovian dynamics over short-timescales, and long-range relationships that follow a power-law occurring over longer timescales. In language, the observed short-range organization conforms to phonological processes, which are well-described by finite-state dynamics, while long-range organization suggests more complex dynamics such as underlying hierarchical organization. Previous analyses of birdsong have only identified short-range Markovian dynamics, making our observation of long-range dynamics in birdsong novel. In Chapter \ref{chapter:lri} I extend our experiment from chapter \ref{chapter:parallels} over human speech to language acquisition. By analyzing corpora of speech throughout language development, we can observe the time course of the emergence of long and short-range relationships over development. Surprisingly, I find that long-range statistical dependencies are present in children's speech as early as 6-12 months, well before complex syntactic structure is present. I discuss these results alongside emerging evidence from computational ethology that long-range relationships are also common to non-linguistic behavioral signals from animals as diverse as zebrafish, drosophila, and whales. Although previous analyses of long-range relationships have suggested that long-range relationships are the product of hierarchical linguistic structure such as syntax and discourse structure, our observations in developmental speech and non-linguistic behaviors suggest that other mechanisms may also be at play. Finally, in Chapter \ref{chapter:cdcp} I probe how sequential dependencies in vocal sequences are integrated behaviorally and physiologically. I developed a behavioral task in which European starlings are trained to classify morphs of syllables of starling song synthesized from an interpolation between two points in the latent space of a neural network (a Variational Autoencoder). These morph syllables are preceded with a separate syllable (a cue syllable), which holds predictive information about the category of the following morph syllable. I find that classification of the morph syllable is contextually modulated by the predictive probability of the cue syllable, which can be well explained by a model of Bayesian integration. With the same behavioral paradigm, I then record chronic electrophysiology data from auditory nuclei while birds performed this context-dependent categorical perceptual decision-making task. I find that neural activity patterns reflect several aspects of our model of perceptual behavior, including the uncertainty in decision making, and prediction-related perceptual modulation
Recommended from our members
Predictive Coding in the Auditory Cortex
Get PDFCharacterization of response properties of neurons in higher-level sensory areas is not well defined. Here we show that firing rates of neurons in a secondary sensory forebrain area of songbirds can be modeled by different representations of birdsong. In this work, we modeled neurons in the caudo-medial nidopallium (NCM) of adult European starlings with three different representations of the natural birdsong called signal, prediction, and error. Prediction spectrogram was computed by training the data as a Gaussian distribution on a loss function given by the negative log likelihood, and then estimating the means and variances of the signal. Using our Maximum Noise Entropy (MNE) model, responses were predicted by the logistic function, the parameters of which are obtained from the MNE model. Predictions of neural responses were computed by using both a full MNE model, and then by only considering the linear parameters of the model. The neural responses to natural stimuli obtained using prediction and error MNEs were close to the actual response in the NCM. The concept of stimulus representations obtained from predictive coding models may be useful for modeling neural responses in higher-order sensory areas whose functions have been poorly understood
Recommended from our members
Contrast, Neutralization, and Systems of Invariance
Get PDFSpeech is a highly-structured auditory signal that carries necessary and sufficient information for language comprehension among neurotypical, hearing populations. Speech perception, accordingly, involves the integration of both bottom-up auditory processing and top-down language knowledge. As explored in Chapter 1, however, while much is known about the brain's sensitivity to various acoustic and phonological manipulations, mechanistic transformative links between auditory sensation and linguistic perception remain poorly understood.This dissertation contributes three studies to the ongoing pursuit of links between auditory and linguistic processing of speech in the brain. Intracranial stereo electroencephalography (sEEG) was recorded while participants listened to conversational English speech, and for each study, different sets of analyses were performed to target different aspects of these links.Chapter 2 introduces a novel comparative technique for the isolation of phonological and morphological identity from the auditory speech signal. Through structured comparisons of the neural response to sounds in allophonic relationships with one another, sites that dissociate phonemic and morphological identity from acoustic similarity were identified. These results provide crucial evidence that the neural response to speech is sensitive to abstract sub-lexical structure.Chapter 3 provides general models to account for the unique contributions of categorical phonemic information and spectrographic information to the neural response. In lower frequency bands, phonemic category information explained a greater proportion of the neural response variance than spectrographic information. These results indicate that the explanatory value of phonemic category features in neural encoding models is not reducible to the surface spectrotemporal features that correlate with those categories.Chapter 4 proposes a model for the relationship between the speech stimulus and neural response that integrates both phonemic and spectrographic information. Inclusion of the stimulus covariance structure increased model accuracy only when categorical phonemic information was available, indicating that phoneme-related neural activity is likely mediated through the acoustic covariance structure of speech. Moreover, phonemic label information conferred no benefit to model fit when participants listened to speech in an unfamiliar language (Catalan). Together these studies show that neural responses associated with categorical phonemic information are language specific and irreducible to speech acoustics, and they scout a path towards mechanistic, transformative links between auditory sensation and language cognition
Recommended from our members
Establishing Songbirds as an Animal Model for the Development of Human Speech Prosthesis
Get PDFLosing the ability to speak—whether from stroke, traumatic brain injury, or other neurological disorders— significantly reduces a person’s quality of life. Research studies demonstrate proof-of-concept Speech-Synthesis Brain-Machine Interface (BMI) systems, but several limitations impede their clinical viability. A major rate limiting factor impeding progress in developing speech prosthesis is the lack of an established animal model to ask basic science questions regarding the neural encoding of vocal communication. This dissertation aims to address this gap by establishing songbirds as an animal model for a human speech prosthesis. Songbirds are a well-established model for vocal learning, and their motor nuclei are homologous to the human motor cortex with respect to function and gene transcription. This work builds upon this basis to demonstrate songbirds’ suitability as a preclinical model to accelerate the development of speech-prosthesis technology.First, we answer basic science questions regarding nuclei important for the song system by characterizing neurophysiological similarities and differences with respect to human motor areas during vocalization. In analyses of data recorded with electrodes chronically implanted in the premotor region HVC of awake free-behaving zebra finches, we detail novel Local Field Potential (LFP) signatures correlated to vocal behavior. These LFP signatures are decomposed using signal processing methods to characterize their relation to vocal production. This work found that HVC exhibits many remarkably similar spectral characteristics to LFP in human motor cortex during speech.
Next, we developed proof-of-concept systems that demonstrate algorithms feasible for real time vocal BMIs. Utilizing simple algorithms, we show that HVC LFP features can be leveraged to predict vocal activity. These algorithms can be run in real time to predict both the identity and onset of syllable production. Leveraging these simple algorithms, we analyze preliminary system requirements necessary for decoding vocal elements. The methods developed to leverage these LFP features to predict vocal behavior can be implemented in real-time and suggest a path for developing a similar system for humans.
Finally, this thesis details both software and hardware designs to enable reproduction and wider adoption of the songbird animal model by the speech prosthesis research field. We developed novel methods to partition freely produced vocal behavior data based on the subjects’ behavior, which are provided to the field as open-source software. We also designed an integrated counterweight and tether management system that dramatically lowers the stress on chronically implanted small animal subjects.
Collectively, these works enrich the literature connecting human and avian vocal-motor production, and we believe strengthen the argument for utilizing songbirds to supplement human speech prosthesis research
Recommended from our members
Tools to investigate composite receptive fields in songbird auditory region
Get PDFNeural coding is primarily concerned with characterizing the relationship between stimulus and neuronal responses and is classified to stimuli encoding and brain response decoding. Although there are existing models for neural coding, most are not sophisticated enough to describe the relationship of population of neural responses to natural stimuli such as human speech or bird songs. In this study we propose utilizing composite receptive fields (CRF) as a new tool for neural coding. CRFs are quadratic receptive fields which are built from mutual information between stimuli and related brain responses. Here we create a pool of 3080 CRFs from a population of 154 cells recorded from the brain auditory region of European starling songbirds. Following, this pool of CRFs is used to build a spatial-temporal map for the population of cells along the brain coronal plane in respect to bird song stimuli. This map has revealed novel information about the relationship between neural responses and related stimuli such as: 1) Natural sound stimuli can be encoded by populations of neurons. 2) The number of cells needed to encode the stimuli can be quantified. 3) Stimuli encoding mechanisms of the brain appeared to be uniform and independent of cells’ topology and their locations. 5) From this map, connectivity between cells as well as their response plasticity to diverse stimuli were observed. 6) CRFs were used as intermediate tools to reconstruct stimuli and predict brain responses. These results have confirmed that quadratic receptive fields can be a novel candidate for population neural coding. Testing neural coding by CRFs was originally performed on cells recorded from the brain coronal depth plane. We expanded this coding method and evaluated CRFs mapping on cells recorded from two novel in-house fabricated electrodes: a surface electrode and a combination of surface and depth electrodes. The CRFs extracted from cells recorded by these electrodes can be employed to create a 2D and 3D spatial-temporal map which is useful to explore neural information distribution and their perception mechanisms from deep brain to cortical surface. Furthermore, the CRF neural coding method and the brain implants described in this study have potentials to be used in BCI prosthetics
European Starling categorical perception chronic ephys and behavior dataset
No full textThis dataset corresponds to the code at https://github.com/timsainb/cdcp_paper
Please refer to the readme for this GitHub repo, which contains all the necessary information for reproducing our analyses or using this data for additional analyses.
If you use this dataset, please cite the most recent (published) version of our paper.
arxiv paper:
@article {Sainburg2022.04.14.488412,
author = {Sainburg, Tim and McPherson, Trevor S and Arneodo, Ezequiel M. and Rudraraju, Srihita and Turvey, Michael and Thielman, Brad and Marcos, Pablo Tostado and Thielk, Marvin and Gentner, Timothy Q},
title = {Context-dependent sensory modulation underlies Bayesian vocal sequence perception},
elocation-id = {2022.04.14.488412},
year = {2022},
doi = {10.1101/2022.04.14.488412},
publisher = {Cold Spring Harbor Laboratory},
abstract = {},
URL = {https://www.biorxiv.org/content/early/2022/04/15/2022.04.14.488412},
eprint = {https://www.biorxiv.org/content/early/2022/04/15/2022.04.14.488412.full.pdf},
journal = {bioRxiv}
Going Beyond Counting First Authors in Author Co-citation Analysis
Get PDFThe present study examines one of the fundamental aspects of author co-citation analysis (ACA) - the way co-citation
counts are defined. Co-citation counting provides the data on which all subsequent statistical analyses and mappings
are based, and we compare ACA results based on two different types of co-citation counting - the traditional type that
only counts the first one among a cited work's authors on the one hand and a non-traditional type that takes into
account the first 5 authors of a cited work on the other hand. Results indicate that the picture produced through this non-traditional author co-citation counting contains more coherent author groups and is therefore considerably clearer. However, this picture represents fewer specialties in the research field being studied than that produced through the traditional first-author co-citation counting when the same number of top-ranked authors is selected and analyzed. Reasons for these effects are discussed
Variations on the Author
Get PDF“Variations on the Author” discusses two of Eduardo Coutinho’s recent films (Um Dia na Vida, from 2010, and Últimas Conversas, posthumously released in 2015) and their contribution to the general question of documentary authorship. The director’s filmography is characterized by a consistent yet self-effacing form of authorial self-inscription: Coutinho often features as an interviewer that rather than express opinions propels discourses; an interviewer that is good at listening. This mode of self-inscription characterizes him as an author who is not expressive but who is nonetheless markedly present on the screen. In Um Dia na Vida, however, Coutinho is completely absent form the image, while Últimas Conversas, on the contrary, includes a confessional prologue that moves the director from the margins to the center of his films. This article examines the ways in which these works stand out in the filmography of a director who offers new insights into the notion of cinematic authorship
- …
