82 research outputs found
Introducing Local Weight Autocorrelation in Deep Neural Networks leads to the Emergence of Orientation Columns and Face Patches
No full textSmoothly varying orientation preference maps and category-selective regions are well-known prop-
erties of the primate visual system. With the aim of reproducing these properties in arti cial neural
network models of the ventral stream, we introduce the Hypercolumn layer, which is a subclass of the
locally-connected layer and an abstraction of the in
uential cortical Hypercolumn model. Our Hyper-
column layer minimizes the spatial distance of the layer weight vectors, which by de nition optimizes
a local weight autocorrelation. We show that deep neural networks using Hypercolumn layers produce
smooth orientation preference maps in shallow layers and category-selective regions in deeper layers when
trained on a categorical classi cation task. The strength of category selectivity is proportionate to the
degree of visual expertise that the model has with the category. We nd no substantial accuracy bene ts
of optimizing weight autocorrelation, although it reduces over tting. The biological implications of the
models are thoroughly discussed
Topographic Neural Networks show neural recycling of labile units during reading acquisition
No full textHere, we ask the question of how the formation of brain regions and changes in neuron-level specialisation
depend on environmental input changes by modelling neural recycling in a topographic
neural network. During reading acquisition, a region in the left hemisphere of the ventral visual
stream of the human brain emerges that is sensitive to written words. The emergence of this area,
the visual word form area, is strongly experience-dependent, but its predictable location points to an
underlying organisational principle that guides its emergence. It is thus hypothesised that it arises
due to ‘recycling’ of neurons that were previously not or weakly involved in object recognition. To
uncover more about the spatial changes during the formation of functionally specialised areas in the
brain, we use use a topographic neural network (TNN) to model neural recycling. To simulate reading
acquisition, we first train the network on a large-scale dataset of natural images (preliterate phase).
Next, the network is trained for 50 epochs more on word images as well (literate phase). We confirm
recycling of ‘labile’ units (non-selective to any category) and face-selective units to word-selective
units after reading acquisition. Word-selective units cluster together, especially in the later layers of
the network. We also confirm the destructive e↵ect of neural recycling on the performance on other
classes of stimuli. We conclude that the TNN serves well as a model of neural recycling, as it captures
various features of neural recycling in the human visual cortex
From visual representations to perceptual decisions: investigating neural mechanisms underlying visual recognition and perceptual decision making
No full textTo successfully interact with the environment, humans must robustly encode sensory inputs into neural representations and translate these representations into adaptive behavior.
In this thesis, I conducted three empirical studies targeting particular neural mechanisms
entailed in this transition from sensory inputs to perceptual decisions.
In Study 1, I probed the robustness of core visual processing mechanisms by characterizing and comparing the neural dynamics of object recognition for natural photographs
and abstracted line drawings. This revealed that core visual processing mechanisms in the
brain are robust to the abstraction of substantial amounts of visual information, such as in
line drawings.
In Study 2, I investigated the influence of feedback processing on object representations by
comparing the neural dynamics of object recognition for stimuli that were either rapidly
followed by a masking stimulus or followed only after a substantial delay. This revealed
that feedback processing fundamentally shapes visual representations in the brain, first
in early than in high-level visual cortex. Feedback enhances the persistence of visual
representations, causes a shift in the representational format in high-level visual cortex,
and affects distinct spectro-temporal windows in the theta to beta frequency bands.
Finally, in Study 3, I examined the link between neural representations of real-world scenes
and behavior under varying task demands. The findings showed that distinct visual representations in the brain are behaviorally relevant depending on the task, that mid-level
visual features underlie these behaviorally relevant representations, and that visual representations can interfere with behavior given task demands that do not align with the
represented information.
By demonstrating the robustness of core visual processing mechanisms to visual abstractions and by characterizing how feedback processing dynamically shapes visual processing, the findings in Study 1 and 2 provide complimentary insights into the neural mechanisms that enable robust encoding of visual information. By identifying and characterizing visual representations relevant for behavior across different task demands, Study 3 provides novel insights into the translation of sensory information into perceptual decisions. Collectively, these results contribute to a large body of research on visual recognition and perceptual decision making, provide potential new theoretical frameworks for understanding the underlying mechanisms, and guide
the way for future research that directly tests and refines these theories.Um erfolgreich mit der Umwelt zu interagieren, müssen Menschen sensorische Reize robust enkodieren und die resultierenden Repräsentationen in Verhalten umwandeln. In
dieser Dissertation habe ich drei empirischen Studie durchgeführt, um spezifische neuronale Mechanismen zu untersuchen, die diesem Übergang von sensorischen Reizen zu
Wahrnehmungsentscheidungen zugrunde liegen.
In Studie 1 habe ich die Robustheit grundlegender visueller Verarbeitungsmechanismen untersucht, indem ich die neuronale Dynamik der Objekterkennung für natürliche Fotografien und abstrahierte Strichzeichnungen charakterisiert und verglichen habe. Die
Ergebnisse zeigten, dass grundlegende visuelle Verarbeitungsmechanismen robust gegenüber der Abstraktion großer Mengen visueller Informationen sind, wie es in Strichzeichnungen der Fall ist.
In Studie 2 untersuchte ich den Einfluss von Feedbackverarbeitung auf Objektrepräsentationen, indem ich die neuronale Dynamik der Objekterkennung für Stimuli verglich, die
entweder schnell von einem Maskierungsreiz gefolgt wurden oder erst nach einer längeren
Verzögerung. Die Ergebnisse zeigten, dass Feedback die visuellen Repräsentationen im
Gehirn grundlegend beeinflusst, zunächst im frühen und später im höheren visuellen Kortex. Feedback erhöht die Beständigkeit visueller Repräsentationen, führt zu einer Veränderung im Repräsentationsformat im höheren visuellen Kortex und beeinflusst spezifische
spektral-temporale Fenster in den Theta- bis Beta-Frequenzbändern.
Abschließend habe ich in Studie 3 den Zusammenhang zwischen neuronalen Repräsentationen realer Szenenbilder und dem Verhalten während verschiedener Aufgaben untersucht. Die Ergebnisse zeigten, dass unterschiedliche visuelle Repräsentationen im Gehirn
je nach Aufgabe verhaltensrelevant sind, dass visuelle Informationen mittlerer Komplexität diesen Repräsentationen zugrunde liegen und dass visuelle Repräsentationen mit Verhalten interferieren können, wenn die Aufgabenanforderungen nicht mit der repräsentierten Information übereinstimmen.
Durch die Demonstration der Robustheit grundlegender visueller Verarbeitunsgsmechanismen gegenüber visueller Abstraktion und die Charakterisierung, wie Feedback visuelle Verarbeitung dynamisch formt, liefern die Ergebnisse aus Studie 1 und 2 komplementäre Einblicke in die neuronalen Mechanismen, die die robuste Enkodierung visueller Informationen ermöglichen. Durch die Identifizierung und Charakterisierung verhaltensrelevanter visueller Repräsentationen, liefert Studie 3 neue Erkenntnisse über die
Umwandlung von sensorischen Reizen in Wahrnehmungsentscheidungen. Insgesamt tragen diese Ergebnisse zu einem breiten Forschungsfeld zu visueller Objekterkennung und Wahrnehmungsentscheidungen bei, liefern neue theoretische Anhaltspunkte für das Verständnis der zugrundeliegenden neuronalen Mechanismen und können zukünftige Forschung
anleiten, die diese Theorien direkt testet
Crowding and the Architecture of the Visual System
No full textClassically, vision is seen as a cascade of local, feedforward computations. This framework has been tremendously successful, inspiring a wide range of ground-breaking findings in neuroscience and computer vision. Recently, feedforward Convolutional Neural Networks (ffCNNs), a kind of deep neural network inspired by this classic framework, have revolutionized computer vision and been adopted as tools in neuroscience. However, despite these successes, there is much more to vision. First, there are flagrant architectural differences between biological systems and the classic framework. For example, recurrence is abundant in the brain but absent from the classic framework and ffCNNs. Although there is widespread agreement about the importance of these recurrent connections, their computational role is still poorly understood. Second, these architectural differences lead to behavioural differences too, highlighted by psychophysical evidence. Relatedly, ffCNNs are extremely vulnerable to small changes to their inputs and do not generalize well beyond the dataset used to train them. Human vision, in contrast, is much more robust. New insights are needed to face up to these challenges.
In this thesis, I use visual crowding and related psychophysical effects as probes into visual processes that go beyond the classic framework. In crowding, perception of a target deteriorates in clutter. I focus on global aspects of crowding, in which perception of a small target is strongly modulated by the global configuration of elements across the visual field. I show that models based on the classic framework, including ffCNNs, cannot explain these effects for principled reasons and identify recurrent grouping and segmentation as a key missing ingredient. Then, I show that capsule networks, a recent kind of deep learning architecture combining the power of ffCNNs with recurrent grouping and segmentation, naturally explain these effects. I provide psychophysical evidence that humans indeed use a similar recurrent grouping and segmentation strategy in global crowding effects.
In crowding, visual elements interfere across space. To study how elements interfere over time, I use the Sequential Metacontrast psychophysical paradigm, in which perception of visual elements depends on elements presented hundreds of milliseconds later. I psychophysically characterize the temporal structure of this interference and propose a simple computational model. My results support the idea that perception is a discrete process. I lay out theoretical implications of these findings. Together, the results presented here provide stepping-stones towards a fuller understanding of the visual system by suggesting architectural changes needed for more human-like neural computations.LPS
Implementing Fixational Eye Movement in a Recurrent Neural Network using Reinforcement Learning to achieve Super Resolution
No full textMy experiment explored Fixational Eye Movements' benefits in a Classification task with input with partially destroyed information. Rucci et al. findings suggest that FEMs play a crucial role in visual perception, especially when it comes to high-frequency data. Only a small part of the retina called the foveola can capture the full details of high-frequency input for many organisms like humans. To see in high-resolution with this physical limitation, we have to efficiently shift the input over this sensitive area. This unconscious process is called FEMs. In my experiment, I succeeded in learning an RNN the benefits of FEMs via Reinforcement learning. I could replicate some of Rucci et al. results and find evidence that FEM's are not just a useless bug
A new empirical challenge for local theories of consciousness
No full textLocal theories of consciousness state that one is conscious of a feature if it is adequately represented and processed in sensory brain areas, given some background conditions. We challenge the core prediction of local theories based on long-lasting postdictive effects demonstrating that features can be represented for hundreds of milliseconds in perceptual areas without being consciously perceived. Unlike previous empirical data aimed against local theories, localists cannot explain these effects away by conjecturing that subjects are phenomenally conscious of features that they cannot report. We also discuss alternative explanations that localists could offer
Neurophenomenal structuralism and the role of computational context
No full textNeurophenomenal structuralism posits that conscious experiences are defined relationally and that their phenomenal structures are mirrored by neural structures. While this approach offers a promising framework for identifying neural correlates of the contents of consciousness, we argue that merely establishing structural correspondences between neural and phenomenal structures is insufficient. This paper emphasizes the critical role of computational context – the network of neural processes within which a given neural activation pattern is used – in determining content. We introduce four criteria to evaluate if neural structures are viable candidates for neural correlates of contents of consciousness within this framework. These criteria highlight that, for neural structures to mirror phenomenal structures meaningfully, they must be actively exploited by the brain’s downstream processes and influence behavior in a structure-preserving way. Our analysis demonstrates that purely anatomical and overly exhaustive causal structures fail to meet specific criteria, whereas activation structures can succeed, provided they are embedded within the appropriate computational context. Our findings challenge local structuralist theories, which overlook the content-constituting role of computational context. We conclude that incorporating computational context is essential for any structuralist account of consciousness
First-person experience cannot rescue causal structure theories from the unfolding argument
No full textWe recently put forward an argument, the Unfolding Argument (UA), that integrated information theory (IIT) and other causal structure theories are either already falsified or unfalsifiable, which provoked significant criticism. It seems that we and the critics agree that the main question in this debate is whether first-person experience, independent of third-person data, is a sufficient foundation for theories of consciousness. Here, we argue that pure first-person experience cannot be a scientific foundation for IIT because science relies on taking measurements, and pure first-person experience is not measurable except through reports, brain activity, and the relationship between them. We also argue that pure first-person experience cannot be taken as ground truth because science is about backing up theories with data, not about asserting that we have ground truth independent of data. Lastly, we explain why no experiment based on third-person data can test IIT as a theory of consciousness. IIT may be a good theory of something, but not of consciousness. We conclude by exposing a deeper reason for the above conclusions: IIT’s consciousness is by construction fully dissociated from any measurable thing and, for this reason, IIT implies that both the level and content of consciousness are epiphenomenal, with no causal power. IIT and other causal structure theories end up in a form of dissociative epiphenomenalism, in which we cannot even trust reports about first-person experiences. But reports about first-person experiences are taken as ground truth and the foundation for IIT’s axioms. Therefore, accepting IIT leads to rejecting its own axioms. We also respond to several other criticisms against the UA.LPS
Why our best theories of perception lead to Anti-reductionism
No full textBasic percepts and observation sentences, such as "the voltmeter is at 7A", provide the ground truth for realistic theories. Reduction is the second backbone of these theories linking, for example, neuroscience to physics. First, we will show, by mathematical proof, that reduction is impossible if ontology is complex. We will provide a toy example which illustrates this point: a hypothetical animal has a sensor, which reacts to red and green light only. When red light is presented, the animal deterministically lifts the right back limb. For green light, the left one. Inputs and outputs are causally linked by a “brain” with just a few (binary) neurons. Even though inputs, outputs, and brain activity are fully available for millions of observations of a “scientist”, it is impossible to decode the output from the brain activity. Hence, neither sensation nor motor actions can be reduced to the underlying neural activity- even though input and output are perfectly correlated. Next, we outline the challenges any perceptual system needs to meet. For example, the light (luminance), which arrives at a photo-receptor of the retina, is a combination of the light shining on an object (illuminance) and the material properties of the object (reflectance). For a given luminance value, there are infinitely many illuminance-reflectance pairs, giving rise to this luminance value (an ill-posed problem). Hence, perception cannot be based on the raw input values. Second, we show how perceptual systems can solve such ill-posed problems. One conclusion of this analysis is that perception is inherently subjective, i.e., the metric of the perceptual system is not isomorphic to the metric of the physical space. We will argue that perception has evolved subjective metrics to exactly cope with the abundant complexity of the physical, non-reducible external world. In conclusion, we propose that reduction is neither possible nor desirable.LPS
- …
