1,721,017 research outputs found
Rethinking pose estimation in crowds: overcoming the detection information bottleneck and ambiguity
No full text<p>#############</p><p>Rethinking pose estimation in crowds: overcoming the detection information-bottleneck and ambiguity, ICCV 2023</p><p>#############</p><p> </p><p>Authors: Zhou, Mu and Stoffl, Lucas and Mathis, Mackenzie Weygandt and Mathis, Alexander</p><p>Affiliation: EPFL</p><p>Date: October, 2023</p><p> </p><p>Here we provide neural networks weights for the best models in our article "Rethinking pose estimation in crowds: overcoming the detection information-bottleneck and ambiguity", ICCV 2023. Each model has the naming convention "dataset"-"modeltype".pth</p><p>These pth files can be loaded with PyTorch. The code to load and use the models is available at The code to load and use the models is available at: <a href="https://github.com/amathislab/BUCTD">https://github.com/amathislab/BUCTD</a></p><p><strong>Note: The weights for OCHuman</strong>, are called COCO-* as one only trains on COCO. So OCHuman-X := COCO-X</p><p>We also share the predictions from various bottom-up models to reproduce the training stored in *.json format (compressed as zip files). See our repository for more details.</p><p> </p><p>Link to the ICCV article: </p><p><a href="https://openaccess.thecvf.com/content/ICCV2023/papers/Zhou_Rethinking_Pose_Estimation_in_Crowds_Overcoming_the_Detection_Information_Bottleneck_ICCV_2023_paper.pdf">https://openaccess.thecvf.com/content/ICCV2023/papers/Zhou_Rethinking_Pose_Estimation_in_Crowds_Overcoming_the_Detection_Information_Bottleneck_ICCV_2023_paper.pdf</a></p><p> </p><p>The weights and predictions are released with Creative Commons Attribution 4.0 license. The code is released under the Apache 2.0 license, see https://github.com/amathislab/BUCTD </p><p><i>If you find our weights, code or ideas useful, please cite:</i></p><p> </p><p>@InProceedings{Zhou_2023_ICCV,</p><p> author = {Zhou, Mu and Stoffl, Lucas and Mathis, Mackenzie Weygandt and Mathis, Alexander},</p><p> title = {Rethinking Pose Estimation in Crowds: Overcoming the Detection Information Bottleneck and Ambiguity},</p><p> booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},</p><p> month = {October},</p><p> year = {2023},</p><p> pages = {14689-14699}</p><p>}</p><p> </p>
Data for Contrasting action and posture coding with hierarchical deep neural network models of proprioception
No full text#############
Contrasting action and posture coding with hierarchical deep neural network models of proprioception, eLife 2023
#############
Authors: Kai J Sandbrink, Pranav Mamidanna, Claudio Michaelis, Matthias Bethge, Mackenzie W Mathis and Alexander Mathis
Affiliation: Brain Mind Institute, School of Life Sciences, École Polytechnique Fédérale de Lausanne, Switzerland, The Rowland Institute at Harvard, Harvard University, United States; Tübingen AI Center, Eberhard Karls Universität Tübingen & Institute for Theoretical Physics, Germany
Date of upload: December, 2024
Earlier the data was available via dropbox (see github).
Link to the eLife article:
https://elifesciences.org/articles/81499
--------------------------------
Here we provide the data and code for this project:
We share the proprioceptive character recognition dataset (contained in 'pcr_data.zip') it has approximately ~29GB when uncompressed.
We share the weights of all the trained networks (contained in 'network-weights.zip'): about ~3.5GB
The compressed code is also available here ('DeepDrawCode.zip').
The activations are shared in a separate Zenodo project (due to the size). Check out the repository below to find the link.
The up to date code is at: https://github.com/amathislab/DeepDraw
--------------------------------
The datasets, weights, activations and predictions are released with Creative Commons Attribution 4.0 license.
If you find this useful, please cite:
@article{sandbrink2023contrasting, title={Contrasting action and posture coding with hierarchical deep neural network models of proprioception}, author={Sandbrink, Kai J and Mamidanna, Pranav and Michaelis, Claudio and Bethge, Matthias and Mathis, Mackenzie Weygandt and Mathis, Alexander}, journal={Elife}, volume={12}, pages={e81499}, year={2023}, publisher={eLife Sciences Publications Limited}}UPAMATHISUPMWMATHI
Going Beyond Counting First Authors in Author Co-citation Analysis
Full text linkThe present study examines one of the fundamental aspects of author co-citation analysis (ACA) - the way co-citation
counts are defined. Co-citation counting provides the data on which all subsequent statistical analyses and mappings
are based, and we compare ACA results based on two different types of co-citation counting - the traditional type that
only counts the first one among a cited work's authors on the one hand and a non-traditional type that takes into
account the first 5 authors of a cited work on the other hand. Results indicate that the picture produced through this non-traditional author co-citation counting contains more coherent author groups and is therefore considerably clearer. However, this picture represents fewer specialties in the research field being studied than that produced through the traditional first-author co-citation counting when the same number of top-ranked authors is selected and analyzed. Reasons for these effects are discussed
Variations on the Author
Full text link“Variations on the Author” discusses two of Eduardo Coutinho’s recent films (Um Dia na Vida, from 2010, and Últimas Conversas, posthumously released in 2015) and their contribution to the general question of documentary authorship. The director’s filmography is characterized by a consistent yet self-effacing form of authorial self-inscription: Coutinho often features as an interviewer that rather than express opinions propels discourses; an interviewer that is good at listening. This mode of self-inscription characterizes him as an author who is not expressive but who is nonetheless markedly present on the screen. In Um Dia na Vida, however, Coutinho is completely absent form the image, while Últimas Conversas, on the contrary, includes a confessional prologue that moves the director from the margins to the center of his films. This article examines the ways in which these works stand out in the filmography of a director who offers new insights into the notion of cinematic authorship
Appropriate Similarity Measures for Author Cocitation Analysis
Full text linkWe provide a number of new insights into the methodological discussion about author cocitation analysis. We first argue that the use of the Pearson correlation for measuring the similarity between authors’ cocitation profiles is not very satisfactory. We then discuss what kind of similarity measures may be used as an alternative to the Pearson correlation. We consider three similarity measures in particular. One is the well-known cosine. The other two similarity measures have not been used before in the bibliometric literature. Finally, we show by means of an example that our findings have a high practical relevance.information science;Pearson correlation;cosine;similarity measure;author cocitation analysis
Neuroscience-inspired computer vision and language modeling for behavior analysis
No full textMeasuring animal behaviors is a crucial task in a range of scientific applications. Modern approaches with deep neural networks calls for developing solutions that solve a wide range of vision and language tasks. This ubiquitous need for understanding behavior in both scientific areas and industrial applications, as well as the difficulty of modeling behavior occurring in the wild, motivates building more generalizable and robust deep neural networks. Motivated by this, this dissertation focuses on solving the following key problems: how do we train deep neural networks to be generalizable across different data domains? How do we train neural networks or neural network-based systems to perform a wide range vision and language tasks? How do we enable the model to adapt and learn at inference time? More specifically, can we enable models to have some level of adaptive intelligence, either by learning from interacting with users and data, or via dynamic computing such as in-context learning? Towards this goal, my aim was to get inspiration from both the progress in artificial intelligence and insights from neuroscience. In Chapter 2, we introduce methods to merge heterogeneous animal pose datasets and training algorithms for the model to counter domain shifts and catastrophic forgetting. I showed that the proposed methods gave rise to the first animal pose foundation model that has zero-shot performance comparable to a fully trained model in downstream tasks. Then in Chapter 3, we report the first keypoint-aware, unsupervised learning approach with transformers for re-identification of animals. To explore the growing utility of large language models (LLMs), in Chapter 4, we proposed the first LLM-based agentic system that uses pre-trained deep neural networks and a Python API as tools and that dynamically learn and adapt at inference time. This proposed system was built with principles inspired from neuroscience at its core, and helped push the frontier of using LLM-based agents to automate behavior analysis using frontier computer vision and large language models. Finally, in Chapter 5 we proposed novel methods to evaluate and improve multi-modal large language models to recognize challenging human actions that occur in daily life. This work aims to better evaluate and refine MLLMs that can recognize human actions.UPMWMATHI
Dispelling the Myths Behind First-author Citation Counts
Full text linkWe conducted a full-scale evaluative citation analysis study of scholars in the XML research field to explore just how different from each other author rankings resulting from different citation counting methods actually are, and to demonstrate the capability of emerging data and tools on the Web in supporting more realistic citation counting methods. Our results contest some common arguments for the continued
use of first-author citation counts in the evaluation of scholars, such as high correlations between author rankings by first-author citation counts and other citation
counting methods, and high costs of using more realistic citation counting methods that are not well-supported by the ISI databases. It is argued that increasingly available digital full text research papers make it possible for citation analysis studies to go beyond what the ISI databases have directly supported and to employ more
sophisticated methods
Mice Use Prediction Errors to Adapt to Visuomotor Perturbations in Virtual Reality
No full textMotor adaptation allows organisms to adjust their movements in response to environmental changes, ensuring the successful execution of actions. While detailed behavioral studies in humans have inferred mechanisms underlying motor adaptationâ such as the role of internal models and sensory prediction errorsâ the neural substrates involved remain poorly understood due to limitations in directly studying these processes in humans. To address this challenge, we developed a novel visuomotor adaptation paradigm in mice using a virtual reality environment that simulates dynamic and immersive settings. Here, we show that mice adapt to visuomotor perturbations primarily through sensory prediction errors and that cortical pathways may contribute to this adaptation. Mice were trained to navigate a virtual track with dynamically changing visual feedback, where visuomotor perturbations altered the relationship between their physical movements and the visual scene. Initially, mice exhibited increased endpoint errors, which decreased as they adapted, while maintaining high reward rates, reflecting continuous updating of their internal models governing visuomotor integration. Optogenetic inhibition of the primary visual cortex (V1) during perturbation trials impaired motor adaptation, evidenced by reduced reward rates and increased end-point errors, indicating V1's significant role in implicit adaptation. However, some mice continued to adapt despite V1 inhibition, suggesting compensatory mechanisms involving other brain regions. Additionally, when exposed to large, fixed visuomotor perturbations, mice demonstrated explicit adaptation driven by reward, highlighting distinct learning strategies based on the type of error signal. These findings advance our ability to study the neural mechanisms underlying visuomotor adaptation by establishing mice as a powerful model system for motor adaptation. Understanding the interplay between sensory and reward prediction errors in motor adaptation has significant implications for developing rehabilitation strategies for motor disorders.UPMWMATHI
- …
