1,720,961 research outputs found
Efficient Video Prediction via Sparsely Conditioned Flow Matching
No full textWe introduce a novel generative model for video prediction based on latent flow matching, an efficient alternative to diffusion-based models. In contrast to prior work, we keep the high costs of modeling the past during training and inference at bay by conditioning only on a small random set of past frames at each integration step of the image generation process. Moreover, to enable the generation of high-resolution videos and to speed up the training, we work in the latent space of a pretrained VQGAN. Finally, we propose to approximate the initial condition of the flow ODE with the previous noisy frame. This allows to reduce the number of integration steps and hence, speed up the sampling at inference time. We call our model Random frame conditioned flow Integration for VidEo pRediction, or, in short, RIVER. We show that RIVER achieves superior or on par performance compared to prior work on common video prediction benchmarks, while requiring an order of magnitude fewer computational resources. Project website: https://araachie.github.io/river
Spatio-Temporal Crop Aggregation for Video Representation Learning
Full text linkWe propose Spatio-temporal Crop Aggregation for video representation LEarning
(SCALE), a novel method that enjoys high scalability at both training and
inference time. Our model builds long-range video features by learning from
sets of video clip-level features extracted with a pre-trained backbone. To
train the model, we propose a self-supervised objective consisting of masked
clip feature prediction. We apply sparsity to both the input, by extracting a
random set of video clips, and to the loss function, by only reconstructing the
sparse inputs. Moreover, we use dimensionality reduction by working in the
latent space of a pre-trained backbone applied to single video clips. These
techniques make our method not only extremely efficient to train but also
highly effective in transfer learning. We demonstrate that our video
representation yields state-of-the-art performance with linear, non-linear, and
KNN probing on common action classification and video understanding datasets
Representation Learning by Detecting Incorrect Location Embeddings
Full text linkIn this paper, we introduce a novel self-supervised learning (SSL) loss for
image representation learning. There is a growing belief that generalization in
deep neural networks is linked to their ability to discriminate object shapes.
Since object shape is related to the location of its parts, we propose to
detect those that have been artificially misplaced. We represent object parts
with image tokens and train a ViT to detect which token has been combined with
an incorrect positional embedding. We then introduce sparsity in the inputs to
make the model more robust to occlusions and to speed up the training. We call
our method DILEMMA, which stands for Detection of Incorrect Location EMbeddings
with MAsked inputs. We apply DILEMMA to MoCoV3, DINO and SimCLR and show an
improvement in their performance of respectively 4.41%, 3.97%, and 0.5% under
the same training time and with a linear probing transfer on ImageNet-1K. We
also show full fine-tuning improvements of MAE combined with our method on
ImageNet-100. We evaluate our method via fine-tuning on common SSL benchmarks.
Moreover, we show that when downstream tasks are strongly reliant on shape
(such as in the YOGA-82 pose dataset), our pre-trained features yield a
significant gain over prior work.Comment: accepted at AAAI2023, https://github.com/Separius/DILEMM
Efficient Self-Supervised Visual Representation Learning via Sparsity
Full text linkLarge collections of labeled data have greatly improved the performance of Deep Neural Networks in computer vision tasks. However, the vast majority of visual data generated daily remains unlabeled, limiting the potential of supervised learning paradigms. This thesis explores novel techniques to guide deep models towards learning generalizable visual patterns without human supervision, with a particular focus on leveraging sparsity as a key principle.
Our primary tool in this endeavor is the design of Self-Supervised Learning (SSL) tasks that do not require manual labeling. Beyond enabling learning from vast amounts of unlabeled data, we demonstrate how sparsity-based self-supervision can capture relevant patterns often overlooked by traditional supervised approaches. We design learning tasks that extract rich representations from various visual modalities: shape information from images, temporal dynamics from videos, and multimodal understanding from vision-language data.
A common thread running through our work is the strategic application of sparsity. In contrastive learning, we show how token sparsity can enhance both computational efficiency and representation quality. For video analysis, we leverage spatio-temporal sparsity to enable efficient and scalable representation learning. In generative tasks, we demonstrate how sparse conditioning can tackle complex problems like video prediction while implicitly modeling world dynamics.
Notably, our task designs follow a unifying principle: the recognition and manipulation of sparse patterns in data. The strong performance of the learned representations on downstream vision tasks such as image classification, video understanding, and multimodal reasoning validates this approach.
By consistently demonstrating that thoughtful application of sparsity can not only reduce computational demands but often improve the quality and generalizability of learned representations, this work lays a foundation for more efficient, scalable, and effective visual understanding systems. Our contributions pave the way for artificial systems with visual perception and reasoning capabilities that can better leverage the vast amounts of unlabeled visual data surrounding us
Going Beyond Counting First Authors in Author Co-citation Analysis
Full text linkThe present study examines one of the fundamental aspects of author co-citation analysis (ACA) - the way co-citation
counts are defined. Co-citation counting provides the data on which all subsequent statistical analyses and mappings
are based, and we compare ACA results based on two different types of co-citation counting - the traditional type that
only counts the first one among a cited work's authors on the one hand and a non-traditional type that takes into
account the first 5 authors of a cited work on the other hand. Results indicate that the picture produced through this non-traditional author co-citation counting contains more coherent author groups and is therefore considerably clearer. However, this picture represents fewer specialties in the research field being studied than that produced through the traditional first-author co-citation counting when the same number of top-ranked authors is selected and analyzed. Reasons for these effects are discussed
Variations on the Author
Full text link“Variations on the Author” discusses two of Eduardo Coutinho’s recent films (Um Dia na Vida, from 2010, and Últimas Conversas, posthumously released in 2015) and their contribution to the general question of documentary authorship. The director’s filmography is characterized by a consistent yet self-effacing form of authorial self-inscription: Coutinho often features as an interviewer that rather than express opinions propels discourses; an interviewer that is good at listening. This mode of self-inscription characterizes him as an author who is not expressive but who is nonetheless markedly present on the screen. In Um Dia na Vida, however, Coutinho is completely absent form the image, while Últimas Conversas, on the contrary, includes a confessional prologue that moves the director from the margins to the center of his films. This article examines the ways in which these works stand out in the filmography of a director who offers new insights into the notion of cinematic authorship
Appropriate Similarity Measures for Author Cocitation Analysis
Full text linkWe provide a number of new insights into the methodological discussion about author cocitation analysis. We first argue that the use of the Pearson correlation for measuring the similarity between authors’ cocitation profiles is not very satisfactory. We then discuss what kind of similarity measures may be used as an alternative to the Pearson correlation. We consider three similarity measures in particular. One is the well-known cosine. The other two similarity measures have not been used before in the bibliometric literature. Finally, we show by means of an example that our findings have a high practical relevance.information science;Pearson correlation;cosine;similarity measure;author cocitation analysis
Dispelling the Myths Behind First-author Citation Counts
Full text linkWe conducted a full-scale evaluative citation analysis study of scholars in the XML research field to explore just how different from each other author rankings resulting from different citation counting methods actually are, and to demonstrate the capability of emerging data and tools on the Web in supporting more realistic citation counting methods. Our results contest some common arguments for the continued
use of first-author citation counts in the evaluation of scholars, such as high correlations between author rankings by first-author citation counts and other citation
counting methods, and high costs of using more realistic citation counting methods that are not well-supported by the ISI databases. It is argued that increasingly available digital full text research papers make it possible for citation analysis studies to go beyond what the ISI databases have directly supported and to employ more
sophisticated methods
- …
