67 research outputs found
Runoff-Forecasting in an Alpine Catchment in the Upper Rhône Basin, Valais, Switzerland, with a Long Short-Term Memory Neural Network
No full textSIE-SECE
Semi-automated refinement of weak labels for detection of marine litter in Satellite imagery
No full textSIE-SECE
WildCLIP: Scene and animal attribute retrieval from camera trap data with domain-adapted vision-language models
No full textWildCLIP: Scene and animal attribute retrieval from camera trap data with domain-adapted vision-language models ############# Authors: Valentin Gabeff, Marc Russwurm, Devis Tuia & Alexander Mathis Affiliation: EPFL Date: January, 2024 Link to the article: https://link.springer.com/article/10.1007/s11263-024-02026-6 -------------------------------- WildCLIP is a fine-tuned CLIP model that allows to retrieve camera-trap events with natural language from the Snapshot Serengeti dataset. This project intends to demonstrate how vision-language models may assist the annotation process of camera-trap datasets. Here we provide the processed Snapshot Serengeti data used to train and evaluate WildCLIP, along with two versions of WildCLIP (model weights). Details on how to run these models can be found in the project github repository. Provided data (images and attribute annotations): The data consists of 380 x 380 image crops corresponding to the MegaDetector output of Snapshot Serengeti with a confidence threshold above 0.7. We considered only camera trap images containing single individuals. A description of the original data can be found on LILA here, released under the Community Data License Agreement (permissive variant). We warmly thank the authors of LILA for making the MegaDetector outputs publicly available, as well as for structuring the dataset and facilitating its access. Adapted CLIP model (model weights): WildCLIP models provided: [New] WildCLIP_vitb16_t1.pth: CLIP model with the ViT-B/16 visual backbone trained on data with captions following template 1. Trained on both base and novel vocabulary (see paper for details). [New] WildCLIP_vitb16_t1_lwf.pth: CLIP model with the ViT-B/16 visual backbone trained on data with captions following template 1, and with the additional VR-LwF loss. Trained on both base and novel vocabulary (see paper for details). WildCLIP_vitb16_t1_base.pth: CLIP model with the ViT-B/16 visual backbone trained on data with captions following template 1. Model used for evaluation and trained on base vocabulary only. (previously named WildCLIP_vitb16_t1.pth) WildCLIP_vitb16_t1t7_lwf_base.pth: CLIP model with the ViT-B/16 visual backbone trained on data with captions following templates 1 to 7, and with the additional VR-LwF loss. Model used for evaluation and trained on base vocabulary only. (previously named WildCLIP_vitb16_t1t7_lwf.pth) We also provide the CSV files containing the train / val / test splits. The train / test splits follow camera split from LILA (https://lila.science/datasets/snapshot-serengeti). The validation split is custom, and also at the camera level. train_dataset_crops_single_animal_template_captions_T1T7_ID.csv: Train set with captions from templates 1 through 7 (column "all captions") or template 1 only (column "template 1") val_dataset_crops_single_animal_template_captions_T1T7_ID.csv: Validation set with captions from templates 1 through 7 (column "all captions") or template 1 only (column "template 1") test_dataset_crops_single_animal_template_captions_T1T8T10.csv: Test set with captions from templates 1, 8, 9 and 10 (columns "all captions") Details on how the models were trained can be found in the associated publication. References: If you find our code, or weights, please cite: @article{gabeff2024wildclip, title={WildCLIP: Scene and animal attribute retrieval from camera trap data with domain-adapted vision-language models}, author={Gabeff, Valentin and Ru{\ss}wurm, Marc and Tuia, Devis and Mathis, Alexander}, journal={International Journal of Computer Vision}, pages={1--17}, year={2024}, publisher={Springer} } If you use the adapted Snapshot Serengeti data please also cite their article: @article{swanson2015snapshot, title={Snapshot Serengeti, high-frequency annotated camera trap images of 40 mammalian species in an African savanna}, author={Swanson, Alexandra and Kosmala, Margaret and Lintott, Chris and Simpson, Robert and Smith, Arfon and Packer, Craig}, journal={Scientific data}, volume={2}, number={1}, pages={1--14}, year={2015}, publisher={Nature Publishing Group} }ECEOZENODO
Instance norm improves meta-learning in class-imbalanced land cover classification
No full textDistribution shift is omnipresent in geographic data, where various climatic and cultural factors lead to different representations across the globe. We aim to adapt dynamically to unseen data distributions with model-agnostic meta-learning, where data sampled from each distribution is seen as a task with only a few annotated samples. Transductive batch normalization layers are often employed in meta-learning models, as they reach the highest numerical accuracy on the class-balanced target tasks used as meta-learning benchmarks. In this work, we demonstrate empirically that transductive batch normalization collapses when deployed on a real class-imbalanced land cover classification problem. We propose a solution to replace batch normalization with instance normalization. This modification consistently outperformed all other normalization alternatives across different meta-learning algorithms in our class-imbalanced land cover classification test tasks.ECE
Improved marine debris detection in satellite imagery with automatic refinement of coarse hand annotations
No full textPlastic litter is a major environmental hazard that endangers human, animal, and plant health on the planet. A substantial portion of plastic pollutants is washed from rivers and beaches into the oceans and aggregates at the surface as marine debris before decomposing into microplastics and being digested by animals or sedimented on the sea floor. The marine debris is inherently challenging to annotate manually on satellite images, as the boundaries of floating objects are not sharp, and a specific mixture of water is always present at the pixel level. Hence, all available annotated marine debris datasets suffer from annotation errors. In this work, we present a label refinement algorithm for marine debris detection that improves upon rough hand annotations and considers the spectral characteristics of marine debris. We show quantitatively that a deep learning model trained with improved annotations achieves a higher classification accuracy on confirmed marine debris on two out of three datasets of confirmed plastic marine debris in Africa (in Ghana and South Africa). Thanks to the refinement module, we improve results for an environmentally important application that would benefit from further research attention to mitigate important associated challenges like label noise, domain shifts, and severe class imbalance.ECEOPublished as a conference paper at ICLR 2023 Workshop in Machine Learning for Remote Sensin
Analysis of the microbial communities in soils of different ages following volcanic eruptions
No full textEl primer autor agradece la beca de investigación de la Fundación Alexander von Humboldt y la Sociedad Max Planck, Alemania.The first author acknowledges the research fellowship from the Alexander von Humboldt Foundation and the Max Planck Society, Germany
Large-scale detection of marine debris in coastal areas with Sentinel-2
No full textDetecting and quantifying marine pollution and macroplastics is an increasingly pressing ecological issue that directly impacts ecology and human health. Here, remote sensing can provide reliable estimates of plastic pollution by regularly monitoring and detecting marine debris in coastal areas. In this work, we present a detector for marine debris built on a deep segmentation model that outputs a probability for marine debris at the pixel level. We train this detector with a combination of annotated datasets of marine debris and evaluate it on specifically selected test sites where it is highly probable that plastic pollution is present in the detected marine debris. We integrate data-centric artificial intelligence principles by devising a training strategy with extensive sampling of negative examples and an automated label refinement of coarse hand labels. This yields a deep learning model that achieves higher accuracies on benchmark comparisons than existing detection models trained on previous datasets.ECE
Classification of Tropical Deforestation Drivers with Machine Learning and Satellite Image Time Series
No full textTropical deforestation is a major environmental problem with severe consequences such as carbon emissions or biodiversity loss. While much research focuses on monitoring and mapping deforestation, less attention is paid to understanding the various reasons and motivations behind it, known as deforestation drivers. Drivers can typically be identified from optical satellite imagery, but it is often necessary to view the deforested site at multiple points in time to determine the driver, making manual annotation of drivers laborious. In this work, we propose a deep learning model that classifies drivers from time series of Sentinel-2 images. The model combines convolutional, LSTM, and attention layers. To train the model, we use a large crowd-sourced dataset spanning across the tropics. We compare its results to other architectures and show that using time series can bring significant improvement in accuracy compared to single images, especially if a suitable architecture is used. Additionally, we analyze the attention scores produced by our model and show that it learns different strategies for different classes.ECE
6. O inominável do corpo nas obras de Melville
No full textInvestigação sobre as relações entre corpo e linguagem na obra do autor de Moby Dick. É no capítulo XXVIII de Moby Dick que nasce a originalidade e a modernidade da escrita melviniana, que tem o seu fundamento na transposição literária do corpo « inominável », essa mesma escrita que mais tarde tanto influenciará certos escritores como Conrad, Kafka, ou ainda Beckett.
Palavras-chave | escrita | voz | silêncio
Abstract
Research on the relationship between body and language in the works of the author of Moby Dick and Bartleby. It can be found in Chapter XXVIII of Moby Dick the originality and the modernity of Melville’s writing style, which has its basis in literary transposition of the "unspeakable" body. This same writing style that will so influence later certain writers as Conrad, Kafka, or Beckett.
Keywords | writing | voice | silence
YVES-MICHEL ERGAL é autor de diversos ensaios sobre literatura, dança e cinema, professor e persquisador da Universidade Marc Bloch, Strassourg, França.
YVES-MICHEL ERGAL is author of numerous essays on literature, dance and cinema. Currently he is professor and researcher at the Marc Bloch University, Strassourg, France
Poiesis and Obstruction in Art Practice
No full textThis PhD thesis examines the concept of poiesis, that is ‘calling into existence that which was not there before’, in the context of obstruction in studio practice. It poses the question ‘Is there a methodology that engages with obstruction which in turn calls new work’? In this thesis, the concept of poiesis emerging from the late Dr. Murray Cox’s ‘Aeolian Mode’, is analyzed alongside a concept of praxis, (a philosophical companion to poiesis), familiar to artistic practice. This thesis describes the orientation of the original idea, The Aeolian Mode, clinically developed by Dr. Murray Cox in Broadmoor Psychiatric Hospital. This PhD seeks to identify if there are similar ‘tenets of approach’ held within the methodology of ‘The Aeolian Mode’, that would be useful or are identifiable in artistic studio practice. This thesis draws on the work of the philosopher, Professor Richard Kearney, specifically Kearney’s ideas on the necessity of ‘the other’ for ‘radical possibility’ to occur. It maps a context of both Freudian and Jungian interpretations of art practice, identifying how these ideas have shaped the way art is seen today. Furthermore, it challenges the Freudian idea of ‘pathography’ and favours a Jungian approach of ‘individuation’ in the understanding of creative processes. It develops a ‘methodology of the conversation’, interviewing students, established artists, tutors about their approaches to obstruction/poiesis in art practice. Additionally, it examines my own obstruction to painting and identifies the methodology that released me from this obstruction. Conducting these interviews on art practice has enabled me to confirm my initial concerns about Freudian ‘pathography’ whilst validating the possibility of the Jungian concept of ‘individuation’ being of use to art practice. Finally, this PhD discusses the implications for further study and research, which have emerged during the ‘methodology of the conversation’ and the task of dissolving my obstruction to painting
- …
