1,645 research outputs found
AlBERTo: Italian BERT language understanding model for NLP challenging tasks based on tweets
No full textRecent scientific studies on natural language processing (NLP) report the outstanding effectiveness observed in the use of context-dependent and task-free language understanding models such as ELMo, GPT, and BERT. Specifically, they have proved to achieve state of the art performance in numerous complex NLP tasks such as question answering and sentiment analysis in the English language. Following the great popularity and effectiveness that these models are gaining in the scientific community, we trained a BERT language understanding model for the Italian language (AlBERTo). In particular, AlBERTo is focused on the language used in social networks, specifically on Twitter. To demonstrate its robustness, we evaluated AlBERTo on the EVALITA 2016 task SENTIPOLC (SENTIment POLarity Classification) obtaining state of the art results in subjectivity, polarity and irony detection on Italian tweets. The pre-trained AlBERTo model will be publicly distributed through the GitHub platform at the following web address: https://github.com/marcopoli/AlBERTo-it in order to facilitate future research
AlBERTo: Modeling Italian Social Media Language with BERT
No full textNatural Language Processing tasks recently achieved considerable interest and progresses following the development of numerous innovative artificial intelligence models released in recent years. The increase in available computing power has made possible the application of machine learning approaches on a considerable amount of textual data, demonstrating how they can obtain very encouraging results in challenging NLP tasks by generalizing the properties of natural language directly from the data. Models such as ELMo, GPT/GPT-2, BERT, ERNIE, and RoBERTa have proved to be extremely useful in NLP tasks such as entailment, sentiment analysis, and question answering. The availability of these resources mainly in the English language motivated us towards the realization of AlBERTo, a natural language model based on BERT and trained on the Italian language. We decided to train AlBERTo from scratch on social network language, Twitter in particular, because many of the classic tasks of content analysis are oriented to data extracted from the digital sphere of users. The model was distributed to the community through a repository on GitHub and the Transformers library (Wolf et al. 2019) released by the development group huggingface.co. We have evaluated the validity of the model on the classification tasks of sentiment polarity, irony, subjectivity, and hate speech. The specifications of the model, the code developed for training and fine-tuning, and the instructions for using it in a research project are freely available
Probing BERT for Ranking Abilities
No full textContextual models like BERT are highly effective in numerous text-ranking tasks. However, it is still unclear as to whether contextual models understand well-established notions of relevance that are central to IR. In this paper, we use probing, a recent approach used to analyze language models, to investigate the ranking abilities of BERT-based rankers. Most of the probing literature has focussed on linguistic and knowledge-aware capabilities of models or axiomatic analysis of ranking models. In this paper, we fill an important gap in the information retrieval literature by conducting a layer-wise probing analysis using four probes based on lexical matching, semantic similarity as well as linguistic properties like coreference resolution and named entity recognition. Our experiments show an interesting trend that BERT-rankers better encode ranking abilities at intermediate layers. Based on our observations, we train a ranking model by augmenting the ranking data with the probe data to show initial yet consistent performance improvements (The code is available at https://github.com/yolomeus/probing-search/ ).Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.Web Information System
BERT Rankers are Brittle: A Study using Adversarial Document Perturbations
No full textContextual ranking models based on BERT are now well established for a wide range of passage and document ranking tasks. However, the robustness of BERT-based ranking models under adversarial inputs is under-explored. In this paper, we argue that BERT-rankers are not immune to adversarial attacks targeting retrieved documents given a query. Firstly, we propose algorithms for adversarial perturbation of both highly relevant and non-relevant documents using gradient-based optimization methods. The aim of our algorithms is to add/replace a small number of tokens to a highly relevant or non-relevant document to cause a large rank demotion or promotion. Our experiments show that a small number of tokens can already result in a large change in the rank of a document. Moreover, we find that BERT-rankers heavily rely on the document start/head for relevance prediction, making the initial part of the document more susceptible to adversarial attacks. More interestingly, we find a small set of recurring adversarial words that when added to documents result in successful rank demotion/promotion of any relevant/non-relevant document respectively. Finally, our adversarial tokens also show particular topic preferences within and across datasets, exposing potential biases from BERT pre-training or downstream datasets. Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.Web Information System
Recommended from our members
Albert "Bert" Thomas Nabor interviewed by Meleia Simon-Reynolds
No full textIn this interview, originally recorded in-person at University of California, Santa Cruz, Albert "Bert" Thomas Nabor speaks with Watsonville is in the Heart team member Meleia Simon-Reynolds. Bert discusses his father, Alberto Nabor's background in La Union, Philippines; his migration to Hawai'i where he worked in the sugarcane fields and the pineapple plantations; and his migrant farm work throughout California and Arizona. Bert also speaks about vivid childhood memories of his whole family accompanying Alberto on the migrant trail. Additionally, Bert discusses Alberto's and his own participation in a late 1970s strike at Carl Dobler and Sons in the Pajaro Valley as well as Alberto's experiences as a member of the First Filipino Regiment during World War II. Throughout the interview, Bert reflects on his father's work ethnic and the values he passed on especially in regard to struggles with racism and discrimination. Bert goes on to discuss his father, Alberto Nabor's involvement with Caballeros de Dimas-Alang and the Filipino Catholic Association. He remembers going to dances and community events with his family as a young child. Bert also discusses his father's marriage to Erlinda Aragon, a Mexican American woman from Colorado. Bert reflects on his father's life and the values that he instilled. Finally, he discusses his experience attending UC Santa Cruz in the 1980s and how the school has changed over the year
Author Ben Ames Williams first met Searsmont farmer Bert McCorrison in 1918, a m
No full textAuthor Ben Ames Williams first met Searsmont farmer Bert McCorrison in 1918, a meeting which the author said had a profound impact on his professional career. McCorrison died in 1931, leaving Williams his Hardscrabble Farm in Searsmount, which became the author\u27s home until his death in 1953
Forged-GAN-BERT: Authorship Attribution for LLM-Generated Forged Novels
No full textThe advancement of generative Large Language Models (LLMs), capable of producing human-like texts, introduces challenges related to the authenticity of the text documents. This requires exploring potential forgery scenarios within the context of authorship attribution, especially in the literary domain. Particularly, two aspects of doubted authorship may arise in novels, as a novel may be imposed by a renowned author or include a copied writing style of a well-known novel. To address these concerns, we introduce Forged-GAN-BERT, a modified GAN-BERT-based model to improve the classification of forged novels in two data-augmentation aspects: via the Forged Novels Generator (i.e., ChatGPT) and the generator in GAN. Compared to other transformer-based models, the proposed Forged-GAN-BERT model demonstrates an improved performance with F1 scores of 0.97 and 0.71 for identifying forged novels in single-author and multi-author classification settings. Additionally, we explore different prompt categories for generating the forged novels to analyse the quality of the generated texts using different similarity distance measures , including ROUGE-1, Jaccard Similarity, Overlap Confident, and Cosine Similarity
New Facility for Membrane Fouling Investigations under Customizable Hydrodynamics: Validation and Preliminary Experiments with Pulsating Cross-Flow
No full textFlux reduction induced by fouling is arguably the most adverse phenomenon in membrane-based separation systems. In this respect, many laboratory-scale filtration studies have shown that an appropriate use of hydrodynamic perturbations can improve both performance and durability of the membrane; however, to fully understand and hence appropriately exploit such effects, it is necessary to understand the underpinning flow processes. Towards this end, in this work we propose and validate a new module-scale laboratory facility with the aim of investigating, at very well-controlled flow conditions, how hydrodynamics affects mass transport phenomena at the feed/membrane interface. The proposed facility was designed to obtain a fully developed and uniform flow inside the test section and to impose both steady and pulsating flow conditions. The walls of the facility were made transparent to grant optical accessibility to the flow. In this paper, we discuss data coming from particle image velocimetry (PIV) measurements and preliminary ultrafiltration tests both under steady and pulsating flow conditions. PIV data indicate that the proposed facility allows for excellent flow control from a purely hydrodynamic standpoint. Results from filtration tests provide promising results pointing towards pulsating flows as a viable technique to reduce fouling in membrane systems
Dave Hunter and Bert McDonald
No full textPhotograph - Dave Hunter Addresses the Haggis at Robbie Burns night at Royal Canadian Legion, Athabasca Branch No. 103, Athabasca, Alberta. Bert McDonald is on the left. February 6, 196
Bert Pary House - 02
No full textPhotograph - This building was built in 1912 and was owned by Bert Pary, a telegrapher and lineman. It was purchased in 1927 by Dean Galloway, a UGG grain buyer and his widow Catherine lived in the house until 1973. Ukrainian Catholic priest Father Karychuk and his wife bought the house and passed ownership to their daughter in 1995. It was demolished in 1995 and the native Friendship Centre was built on the sit
- …
