290 research outputs found
Elo Uncovered: Robustness and Best Practices in Language Model Evaluation
No full textIn Natural Language Processing (NLP), the Elo rating system, originally
designed for ranking players in dynamic games such as chess, is increasingly
being used to evaluate Large Language Models (LLMs) through "A vs B" paired
comparisons. However, while popular, the system's suitability for assessing
entities with constant skill levels, such as LLMs, remains relatively
unexplored. We study two fundamental axioms that evaluation methods should
adhere to: reliability and transitivity. We conduct extensive evaluation of Elo
behaviour, illustrating that individual Elo computations exhibit volatility and
delving into the impact of varying the Elo rating system's hyperparameters. We
show that these axioms are not always satisfied raising questions about the
reliability of current comparative evaluations of LLMs. If the current use of
Elo scores is intended to substitute the costly head-to-head comparison of
LLMs, it is crucial to ensure the ranking is as robust as possible. Guided by
the axioms, our findings offer concrete guidelines for enhancing the
reliability of LLM evaluation methods, suggesting a need for reassessment of
existing comparative approaches.Comment: 22 pages, 7 figures, 2 tables. Revised version of the paper accepted
at GEM Workshop, EMNLP 202
Which Prompts Make The Difference? Data Prioritization For Efficient Human LLM Evaluation
No full textHuman evaluation is increasingly critical for assessing large language
models, capturing linguistic nuances, and reflecting user preferences more
accurately than traditional automated metrics. However, the resource-intensive
nature of this type of annotation process poses significant challenges. The key
question driving our work: "is it feasible to minimize human-in-the-loop
feedback by prioritizing data instances which most effectively distinguish
between models?" We evaluate several metric-based methods and find that these
metrics enhance the efficiency of human evaluations by minimizing the number of
required annotations, thus saving time and cost, while ensuring a robust
performance evaluation. We show that our method is effective across widely used
model families, reducing instances of indecisive (or "tie") outcomes by up to
54% compared to a random sample when focusing on the top-20 percentile of
prioritized instances. This potential reduction in required human effort
positions our approach as a valuable strategy in future large language model
evaluations.Comment: 37 pages, 8 figure
LE ROI DE KAHEL ET LE TERRORISTE NOIR DE MONENEMBO: UNE REECRITURE POSTCOLONIALE DE L’HISTOIRE
No full textThe historical evolution of the African continent was not smooth; the region experienced many upheavals including two powerful traumas that are slavery and colonialism. It is therefore not surprising that African writers, who are mainly inspired by the world around them, revisit History to write their novels. Among them, the Guinean writer Tierno Monénembo. The writer sheds the light on the history that has painfully shaken his continent, West Africa more precisely. His novels translate the almost obsessive concern to narrate the story of the African continent. We propose to study two of his novels: The king of Kahel (2008) and The black terrorist (2012). In these novels, the author offers a different reading of colonial memory, moving away from the long-standing Manichaeism between white and black. Through the romanticized biography of Viscount Sanderval - driven by a sincere desire to civilize the negroes but at the same time to conquer a kingdom in the region of Fouta-Djalon-, or that of Addi Bâ, a young Senegalese gunner who will enter, during the Second World War, in France, the reader is invited to dive into a troubled past and discover all the complexity of Franco-African politics. Our purpose is to show the way History fits in the novels of Monénembo, by studying literary devices in the novels as well as the representation of two historical figures: Sanderval and Addi Bâ
Style in Translation A Comparative Stylistic Analysis of The Sand Child by Tahar Ben Jelloun
No full textRecently, Theorists began to think that studying language linguistically is insufficiently beneficial, literary critics also believed that a mere literary analysis easily fall into dilemma, because the scope of their work deals only with language as a combination of words to convert message. Some questions whether the language expressed in literature may carry beyond what is supposed and that affects a generation of thoughts. A systematic wide range interdisciplinary seemed to find answers in the highly specialized language studies; this is what stylistics shows as it concerns. The interdisciplinary nature of the field means that stylistics typically cooperates some acquaintance with linguistics. As far as our research concerns, we attempted to purposefully relate this discipline with literary translation.
The main research question that leads us to accomplish this research paper was if the translator ability to transform the style of the original author would implicitly or explicitly transfer the meaning embedded in those stylistic features of the original text. A comparative analysis has to be underlined so we can grasp the traces of stylistic aspects in both translated and original texts. The Sand Child looked the best example for many reasons, mentioned in paper; we tried to provide a systematic analytic comparison between both styles. Eventually, we figured out the translator may succeed in reaching a faithful translation where he worked to keep stylistics prints of its original author; and thus, meaning upon the two texts is well-communicated. In fact, we believe, stylistics enhances our ability to interpret various meanings. Stylistics is more than studying language; it shortens distance between us and authors. A study that really merits much attention and further studies
"THE CORONA CRASH", BY GRACE BLAKELEY: A BOOK REVIEW
No full textIn this note, we come up with a review and presentation of the book “The Corona Crash: How the Pandemic Will Change Capitalism” by Grace Blakeley, a political and economics specialist who had various professional experiences, with companies such as KPMG and IPPR North in England.
In 2019 Grace released her early book, Stolen: How to Save the World from Financialization, She also worked as a journalist and a columnist for Tribune Newspaper.
Through her book - subject of this presentation and review-, the author exposed the fact that the current world has faced an epidemic and financial crisis because of the capitalist policies pursued in the leading countries for decades. In addition, she includes a reflexion by suggesting some solutions that maintain a socialist and environmental approach, based on social justice, solidarity, equality and the environment protecting policies
Women and politics in Algeria: essays on political representation
No full textGender and politics scholars have sought to determine whether there is a link between women’s descriptive representation, operationalized as the proportion of seats held by women in a national legislature, and women’s substantive representation, usually operationalized as laws that advance women’s rights. But, except for a few studies, the Arab world has not received significant attention which is surprising because the region has experienced a significant increase in women’s presence in politics. Further, most of this work has focused on democratic contexts, obscuring whether hypothesized links between women’s descriptive and substantive representation work in the same way in authoritarian contexts. To fill these gaps in the literature, I focus on the case of Algeria, where women’s presence in parliament increased from 8% to 31.6% after the adoption of a gender quota in 2012.
Drawing on in-depth interviews with a wide range of stakeholders, my research examines the backgrounds of women elected, constituency service priorities, legislative dynamics, and women’s agency. I argue that the “authoritarian toolkit,” i.e., the resources available to authoritarian governments to manage and control political outcomes, shapes women’s descriptive, substantive, and symbolic representation in ways that are distinct from how these dynamics operate in more democratic contexts.
While women parliamentarians reject the notion that they have an obligation to introduce and pass women’s rights laws, they invest time in helping their male and female constituents solve their everyday problems to challenge the notion that women do not belong in politics. Therefore, there may not be strong links between women's descriptive and substantive representation. However, the efforts of elected women on behalf of their male constituents may advance women’s symbolic representation by demonstrating women's abilities in the political realm.Ph.D.Includes bibliographical reference
Le discours politique des islamistes marocains au pouvoir : le discours populiste
No full textAt the end of this chapter entitled "Conflictual affective discourse", the study focused on a number of legitimising discursive strategies mobilised by Benkirane. In fact, the discourses that we have tried to analyse are part of a context marked by a social tension unseen in the history of Morocco (after the Arab Spring). These social realities, marked by a tension, or even a calculation of ideological repositioning between Islamists, communists, socialists, technocrats, etc., are, from the sociolinguistic interactional point of view, manifest in the discourse where the speaker seeks to impose his identity. This observation has led us to set up a typology that better defines this conflictual context, hence the interest in the grid developed by Windisch, whose author insists considerably on the internal functioning of a conflictual discourse produced in a situation of tension.
All the studies aimed at analysing legitimating discourse agree on two fundamental points: on the one hand, discourse is perceived as a linguistic production anchored in symbolic relations of force, negotiation, manipulation and persuasion between individuals; on the other hand, in order for each of them to achieve their objectives, the social actor is confronted with certain constraints in order to convey the meaning he or she aims for. In addressing the issue of dramatization in the so-called populist discourse, Patrick Charaudeau insists considerably on the affective dimension in the interaction with the mass.
Benchmark of Concentrating Solar Power Plants: Historical, Current and Future Technical and Economic Development
No full textAbstractThis paper compares the evolution of Concentrating Solar Power (CSP) Plants. The study is based on a data of 240 CSP plants whose give us a roadmap of technical and economic characteristics of these systems. On the basis of this information collected from the literature, we benchmarked and analyzed the state of the CSP plants from 1982 to 2020. As statistical results show, 87% of CSP projects are located in Spain and USA, but other countries have invested in CSP since 2012. Solar technologies analyzed are Parabolic Trough collector (PTC), Solar Power Tower (SPT), Linear Fresnel Reflector (LFR), and Parabolic Dish Collector (PDC). As detailed data studies for Fresnel and Dish are hardly available, special attention was given to parabolic trough system. The progressive reduction in investment cost of PTC technology over time is presented, taking also into account the energy storage option. Our sensitivity analysis indicates that PTC with thermal oil and molten salt storage at 50 MW is the most mature system, but SPT plants are promising and might have the greatest potential by early 2018
Short Query Expansion for Microblog Retrieval
No full textAbstractTwitter has emerged as the most popular among microblogging service providers. The content provided in Twitter is large, diverse, and huge in quantity. Given the increasing amount of information available through such microblogging sites, it would be interesting to be able to retrieve useful tweets in response to a given information need. However, Twitter's subscribers often have many difficulties dealing with its content. Especially in searching for tweets that satisfy their information needs. This problem becomes more complicated when the user-defined queries are short and precise. This paper deals with short and precise queries problem for micoblog retrieval. We expand short queries by semantically related terms extracted from Wikipedia, DBpedia and unstructured texts using textmining techniques. Experiments on TREC 2011 microblog collection show significant improvement in the retrieval performance
L’entre-deux méditerranéen dans Origines d’Amin Maalouf
No full textThe “in-between” in French-Lebanese writer Amin Maalouf is reflected in his position as a smuggler in the East and the West. In almost all of his literary production, we find this duo that constitutes a spatial deictic, serving as a bedrock to his narrative. Indeed, he often highlights the opposition that lies between these two areas. In his novel Origins, the author tries to remove the shadow that befits his mysteries and family riddles. He makes his scriptural space, at the same time, a place of healing because of his quest for origins, and a place of exile, since it is appropriate that his “only homeland is writing”. It is therefore important for us to see how this “Mediterranean in-between” becomes a ground for conciliation between the Here and the Elsewhere, a cultural mediator, a frontier lever, and even a scriptural exile.El «intermedio» en el escritor franco-libanés Amin Maalouf se refleja en su posición como contrabandista entre Oriente y Occidente. En su producción cuasi literaria, tenemos un dúo que constituye un deíctico espacial, que sirve de base para su narración. De hecho, a menudo resalta la oposición entre estas dos áreas. En Orígenes, el autor intenta eliminar la sombra que corresponde a sus misterios y enigmas familiares. Él, hace de su espacio bíblico, al mismo tiempo, un lugar de curación debido a su búsqueda de los orígenes, y un lugar de exilio, ya que es apropiado que su «única patria es la escritura». Es muy importante para nosotros ver cómo este «Mediterráneo en el medio» se convierte en un lugar de reconciliación entre Aquí y En otra parte, un mediador cultural, una palanca de fronteras, incluso un exilio bíblico
- …
