1,720,975 research outputs found
Statistically Validated Networks for assessing topic quality in LDA models
No full textProbabilistic topic models have become one of the most widespread machine learning technique for textual analysis purpose. In this framework, Latent Dirichlet Allocation (LDA) (Blei et al., 2003) gained more and more popularity as a text modelling technique. The idea is that documents are represented as random mixtures over latent topics, where a distribution overwords characterizes each topic. Unfortunately, topic models do not guarantee the interpretability of their outputs. The topics learned from the model may be only characterized by a set of irrelevant or unchained words, being useless for the interpretation. Although many topic-quality metrics were proposed (Newman et al., 2009; Aletras and Stevenson,2013; Roder et al., 2015; Nikolenko et al., 2017), the automatic evaluation of the coherence of topics remains an open research area. The main contributions of this paper are: i) to define a coherence measure (SVN-Coherence) based on a rigorous statistical model that approximates human ratings better than state-of-the-art methods, and ii) to filter out marginal associations of words and facilitate the graphical representation and interpretation of the obtained topics through Statically Validated Networks (SVN) (Tumminello et al., 2011). Specifically, the method builds a co-occurrence network for each topic whose most probable words are the nodes. We set a link between two nodes (words) in each network if their co-occurrences are statistically significant. The Hypergeometric distribution describes the probability mass function under the null hypothesis and it models the probability of co-occurrence between words conditionally to their marginals. Indeed, it allows taking into account the heterogeneity of the vocabulary on a collection of texts. Finally, we derive a global measure of coherence for each topic by considering the number of statistically validated links, the strength of the association between word pairs, and the relative relevance of each word in the topic. We claim that these links carry relevant information about the structure of topics, i.e., the more connected the network, the more semantically coherent the corresponding topic. The new measure provides a coherence-based ranking that distinguishes between high-quality and low-quality topics. We designed a survey to obtain human judgment, which we use as ground truth, to compare our method with the state-of-art coherence measures. Specifically, we asked 222 PhD students to evaluate the coherence of 32 topics (extracted from the New York Times articles dataset) on a 4-point scale. The results show that the proposed SVN-Coherence substantially outperforms all the state-of-art coherence metrics
Statistically Validated Network approach for document clustering and topic modeling
No full textIn machine learning, document clustering and topic modeling are scientific challenges
concerning the extraction of useful information from a collection of texts. Traditional
approaches, such as Latent Dirichlet Allocation (LDA), rely on maximising likeli-
hood functions. In this paper, we explore a paradigm shift towards network represen-
tation of textual data and the associated challenges of community detection [3]. We
proposes a new method to face the tasks of document clustering and topic modeling,
representing a collection of documents as a bipartite network. Then, we introduce the
application of Statistically Validated Networks (SVN) to filter out irrelevant con-
nections within the projected networks of words and documents. The SVN method is
promising in the framework of topic modeling. For instance, Simonetti et al. (2022) recently proposed a new application of SVN to measure the coherence of topics. In-
stead, we aim to identify the topics themselves. By doing so, we can naturally find topics
with high coherence according to the measure proposed by the authors. Moreover, the
modularity contribution of each community (topic) can be interpreted as a measure of
coherence since it is an intensive quantity that assesses the tendency of words within a
given topic to occur in the same sentences jointl
MEASURING TOPIC COHERENCE THROUGH STATISTICALLY VALIDATED NETWORKS
No full textTopic models arise from the need of understanding and exploring large text
document collections and predicting their underlying structure. Latent Dirichlet
Allocation (LDA) (Blei et al., 2003) has quickly become one of the most popular
text modelling techniques. The idea is that documents are represented as random
mixtures over latent topics, where a distribution over words characterizes each topic.
Unfortunately, topic models give no guaranty on the interpretability of their outputs.
The topics learned from texts may be characterized by a set of irrelevant or
unchained words. Therefore, topic models require validation of the coherence of
estimated topics. However, the automatic evaluation of the latent space of a topic
model is a difficult task. Formerly, the most used metric for evaluating the quality of
a topic model was the held-out likelihood. Still, the literature has shown that this
method emphasizes complexity rather than interpretability. Although many
procedures were recently proposed (Röder et al., 2015), the automatic evaluation of
topic coherence remains an open research area. Our work aims to provide a new
technique based on Statistically Validated Network (Tumminello et al., 2011). Our
approach consists in representing each topic as a network of its most probable
words. The presence of a link between each pair of words is assessed by statistically
validating their co-occurrences in sentences against the null hypothesis of random
co-occurrence. Thus, we propose a new coherence measure based on the structure of
the statistically validated network. Furthermore, the new measure provides a ranking
of topics and distinguishes high-quality from low-quality topics. The intuition is that
the pairwise associations of words is strictly related to the semantic coherence and
interpretability of a topic
Language of Bankruptcy: Analysis of Word Sequences and Word Context
No full textAcademics and practitioners searched for reliable indicators
of companies’ failure focusing only on quantitative data such as financial
ratios and market variables. However, recent literature aims to quantify
textual information of financial reports studying features such as topics
and words’ co-occurrences, confirming their usefulness in predicting company
bankruptcy. In this work, we propose a new approach to analysing
texts that focuses on sentences interpreted as ordered sequences of words.
We propose a new approach, based on Language Model, to predict the
company’s bankruptcy that was released in the next year. Given the high
predictive power of the model, we investigate the sentences of texts to
gain insights into how failing companies’ language differs from the nonfailing
one. Our approach allows us to move away from fixed word-lists,
exploring linguistic features to understand how a word is used in different
contexts. The results of our analysis lead us to observe that the concept
of bankruptcy can take on different meanings arising from the different
legitimisation strategies that companies facing bankruptcy may use
Marked Hawkes processes for Twitter data
No full textIn this paper, we propose to model retweet event sequences using a marked Hawkes process, which is a self-exciting point process where the occurrence of previous events in time increases the probability of further events. The aim is to analyse Twitter data combining temporal point processes theory and textual analysis. Since each retweet event carries a set of properties, we mark the process by different characteristics drawn from the textual analysis, finding that the tone of the description of the Twitter user is a good predictor of the number of retweets in a single cascade
Statistically Validated Networks for evaluating coherence in topic models
Full text linkProbabilistic topic models have become one of the most widespread machine learning
technique for textual analysis purpose. In this framework, Latent Dirichlet Allocation
(LDA) gained more and more popularity as a text modelling technique. The idea is
that documents are represented as random mixtures over latent topics, where a distribution
over words characterizes each topic. Unfortunately, topic models do not guarantee
the interpretability of their outputs. The topics learned from the model may be characterized
by a set of irrelevant or unchained words, being useless for the interpretation.
In the framework of topic quality evaluation, the pairwise semantic cohesion among
the top-N most probable words (for a given topic) is calculated by measures based
on words co-occurrences. Many topic-quality metrics were proposed defining different
score measures such as: Pointwise Mutual Information (PMI), also called UCI; an
asymmetrical measure called UMass; Normalized Pointwise Mutual Information
(NPMI), a measure based on tf-idf scores , and a measure called CV proposed
by Roder et al. Although these several measures in the literature have already considered cooccurrence
between words as a measure of association, none has undertaken a statistical
approach based on hypotheses testing to assess whether the co-occurrence obtained between
two words can be attributed to the chance or if these links carry relevant information
about the structure of topics. Thus, we propose a new coherence measure based on
Statistically Validated Network to evaluate the interpretability of the top words of a
topic
Going Beyond Counting First Authors in Author Co-citation Analysis
Full text linkThe present study examines one of the fundamental aspects of author co-citation analysis (ACA) - the way co-citation
counts are defined. Co-citation counting provides the data on which all subsequent statistical analyses and mappings
are based, and we compare ACA results based on two different types of co-citation counting - the traditional type that
only counts the first one among a cited work's authors on the one hand and a non-traditional type that takes into
account the first 5 authors of a cited work on the other hand. Results indicate that the picture produced through this non-traditional author co-citation counting contains more coherent author groups and is therefore considerably clearer. However, this picture represents fewer specialties in the research field being studied than that produced through the traditional first-author co-citation counting when the same number of top-ranked authors is selected and analyzed. Reasons for these effects are discussed
Variations on the Author
Full text link“Variations on the Author” discusses two of Eduardo Coutinho’s recent films (Um Dia na Vida, from 2010, and Últimas Conversas, posthumously released in 2015) and their contribution to the general question of documentary authorship. The director’s filmography is characterized by a consistent yet self-effacing form of authorial self-inscription: Coutinho often features as an interviewer that rather than express opinions propels discourses; an interviewer that is good at listening. This mode of self-inscription characterizes him as an author who is not expressive but who is nonetheless markedly present on the screen. In Um Dia na Vida, however, Coutinho is completely absent form the image, while Últimas Conversas, on the contrary, includes a confessional prologue that moves the director from the margins to the center of his films. This article examines the ways in which these works stand out in the filmography of a director who offers new insights into the notion of cinematic authorship
Appropriate Similarity Measures for Author Cocitation Analysis
Full text linkWe provide a number of new insights into the methodological discussion about author cocitation analysis. We first argue that the use of the Pearson correlation for measuring the similarity between authors’ cocitation profiles is not very satisfactory. We then discuss what kind of similarity measures may be used as an alternative to the Pearson correlation. We consider three similarity measures in particular. One is the well-known cosine. The other two similarity measures have not been used before in the bibliometric literature. Finally, we show by means of an example that our findings have a high practical relevance.information science;Pearson correlation;cosine;similarity measure;author cocitation analysis
- …
