2,807 research outputs found
Stemming and n-grams in Spanish: an evaluation of their impact on information retrieval
No full textSe analizan modelos y técnicas utilizadas para los recuentos de frecuencias de palabras que aparecen tanto en los documentos como en las preguntas formuladas en los Sistemas de Recuperacion de Información. Se describen pruebas realizadas para los documentos en español, que implicaron algunas técnicas utilizadas en inglés, así como el uso de n-gramas, y se comparan los resultados.At some stage, most of the models and techniques implemented in IR use frequency counts of the terms appearing in documents and in queries. However, many words, since they are derived from the same stem, have very close semantic contents. This makes a grouping of such variants under a single term advisable. Otherwise, dispersal occurs in the calculation of frequency of these terms, and it also becomes difficult to compare queries and documents. On the other hand, there are notable differences between different languages in the way of forming derivatives and inflected forms, so that the application of specific techniques can produce unequal results according to the language of the documents and queries. A description is given of the tests carried out for documents in Spanish, which involved some stemming techniques widely used in English, as well as the application of n-grams, and the results are compared
Character N-Grams for Detecting Deceptive Controversial Opinions
No full text[EN] Controversial topics are present in the everyday life, and opinions about them can be either truthful or deceptive. Deceptive opinions are emitted to mislead other people in order to gain some advantage. In the most of the cases humans cannot detect whether the opinion is deceptive or truthful, however, computational approaches have been used successfully for this purpose. In this work, we evaluate a representation based on character n-grams features for detecting deceptive opinions. We consider opinions on the following: abortion, death penalty and personal feelings about the best friend; three domains studied in the state of the art. We found character n-grams effective for detecting deception in these controversial domains, even more than using psycholinguistic features. Our results indicate that this representation is able to capture relevant information about style and content useful for this task. This fact allows us to conclude that the proposed one is a competitive text representation with a good trade-off between simplicity and performance.We would like to thank CONACyT for partially supporting this work under grants 613411, CB-2015-01-257383, and FC-2016/2410. The work of the last author was partially funded by the Spanish MINECO under the research project SomEMBED (TIN2015-71147-C2-1-P).Sánchez-Junquera, JJ.; Luis Villaseñor Pineda; Montes Gomez, M.; Rosso, P. (2018). Character N-Grams for Detecting Deceptive Controversial Opinions. Lecture Notes in Computer Science. 11018:135-140. https://doi.org/10.1007/978-3-319-98932-7_13S13514011018Aritsugi, M., et al.: Combining word and character n-grams for detecting deceptive opinions, vol. 1, pp. 828–833. IEEE (2017)Buller, D.B., Burgoon, J.K.: Interpersonal deception theory. Commun. Theory 6(3), 203–242 (1996)Cagnina, L.C., Rosso, P.: Detecting deceptive opinions: intra and cross-domain classification using an efficient representation. Int. J. Uncertainty Fuzziness Knowl. Based Syst. 25(Suppl. 2), 151–174 (2017)Feng, S., Banerjee, R., Choi, Y.: Syntactic stylometry for deception detection, pp. 171–175. Association for Computational Linguistics (2012)Fusilier, D.H., Montes-y-Gómez, M., Rosso, P., Cabrera, R.G.: Detection of opinion spam with character n-grams. In: Gelbukh, A. (ed.) CICLing 2015. LNCS, vol. 9042, pp. 285–294. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-18117-2_21Hernández-Castañeda, Á., Calvo, H., Gelbukh, A., Flores, J.J.G.: Cross-domain deception detection using support vector networks. Soft Comput. 21(3), 1–11 (2016)Mihalcea, R., Strapparava, C.: The lie detector: explorations in the automatic recognition of deceptive language. In: Proceedings of the ACL-IJCNLP 2009 Conference Short Papers, pp. 309–312. Association for Computational Linguistics (2009)Ott, M., Choi, Y., Cardie, C., Hancock, J.T.: Finding deceptive opinion spam by any stretch of the imagination. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1, pp. 309–319. Association for Computational Linguistics (2011)Pérez-Rosas, V., Mihalcea, R.: Cross-cultural deception detection. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), vol. 2, pp. 440–445 (2014)Sapkota, U., Solorio, T., Montes-y-Gómez, M., Bethard, S.: Not all character n-grams are created equal: a study in authorship attribution. In: Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 93–102 (2015)Vrij, A.: Detecting Lies and Deceit: Pitfalls and Opportunities. Wiley, Hoboken (2008
The effects of certain dietary factors on the elimination of various chlorinated hydrocarbons
No full textFive experiments were performed utilizing Japanese quail (Coturnix c. japonica) to determine the effects of the lipotropic agents, chlorine chloride and inositol, on the elimination of selected chlorinated hydrocarbons from the body. Supplementation of the diet with 2645 grams of chlorine chloride and 2000 grams of inositol per 1000 kilograms was effective in accelerating the rate of elimination of the chlorinated hydrocarbons, DDT and heptachlor. The addition of 10 percent vegetable oil to the basal diet increased the quantity of the chlorinated hydrocarbon residues retained in the tissue samples studied. An increase in liver weight was observed in the groups receiving the diets supplemented with chlorine chloride and inositol. This indicates increased hepatic activity due to a more rapid mobilization and metabolism of the chlorinated hydrocarbons studied. Supplementation of the basal diet with chlorine chloride or inositol singly at levels of 2645 grams or 2000 grams per 1000 kilograms, respectively, was effective in reducing the total quantity of body fat but was not effective in reducing the total carcass residue of DDT and metabolites. Under these experimental conditions, supplementation of the basal diet of mature laying Japanese quail with 1323 grams of chlorine chloride and 1000 grams of inositol per 1000 kilograms was as effective as supplementation with 2645 grams of chlorine chloride and 2000 grams of inositol per 1000 kilograms in accelerating the elimination of DDT and metabolites from the body.
Using n-grams to identify time periods of cultural influence
No full textAn author's literary style is influenced by the cultural time period in which the author lives. The author's ideas, and the words chosen to express them, can help identify the cultural time period that most influenced the author. Ideas are expressed in language through sequences of words called n-grams. Over the past several years, Google has been engaged in digitizing millions of books. As part of this endeavor, Google has created a database of n-grams extracted from these digitized books, and has made the database available to researchers online. This is the first time ever that such an extensive repository of cultural data has been made available. This study develops and tests an original method for utilizing Google's database to identify the cultural time period that most influenced the author of a published work. Several undisputed literary works are examined, from which sets of n-grams are extracted and compared against the Google database. The frequency and distribution of n-gram matches allow us to determine the cultural time period that most influenced the author. The method is also tested against several literary works having uncertain or disputed authorship and period of composition. The results suggest that the method developed provides a reasonable approximation of the time period of greatest cultural influence for each book. Unexpectedly, the results tend to support conclusions reached by another researcher with regard to prior literary influences on the Ern Malley Poems. In addition, they lend support to a well-known alternate theory on the authorship of the Book of Mormon.M.S
Limited feeding of commercial layers with diets of different nutrient densities
No full textA study was undertaken to determine the feasibility of restricting the feed intake of hens fed different density diets. Performance traits, interior and exterior egg quality, and egg composition of individually caged commercial layers were studied for twelve 28-day periods. Experimental diets of three densities were used. The control diet was fed ad libitum and the assumption was made that the average daily intake for this diet would be 110 grams per hen per day. Two higher density diets were adjusted to contain the same amounts of protein, energy, vitamins, calcium, phosphorus and salt as the ad libitum diet but in 105 and 100 grams of feed. The daily feed consumption of the experimental groups was limited to these amounts. The higher nutrient density in the restricted diets (105 and 100 grams per hen per day) was obtained mainly by using supplements of vegetable fat, L-lysine and DL-methionine. Body weight was heavier for the ad libitum fed birds than for the restrictively fed groups. The hens fed ad libitum tended to over-consume, gaining extra body weight; this was negatively correlated with most of the other performance traits. Hen-day egg production and feed efficiency were improved by the diet of intermediate density (105 grams). Egg production was approximately equal for the ad libitum and the restricted groups receiving 100 grams per day, but egg size and feed efficiency were higher for the hens restricted to 100 grams per day. Mortality was significantly influenced by dietary density. As dietary fat was increased in the ration, livability was reduced. Production costs per unit of eggs were calculated and were highly influenced by the feed to egg ratios, thus the diet of intermediate density (105 grams) gave the lowest feed cost to produce a kilogram of eggs. Exterior and interior egg quality was also influenced by increasing dietary density. Shell thickness, shell weight and percent shell tended to improve as nutrient density was increased in the ration. The highest density diet (100 grams per hen per day) gave the best shell quality. However, the Haugh unit score was lower for the hens fed the highest density diet. The restrictively fed hens receiving 105 grams; gave the highest Haugh unit score. Both the protein content for the whole egg and the cholesterol content in the yolk increased significantly as nutrient density increased in the ration. Irrespective of dietary factors, cholesterol increased as age of the bird increased. On the contrary, the percent fat in the whole egg decreased as dietary density was increased, but lacked statistical significance
Recommended from our members
Identifying idiolect in forensic authorship attribution: an n-gram textbite approach
No full textForensic authorship attribution is concerned with identifying authors of disputed or anonymous documents, which are potentially evidential in legal cases, through the analysis of linguistic clues left behind by writers. The forensic linguist “approaches this problem of questioned authorship from the theoretical position that every native speaker has their own distinct and individual version of the language [. . . ], their own idiolect” (Coulthard, 2004: 31). However, given the diXculty in empirically substantiating a theory of idiolect, there is growing concern in the Veld that it remains too abstract to be of practical use (Kredens, 2002; Grant, 2010; Turell, 2010). Stylistic, corpus, and computational approaches to text, however, are able to identify repeated collocational patterns, or n-grams, two to six word chunks of language, similar to the popular notion of soundbites: small segments of no more than a few seconds of speech that journalists are able to recognise as having news value and which characterise the important moments of talk. The soundbite oUers an intriguing parallel for authorship attribution studies, with the following question arising: looking at any set of texts by any author, is it possible to identify ‘n-gram textbites’, small textual segments that characterise that author’s writing, providing DNA-like chunks of identifying material
Detection of opinion spam with character n-grams
No full textThe final publication is available at Springer via http://dx.doi.org/10.1007/978-3-319-18117-2_21In this paper we consider the detection of opinion spam as a stylistic classi cation task because, given a particular domain, the deceptive and truthful opinions are similar in content but di ffer in the way opinions are written (style). Particularly, we propose using character ngrams as features since they have shown to capture lexical content as well as stylistic information. We evaluated our approach on a standard
corpus composed of 1600 hotel reviews, considering positive and negative
reviews. We compared the results obtained with character n-grams against the ones with word n-grams. Moreover, we evaluated the e ffectiveness of character n-grams decreasing the training set size in order to simulate real training conditions. The results obtained show that character n-grams are good features for the detection of opinion spam; they seem to be able to capture better than word n-grams the content of deceptive opinions and the writing style of the deceiver. In particular,
results show an improvement of 2:3% and 2:1% over the word-based representations in the detection of positive and negative deceptive opinions
respectively. Furthermore, character n-grams allow to obtain a good performance
also with a very small training corpus. Using only 25% of the training set, a Na ve Bayes classi er showed F1 values up to 0.80 for both opinion polarities.This work is the result of the collaboration in the frame-work of the WIQEI IRSES project (Grant No. 269180) within the FP7 Marie Curie. The second author was partially supported by the LACCIR programme under project ID R1212LAC006. Accordingly, the work of the third author was in the framework the DIANA-APPLICATIONS-Finding Hidden Knowledge inTexts: Applications (TIN2012-38603-C02-01) project, and the VLC/CAMPUS Microcluster on Multimodal Interaction in Intelligent Systems.Hernández Fusilier, D.; Montes Gomez, M.; Rosso, P.; Guzmán Cabrera, R. (2015). Detection of opinion spam with character n-grams. En Computational Linguistics and Intelligent Text Processing: 16th International Conference, CICLing 2015, Cairo, Egypt, April 14-20, 2015, Proceedings, Part II. Springer International Publishing. 285-294. https://doi.org/10.1007/978-3-319-18117-2_21S285294Blamey, B., Crick, T., Oatley, G.: RU:-) or:-(? character-vs. word-gram feature selection for sentiment classification of OSN corpora. Research and Development in Intelligent Systems XXIX, 207–212 (2012)Drucker, H., Wu, D., Vapnik, V.N.: Support Vector Machines for Spam Categorization. IEEE Transactions on Neural Networks 10(5), 1048–1054 (2002)Feng, S., Banerjee, R., Choi, Y.: Syntactic Stylometry for Deception Detection. Association for Computational Linguistics, short paper. ACL (2012)Feng, S., Xing, L., Gogar, A., Choi, Y.: Distributional Footprints of Deceptive Product Reviews. In: Proceedings of the 2012 International AAAI Conference on WebBlogs and Social Media (June 2012)Gyongyi, Z., Garcia-Molina, H., Pedersen, J.: Combating Web Spam with Trust Rank. In: Proceedings of the Thirtieth International Conference on Very Large Data Bases, vol. 30, pp. 576–587. VLDB Endowment (2004)Hall, M., Eibe, F., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.: The WEKA Data Mining Software: an Update. SIGKDD Explor. Newsl. 10–18 (2009)Hernández-Fusilier, D., Guzmán-Cabrera, R., Montes-y-Gómez, M., Rosso, P.: Using PU-learning to Detect Deceptive Opinion Spam. In: Proceedings of the 4th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis for Computational Linguistics: Human Language Technologies, Atlanta, Georgia, USA, pp. 38–45 (2013)Hernández-Fusilier, D., Montes-y-Gómez, M., Rosso, P., Guzmán-Cabrera, R.: Detecting Positive and Negative Deceptive Opinions using PU-learning. Information Processing & Management (2014), doi:10.1016/j.ipm.2014.11.001Jindal, N., Liu, B.: Opinion Spam and Analysis. In: Proceedings of the International Conference on Web Search and Web Data Mining, pp. 219–230 (2008)Jindal, N., Liu, B., Lim, E.: Finding Unusual Review Patterns Using Unexpected Rules. In: Proceedings of the 19th ACM International Conference on Information and Knowledge Management, CIKM 2010, pp. 210–220(October 2010)Kanaris, I., Kanaris, K., Houvardas, I., Stamatatos, E.: Word versus character n-grams for anti-spam filtering. International Journal on Artificial Intelligence Tools 16(6), 1047–1067 (2007)Lim, E.P., Nguyen, V.A., Jindal, N., Liu, B., Lauw, H.W.: Detecting Product Review Spammers Using Rating Behaviours. In: CIKM, pp. 939–948 (2010)Liu, B.: Sentiment Analysis and Opinion Mining. Synthesis Lecture on Human Language Technologies. Morgan & Claypool Publishers (2012)Mukherjee, A., Liu, B., Wang, J., Glance, N., Jindal, N.: Detecting Group Review Spam. In: Proceedings of the 20th International Conference Companion on World Wide Web, pp. 93–94 (2011)Ntoulas, A., Najork, M., Manasse, M., Fetterly, D.: Detecting Spam Web Pages through Content Analysis. Transactions on Management Information Systems (TMIS), 83–92 (2006)Ott, M., Choi, Y., Cardie, C., Hancock, J.T.: Finding Deceptive Opinion Spam by any Stretch of the Imagination. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Portland, Oregon, USA, pp. 309–319 (2011)Ott, M., Cardie, C., Hancock, J.T.: Negative Deceptive Opinion Spam. In: Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Atlanta, Georgia, USA, pp. 309–319 (2013)Raymond, Y.K., Lau, S.Y., Liao, R., Chi-Wai, K., Kaiquan, X., Yunqing, X., Yuefeng, L.: Text Mining and Probabilistic Modeling for Online Review Spam Detection. ACM Transactions on Management Information Systems 2(4), Article: 25, 1–30 (2011)Stamatatos, E.: On the robustness of authorship attribution based on character n-gram features. Journal of Law & Policy 21(2) (2013)Wu, G., Greene, D., Cunningham, P.: Merging Multiple Criteria to Identify Suspicious Reviews. In: RecSys 2010, pp. 241–244 (2010)Xie, S., Wang, G., Lin, S., Yu, P.S.: Review Spam Detection via Time Series Pattern Discovery. In: Proceedings of the 21st International Conference Companion on World Wide Web, pp. 635–636 (2012)Zhou, L., Sh, Y., Zhang, D.: A Statistical Language Modeling Approach to Online Deception Detection. IEEE Transactions on Knowledge and Data Engineering 20(8), 1077–1081 (2008
Television Programming Trends from 1967-71: A Comparative Study of the Chicago and Milwaukee Markets
No full textIt is the purpose of this study to discover programming trends and patterns in television. The author will compare a pilot survey designed to observe programming in the Milwaukee television market, in 1971, with a similar study carried out in Chicago. Major topics include: 1) the development and establishment of methodology used in the Milwaukee survey which in turn was also used in the author\u27s subsequent Chicago study. 2) the collection of data implementing techniques developed in the research design. 3) a look at the individual program categories with regard to similarities and differences within the two television markets, and 4) final concluding statements and analyses
Word Length n-Grams for Text Re-use Detection
No full textThe automatic detection of shared content in written docu- ments –which includes text reuse and its unacknowledged commitment, plagiarism– has become an important problem in Information Retrieval. This task requires exhaustive comparison of texts in order to determine how similar they are. However, such comparison is impossible in those cases where the amount of documents is too high. Therefore, we have de- signed a model for the proper pre-selection of closely related documents in order to perform the exhaustive comparison afterwards. We use a sim- ilarity measure based on word-level n-grams, which proved to be quite effective in many applications As this approach becomes normally im- practicable for real-world large datasets, we propose a method based on a preliminary word-length encoding of texts, substituting a word by its length, providing three important advantages: (i) being the alphabet of the documents reduced to nine symbols, the space needed to store n-gram lists is reduced; (ii) computation times are decreased; and (iii) length n-grams can be represented in a trie, allowing a more flexible and fast comparison. We experimentally show, on the basis of the perplex- ity measure, that the noise introduced by the length encoding does not decrease importantly the expressiveness of the text. The method is then tested on two large datasets of co-derivatives and simulated plagiarism
- …
