1,720,987 research outputs found
sj-docx-1-ltj-10.1177_02655322231152620 – Supplemental material for Strategy use in a spoken dialog system–delivered paired discussion task: A stimulated recall study
No full textSupplemental material, sj-docx-1-ltj-10.1177_02655322231152620 for Strategy use in a spoken dialog system–delivered paired discussion task: A stimulated recall study by Nazlinur Gokturk and Evgeny Chukharev-Hudilainen in Language Testing</p
sj-pdf-1-wcx-10.1177_07410883211052104 – Supplemental material for A Product- and Process-Oriented Tagset for Revisions in Writing
No full textSupplemental material, sj-pdf-1-wcx-10.1177_07410883211052104 for A Product- and Process-Oriented Tagset for Revisions in Writing by Rianne Conijn, Emily Dux Speltz, Menno van Zaanen, Luuk Van Waes and Evgeny Chukharev-Hudilainen in Written Communication</p
A systemic functional perspective on automated writing evaluation: formative feedback on causal discourse
Get PDFMaking explanations is a very important communicative function in academic literacy; several disciplines including science are dominated by causal explanations (Mohan & Slater, 2004; Slater, 2004; Wellington & Osborne, 2001). For academic success, students need to write about causes and effects well with the help of their instructors, which means that formative assessment of causal discourse is necessary (Slater & Mohan, 2010). However, manual evaluation of causal discourse is time-consuming and impractical for writing instructors. For this reason, automated evaluation of causal discourse, which current automated writing evaluation (AWE) systems cannot perform, is required. Addressing these needs, this dissertation aimed to develop an automated causal discourse evaluation tool (ACDET) and empirically evaluate learners’ causal discourse development with ACDET in academic writing classes.
ACDET was developed using three approaches: a functional linguistic approach, a hybrid natural language processing approach combining rule-based and statistical approaches, and a pedagogical approach. The linguistic approach helped identify causal discourse features by analyzing a small corpus of texts about causes and effects of economic events. ACDET detects seven types of causal discourse features and generates formative feedback based on them: causal conjunctions, causal adverbs, causal prepositions, causal verbs, causal adjectives, and causal nouns. The natural language processing approach allowed for assigning part-of-speech tags to sentences and words and creating hand-coded rules for the detection of causal discourse features. The pedagogical approach determined feedback features of ACDET, and it was informed by the theoretical perspectives of the Interaction Hypothesis and Systemic Functional Linguistics and findings of research on causal discourse development.
Causal discourse development with ACDET was empirically evaluated through a qualitative study in which four research questions investigated two criteria of computer-assisted language learning evaluation framework: language learning potential (i.e., focus on causal discourse form, interactional modifications, and causal discourse development) and focus on causal meaning. Participants of the study were 32 English as a second language learners who were students in two academic writing classes. Data consisted of pre- and post-tests, ACDET’s text-level feedback reports, cause-and-effect assignment drafts, screen capturing recordings, semi-structured interviews, and questionnaires.
The findings indicate language learning potential of ACDET: ACDET drew learners’ attention to causal discourse form and created opportunities for interactional modifications, however, resulted in limited causal discourse development. Findings also reveal that ACDET drew learners’ attention to causal meaning.
This study is an important attempt in the field of AWE to analyze meaning in written discourse automatically and provide causal discourse specific feedback. The fact that empirical evaluation of ACDET was based on process-oriented data revealing how students used ACDET in class is noteworthy. The findings of this study have important implications for the refinement of ACDET, the development of AWE systems, and research on causal discourse development.</p
Requirement Text Detection from Contract Packages to Support Project Definition Determination
Get PDFProject requirements are wishes and expectations of the client toward the design, construction, and other project management processes. The project definition is typically specified in a contract package including a contract document and many other related documents such as drawings, specifications, and government codes. Project definition determination is critical to the success of a project. Due to the lack of efficient tools for requirement processing, the current practices regarding project scoping still heavily rely on a manual basis which is tedious, time-consuming, and error-prone. This study aims to fill that gap by developing an automated method for identifying requirement texts from contractual documents. The study employed Naïve Bayes to train a classification model that can be used to separate requirement statements from non-requirement statements. An experiment was conducted on a manually labeled dataset of 1191 statements. The results revealed that the developed requirement detection model achieves a promising accuracy of over 90%.This is a post-peer-review, pre-copyedit version of a proceeding published as Le, Tuyen, Chau Le, H. David Jeong, Stephen B. Gilbert, and Evgeny Chukharev-Hudilainen. "Requirement text detection from contract packages to support project definition determination." In: Advances in Informatics and Computing in Civil and Construction Engineering (2019): 569-576. The final authenticated version is available online at DOI: 10.1007/978-3-030-00220-6_68. Posted with permission.</p
The effect of automated fluency feedback on written text production
Get PDFFluency is undoubtedly an important aspect of written language production, but little is known about the best ways to encourage the fluent production of text. This article presents a new intervention for improving first language (L1) writing fluency and reports an empirical study investigating writing quality with this intervention. The intervention explicitly encourages fluent text production by providing automated real-time feedback to the writer. The design of this intervention was informed by previous studies on strategy-focused interventions and by two learning theories: skill acquisition theory and the cognitive process theory of writing. Guided by previous research and these theories, this study developed two research questions concerning the new intervention. These questions concerned the impact of this intervention on product and process measures of writing and on users' perceptions of the intervention.
To address these research questions, this study employed a mixed-methods approach. It collected quantitative and qualitative data from twenty native-English-speaking undergraduate students at a large Midwestern university. The quantitative data consisted of scores earned by the participants upon completing two writing tasks: one which included the new fluency intervention and one which served as the control condition. These tasks were conducted using an online text editor with embedded keystroke logging capabilities. Linear mixed-effect models were run to analyze the effect of the intervention on the final product of writing (i.e., the text that is produced) and the process of writing (i.e., the time-course of the moment-by-moment actions that taken to produce the text). Findings demonstrated that there were significant differences between the fluency intervention condition and the control condition in terms of the product and the process. Specifically, participants wrote more text, expressed more ideas, and produced a higher-quality text in the fluency intervention condition. The qualitative data consisted of responses to questionnaires in which participants reported their perceptions of the intervention upon completing it. They expressed some potential benefits of the intervention, including being able to think faster and generate more ideas, feeling motivated to write, and writing more intentionally. After presenting these findings in more detail, this thesis concludes by discussing potential practical applications of this intervention.</p
Designing, implementing, and evaluating an automated writing evaluation tool for improving EFL graduate students’ abstract writing: a case in Taiwan
Get PDFWriting English research article (RA) abstracts is a difficult but mandatory task for Taiwanese engineering graduate students (Feng, 2013). Understanding the current situation and needs of Taiwanese engineering graduate students, this dissertation aimed to develop and evaluate an automated writing evaluation (AWE) tool to assist their research article (RA) abstract writing in English by following a Design-Based Research (DBR) approach as the methodological framework. DBR was chosen because it strives to solve real-world problems through multiple iterations of development and building on results from each iteration to advance the project.
Six design iterations were undertaken to develop and to evaluate the AWE tool in this dissertation, including (1) corpus compilation of engineering RAs, (2) genre analysis of engineering abstracts, (3) machine learning of move classification in abstracts, (4) analysis of lexical bundles used to express moves, (5) analysis of the choice of verb categories associated with moves, and finally, (6) AWE tool development based on previous findings, classroom implementation, and evaluation of the AWE tool following Chapelle’s (2001) computer-assisted language learning (CALL) framework.
To begin with, I collected a corpus of 480 engineering RAs (Corpus-480) to extract appropriate linguistic properties as pedagogical materials to be implemented in the AWE tool. A sub-corpus (Corpus-72) was compiled with 72 RAs randomly chosen from Corpus-480 for manual and automated analyses. Next, to seek the best descriptive framework for the structure of engineering RA abstracts, two move schemata were compared: (1) IMRD (Introduction, Methodology, Results, and Discussion) and (2) CARS (Create-A-Research-Space, Swales, 1990). Abstracts in Corpus-72 were annotated and these two schemas were evaluated according to three quantitative metrics devised specifically for this comparison.
Applying a statistical natural language processing (StatNLP) approach, a Support Vector Machine (SVM) was trained for automated move classification in abstracts. Formulaic language in engineering RA sections was used as linguistic features to automatically classify moves in abstracts. Additionally, four-word lexical bundles and verb categories were identified from Corpus-480 and Corpus-72, respectively. Four-word lexical bundles associated with moves in abstracts were extracted automatically. Additionally, verb categories (i.e., tense, aspect, and voice) in moves of abstracts were identified using CyWrite::Analyzer, a hybrid (statistical and rule-based) NLP software.
Finally, the AWE tool was developed, based on the findings from the previous iterations, and implemented in an English-as-a-foreign-language (EFL) classroom setting. Through analyzing students’ drafts before and after using the tool, and responses to a questionnaire and a semi-structured interview, the AWE tool was evaluated based on Chapelle’s (2001) CALL evaluation framework. The findings showed that students attempted to improve their abstracts by adding, deleting, or changing the sequences of their sentences, lexical bundles, and verb categories in their abstracts. Their attitudes toward the effectiveness and appropriateness of the tool were quite positive. Overall, the AWE tool drew students’ attention to the use of lexical bundles and verb categories to achieve the communicative purposes of each move in their abstracts.
In conclusion, this dissertation started from Taiwanese engineering students’ needs to improve their English abstract writing, and attempted to develop and evaluate an AWE tool for assisting them. Following DBR, the findings from this dissertation are discussed to improve the next generation of the AWE tools. Having these iterations in place, future studies can focus on developing pedagogical materials from genre-based analysis in different disciplines to fulfill learners’ needs.</p
Semantic relatedness in L2 vocabulary learning: Does it really matter?
Get PDFSecond language (L2) textbooks often organize new vocabulary in lists of semantically related words under a common superordinate concept, such as food or family members. However, research on this topic has shown mixed results, with some studies suggesting that related lists facilitate learning, and others showing inhibiting effects. Importantly, all studies to date have been carried out in a laboratory or strictly controlled classroom setting where individual differences among students are often controlled for. Given that these differences may result in different learning gains in the authentic classroom environment compared to a controlled setting, the potential effects of semantic relatedness on vocabulary acquisition may similarly manifest differently when students are left to their own devices. This thesis reports on the first empirical study (to the author's knowledge) to test the effects of semantic relatedness on vocabulary learning in a truly authentic classroom environment. Two hundred and twelve students in beginner- and intermediate-level Spanish classes at Iowa State University were tested on their ability to translate items from one related list and one unrelated list from their course textbooks near the end of their respective units. Data were analyzed using mixed-effects logistic regression models under strict and sensitive scoring protocols.
Results indicated no evidence for a significant difference between scores on related and unrelated lists. Further regression analysis indicated a significant effect of individual lexical items on the learning outcomes, and item analyses suggested that some control over item-level characteristics may be needed to facilitate research even in the authentic classroom environment. Implications for teachers, materials developers, and researchers are discussed.</p
Not all word stress errors are created equal: Validating an English Word Stress Error Gravity Hierarchy
Get PDFSpoken language has no spaces between its words. Therefore, one of the major tasks facing listeners of any language is determining from the largely continuous stream of speech where the invisible word boundaries lie. Although English is not a language where the position of a word's stressed syllable is reliably fixed, its lexical stress is nevertheless fixed enough that L1 English listeners initially apply the heuristic that strong syllables mark the first syllable of a new word, attempting alternative resegmentations only when this heuristic fails to identify a viable word string (Cutler & Butterfield, 1992; Cutler & Carter, 1987). Thus, English word stress errors can severely disrupt listener processing. This study uses auditory lexical decision and delayed word identification tasks to test a hypothesized English Word Stress Error Gravity Hierarchy synthesizing previous research that has identified vowel quality (Bond, 1979, 1999; Bond & Small, 1983; Cutler, 2012, 2015) and direction of stress shift (Cutler & Clifton, 1984; Field, 2005) as key predictors for the intelligibility (Munro & Derwing, 1995, 2006) of nonstandard stress pronunciations. Results indicate that English word stress errors, when they introduce concomitant vowel errors, matter – and that the intelligibility impact of any particular lexical stress error can indeed be predicted for both L1 and L2 English listeners by this study’s English Word Stress Error Gravity Hierarchy. These findings have implications for L1 and L2 English pronunciation research, teaching, and testing.</p
Going Beyond Counting First Authors in Author Co-citation Analysis
Get PDFThe present study examines one of the fundamental aspects of author co-citation analysis (ACA) - the way co-citation
counts are defined. Co-citation counting provides the data on which all subsequent statistical analyses and mappings
are based, and we compare ACA results based on two different types of co-citation counting - the traditional type that
only counts the first one among a cited work's authors on the one hand and a non-traditional type that takes into
account the first 5 authors of a cited work on the other hand. Results indicate that the picture produced through this non-traditional author co-citation counting contains more coherent author groups and is therefore considerably clearer. However, this picture represents fewer specialties in the research field being studied than that produced through the traditional first-author co-citation counting when the same number of top-ranked authors is selected and analyzed. Reasons for these effects are discussed
- …
