NII Repository (National Institute of Informatics)
Not a member yet
2035 research outputs found
Sort by
IDEA at the NTCIR-17 FinArg-1 Task: Argument-based Sentiment Analysis
Get PDFAlthough argument mining has been discussed for several years, financial argument mining is still in the early stage. The IDEA team participates in Argument Unit Classification (for Earnings Conference Call) and Argument Relation Classification (for Earnings Conference Call) subtasks of the NTCIR-17 FinArg-1 Task. This paper presents our work on the two subtasks. For Argument Unit Classification subtask, we successively construct the models based on BERT and Roberta to classify a given argumentative sentence. To better extract the semantic features, we combine the pre-trained model with CNN.Micro-F1 and Macro-F1 achieve 76.47% and 76.46% in official evaluation results of the first run (i.e., IDEA-1), respectively, outperforming most approaches of other teams. For Argument Relation Classification subtask, we classify sentence pairs based on the pre-trained model and Prompt-Tuning. And Micro-F1 and Macro-F1 achieve 81.74% and 51.85% in official evaluation results of the third run (i.e., IDEA-3), respectively.conference pape
Zero-shot classification of TNM staging for Japanese radiology report using ChatGPT at RR-TNM subtask of NTCIR-17 MedNLP-SC
Get PDFWe describe our submission to the RR-TNM subtask of the NTCIR-17 MedNLP-SC shared task. In the RR-TNM subtask, we developed our system for automatic extraction and classification of the TNM staging from Japanese radiology reports of lung cancers. In our system, zero-shot classification and prompt engineering were performed using ChatGPT and LangChain, respectively. According to the accuracies calculated by the organizers of the RR-TNM subtask, the accuracies of N and M factors in the TNM staging were higher in our submission than in the other submissions. These results indicate that our system with ChatGPT and LangChain may be promising.conference pape
Omuokdlb at the NTCIR-17 QA Lab-Poliinfo-4 Task
Get PDFThe omuokdlb team participated in two subtasks in NTCIR 17 QA Lab-Poliinfo-4: Question Answering-2 and Answer Verification. In Question Answering-2, we use Bidirectional Encoder Representations from Transformers (BERT) to match the question summary and the answer utterances. Then, we generated a summary of the answer to the question by using Text-to-text Transfer Transformer (T5). In Answer Verification, we created binary classifiers using BERT to determine whether or not answers, and we confirmed the effectiveness of the combination of the training data.conference pape
AKBL at the NTCIR-17 QA Lab-PoliInfo-4 Task
Get PDFAKBL team participated in the Question Answering-2, Answer Ver- ification, Stance Classification-2, and Minutes-to-Budget Linking subtasks. For the Question Answering-2 subtask, Our system ex- tracts relevant transcripts from question metadata and summarizes them using a T5 model pre-trained in Japanese. For the Answer Verification subtask, our method first generates the pseudo-fake data automatically by round-trip translation, and then fine-tunes the pre-trained BERT with the training data and pseudo-fake data. For the Stance Classification-2 subtask, our best system is a binary classifier using RoBERTa. For the Minutes-to-Budget Linking sub- task, it was realized using a ranking method based on Okapi BM25.conference pape
Quack at the NTCIR-17 FinArg-1 Task : Boosting and MLM Enhanced Financial Knowledge Sequence Classification
Get PDFIn the exploration of the task "Identifying Attack and Support Argumentative Relations in Social Media Discussion Threads" (SMDT) [1], we aim to discern differences between proponents and adversaries in financial discussions on the internet. For classification tasks such as these, fine-tuning Transformer models like BERT [2] is an intuitive approach. In this study, we build upon this foundation by incorporating the Masked Language Model technique to enrich the model’s domain knowledge within the financial field. Furthermore, we optimize the model’s performance by adjusting the weights in the loss function. Experimental results confirm that both methods effectively enhance the model’s performance. This research introduces three simple yet effective methods to improve the Transformer model’s ability for SMDT. The code and model for this study are available at https://github.com/leonardo-lin/NTCIR.conference pape
RMIT_IR at the NTCIR-17 FairWeb-1 Task
Get PDFThis report describes the participation of the RMIT IR group at the NTCIR-17 FairWeb-1 task. We submitted five runs with the aim of exploring the role of explicit search result diversification (SRD) and ranking fusion to generate fair rankings considering multiple fairness attributes. We also explored the use of a linear combination-based technique (LC) to take into consideration the relevance while re-ranking. In this report, we compared results from all our submitted runs against each other and the retrieval baselines along each topic type separately (i.e., Researcher, Movie, YouTube). Overall, our results show that neither the SRD-based runs nor the linear combination-based runs show any statistically significant improvement over the retrieval baselines. The source code of the framework for generating group memberships is made available at https://github.com/rmit-ir/fairweb-1.conference pape
Overview of the NTCIR-17 FairWeb-1 Task
Get PDFThis paper provides an overview of the NTCIR-17 FairWeb-1 Task. FairWeb-1 is an English web search task which seeks more than an ad-hoc web search task. Our task considers not only document relevance but also group fairness.We designed three types of search topics for this task: researchers (R), movies (M), and Youtube contents (Y). For each topic type, attribute sets are defined for considering group fairness. We utilise a deduped version of the Chuweb21 corpus as the target corpus. We received 28 runs from six teams, including six runs from the organisers team. In this paper, we describe the task, the test collection construction and the official evalution results of the submitted runs.conference pape
Decoy Effect in Search Interaction: A Pilot Study
Get PDFIn recent years, the influence of cognitive effects and biases on users' thinking, behaving, and decision-making has garnered increasing attention in the field of interactive information retrieval. The decoy effect, one of the main empirically confirmed cognitive biases, refers to the shift in preference between two choices when a third option (the decoy) which is inferior to one of the initial choices is introduced. However, it is not clear how the decoy effect influences user interactions with and evaluations on Search Engine Result Pages (SERPs). To bridge this gap, our study seeks to understand how the decoy effect at the document level influences users' interaction behaviors on SERPs, such as clicks, dwell time, and usefulness perceptions. We conducted experiments on two publicly available user behavior datasets and the findings reveal that, compared to cases where no decoy is present, the probability of a document being clicked could be improved and its usefulness score could be higher, should there be a decoy associated with the document.conference pape
