41 research outputs found
Handling highly imbalanced output class label: a case study on Fantasy Premier League (FPL) virtual player price changes prediction using machine learning / Muhammad Muhaimin Khamsan and Ruhaila Maskat
Get PDFIn practice, a balanced target class is rare. However, an imbalanced target class can be handled by resampling the original dataset, either by oversampling/upsampling or undersampling/downsampling. A popular upsampling technique is Synthetic Minority Over-sampling Technique (SMOTE). This technique increases the minority class by generating synthetic class labels and assigned the class based on the K-Nearest Neighbour (K-NN). SMOTE upsampling can only upsample at most one minority class at a time, which means for a multiclass dataset, it needs to undergo multilayer SMOTE to balance the class label distribution. This paper aims to find a suitable method in handling imbalanced class using dataset from Fantasy Premier League (FPL) virtual player to predict price changes. The cleaned dataset has a highly imbalanced class distribution, where the frequency of “Price Remain Unchanged (PRU)” is higher than “Price Fall (PF)” and “Price Rise (PR)”. This paper compared between the baseline (original) dataset, SMOTE-applied dataset and shuffled, linear and stratified sampling in split train-test subset, based on a deep learning algorithm. This paper also proposed criteria of low values in standard deviation (distribution of true positive on each class label on accuracy) as a measurement for finding the best method in handling imbalanced class labels. As a result, multilayer SMOTE until all the classes distribution is the same, combined with stratified sampling in split training and testing subset, get the lower standard deviation (5.7873), high accuracy (80.06%) and less execution runtime (1 minute 41 seconds) compared to the original highly imbalanced dataset
A Bi-Annotated Malay-English Code-switching Dataset of X posts (Formerly Twitter) for Biological Gender Identification and Authorship Attribution
No full textThis dataset is conditionally free. To use it, simply cite the companion paper. Information on the paper can be found in the "Citation" box. Thank you
Enhancing Waste Detection: A Novel Aerial View Dataset for Machine Learning on Paper Home Waste in Low Light and Blurred Environments
No full textThis dataset is conditionally free. Its companion paper is being reviewed. Please come back and view the "Citation" box for the upcoming reference detail. Thank you for your understanding
A taxonomy of Malay social media text
Get PDFIn this paper, we proposed a preliminary taxonomy of Malay social media text. Performing text analytics on Malay social media text is a challenge. The formal Malay language follows specific spelling and sentence construction rules. However, the Malay language used in social media differs in both aspects. This impedes the accuracy of text analytics. Due to the complexity of Malay social media text, many researches has chosen to focus on classifying the formal Malay language. To the best of our knowledge, we are the first to propose a formal taxonomy for Malay text in social media. Narrow and informal categorisations of Malay social media text can be found amidst efforts to pre-process social media text, yet cherry-picked only some categories to be handled. We have differentiated Malay social media text from the formal Malay language by identifying them as Social Media Malay Language or SMML. They consists of spelling variations, Malay-English mix sentence, Malay-spelling English words, slang-based words, vowel-les words, number suffixes and manner of expression.This taxonomy is expected to serve as a guideline in research and commercial products
Psychometric properties of depression anxiety stress scale 21-item (DASS-21) Malay version among a big sample population
No full textEstablishing the psychometric properties of a measurement tool is paramount in order to determine the validity of any study results. The Depression Anxiety Stress Scale (DASS)-21 has preliminary evidence of a good tool in Malaysia. This study among a very big sample size would further strengthen this evidence. Objectives: To validate the DASS-21 Malay version among 367, 860 completed online assessment by looking at its Exploratory Factor Analysis and Construct validity. Methodology: The data was captured through online web-based assessment. Incomplete entries were excluded and analysis was based on 367, 860 completed entries which fulfilled the inclusion criteria. We obtained Cronbach’s
alpha values of 0.95, 0.85 and 0.87 respectively for Depression, Anxiety and Stress domains. Rotated component analysis showed very good factor loading for most items to their respective domains. The non-fixed analysis showed the Malay DASS-21 consisted of 2 domains and Stress items loaded equally to Depression and Anxiety domains. The current findings further enhanced the evidence that (DASS)-21 is an excellent tool for Malaysians
A review of visualization techniques for duplicate detection in cancer datasets
Get PDFAs clinical cancer research increasingly depends on large, diverse datasets, concerns about data duplication have grown. Duplicates can undermine data integrity, skew analytical results, and reduce the reproducibility of studies. This review explores how visualization can play a critical role in identifying and managing duplicates in non-image clinical cancer data. Drawing from literature in biomedical informatics, data quality, and visual analytics, it synthesizes current approaches and highlights key challenges. Using a scoping review methodology, we analyzed studies published over the past two decades, focusing on non-image clinical datasets. Studies were selected based on relevance to duplicate detection and visualization, excluding those centered on image or video data. Major datasets like The Cancer Genome Atlas (TCGA), The Cancer Imaging Archive (TCIA), and the North American Association of Central Cancer Registries (NAACCR) are examined to show how duplication occurs across genomic, clinical, and registry data. The review assesses existing visualization techniques based on their scalability, interactivity, integration with deduplication algorithms, and how well they address core data quality dimensions. While some tools offer scalable and interactive features, few provide clear visual representations of duplicates, especially those involving complex temporal and multidimensional patterns. Several methodological gaps are identified, including limited integration of data quality metrics, inadequate support for tracking changes over time, and a lack of standardized evaluation frameworks. To address these issues, the review advocates for the development of practical, user-friendly visualization tools that combine duplicate detection with key indicators of data quality. By offering a more complete and intuitive view of clinical datasets, such tools can help researchers and clinicians make better-informed decisions, ultimately improving the reliability and impact of cancer research. Bridging the gap between technical detection and visual understanding is essential for advancing data-driven healthcare and ensuring high-quality, reproducible outcomes
A scoping review of topic modelling on online data
Get PDFWith the increasing prevalence of unstructured online data generated (e.g., social media, online forums), mining them is important since they provide a genuine viewpoint of the public. Due to this significant advantage, topic modelling has become more important than ever. Topic modelling is a natural language processing (NLP) technique that mainly reveals relevant topics hidden in text corpora. This paper aims to review recent research trends in topic modelling and state-of-the-art techniques used when dealing with online data. Preferred reporting items for systematic reviews and meta-analysis (PRISMA) methodology was used in this scoping review. This study was conducted on recent research works published from 2020 to 2022. We constructed 5 research questions for the interest of many researchers. 36 relevant papers revealed that more work on non-English languages is needed, common pre-processing techniques were applied to all datasets regardless of language e.g., stop word removal; latent dirichlet allocation (LDA) is the most used modelling technique and also one of the best performing; and the produced result is most evaluated using topic coherence. In conclusion, topic modelling has largely benefited from LDA, thus, it is interesting to see if this trend continues in the future across languages
A prototype tool to assess the functional quality of e-commerce websites
Get PDFE-Commerce website is the new marketing and sales method to reach new customers.By overcoming the geographical constraint, e-commerce website allows users to perform purchasing activities without being physically present.Though many e-commerce websites have achieved great success, many have also failed.Their failures are mainly due to the
absence of quality.This study presents a prototype software tool that could assist in assessing the functional quality of e-commerce websites in an automatic manner.Unlike the currently available methods of evaluation, this tool does not necessitate the assessment of e-commerce websites by a human expert.On the other hand, the tool performs a scanning activity to allocate the existence of e-commerce characteristics.Adopting the quality model by Stefani and Xenos (2001), six different e-commerce characteristics were selected from the functional factor of the model.Three other factors identified by the model are usability, reliability and efficiency, which are not within the scope of this project.Three major script languages have been sited to be regularly used in e-commerce website development.They are JavaServer Pages (JSP) from Sun Microsystems, ASP from Microsoft and PHP an open source language.Though they share the same functionalities, they differ considerably in syntax.Therefore, this project has focused on evaluating e-commerce websites, which were built using JavaServer Pages (JSP).The objective of this paper is to develop a prototype tool capable of evaluating JSP-based e-commerce website based on the characteristics chosen.The tool is competent to perform the data collection through fully automatic means and finally generates result of evaluation into XML format
