DR-NTU (Digital Repository of NTU)
Not a member yet
116018 research outputs found
Sort by
Empowering natural language processing in low-resource regimes
No full textAs a vital subfield of artificial intelligence (AI), natural language processing (NLP) strives to enable computers to understand, process, and analyze text as humans do. NLP has a broad range of applications, including text classification, information extraction, and chatbot dialogue systems. However, most NLP models demand substantial amounts of training data, which is often labor-intensive and expensive to collect and annotate. This issue, known as the low-resource NLP challenge, is defined as the lack of sufficient data and is considered one of the most critical obstacles in NLP.
To tackle the low-resource NLP challenge, existing solutions either expand the training data or reduce data requirements by enhancing the training efficiency. Techniques like data augmentation, distant supervision, and semi-supervised learning generate synthetic data to increase the training set, enabling NLP models to achieve greater generalization across various NLP tasks. Sequential transfer learning methods, including fine-tuning and prompt-based learning, substantially boost the learned text representations with limited data, thereby reducing the overall data demand. These approaches employ lightweight strategies to add adaptation modules or adjust the parameters of pre-trained language models (PLMs), effectively tailoring powerful PLMs to specific downstream NLP tasks.
Despite their success, most of these solutions overlook interpretability in their development, failing to create more targeted approaches for NLP models. The first category of interpretability is local explanations, which elucidate the model's decision-making process for individual inputs. By understanding how different input words influence model outputs, we can design more efficient strategies for utilizing and processing limited training data. The second category is global explanations, which analyze how the model's overall structure, weights, and parameters affect its prediction process. Leveraging these explanations enables us to devise better strategies for adapting PLMs to NLP tasks, thereby reducing the data requirements even further. However, effectively integrating interpretability into the development of low-resource NLP solutions remains an unresolved research question.
This thesis aims to integrate interpretability into the design of more targeted methods to address the low-resource challenge. By leveraging both local and global explanations, we develop efficient strategies for data augmentation, fine-tuning, and prompt-based learning across various natural language understanding (NLU) and natural language generation (NLG) tasks. Specifically:
Tailored Text Augmentation Using Local Explanations. We employ local explanations, such as the word's importance and discriminative power for the prediction, to tailor data augmentation operations for different word types. Discriminative words are used to introduce more task-oriented knowledge into synthetic data, while irrelevant words are controlled to be evenly distributed in synthetic data for low-resource sentiment analysis.
Boosting Fine-Tuning with Knowledge-Driven Interpretability. We utilize external knowledge and local explanations of word importance to create text-saliency graphs for each input text. These graphs are encoded and combined with the original text representations to enhance the discriminative power of text representation learning, significantly improving model performance in low-resource hierarchical text classification (HTC).
Enhanced Input Saliency for Prompt Manipulation. We develop a simple yet efficient local explanation method that leverages token distribution dynamics (TDD) to elucidate the prompt’s influence on large language model (LLM) outputs. Using these reliable local explanations derived from input saliency, we manipulate prompts by identifying and modifying keywords in input prompts for zero-shot controllable text generation (CTG).
Learning-Free Text Generation with Global Interpretability. We investigate global explanations, focusing on how the weights of feed-forward network (FFN) modules in LLMs impact their outputs. We then establish control centers using these FFNs and adaptively update their weights for CTG. Remarkably, our proposed method does not require any training data or learning process, significantly mitigating the low-resource challenge.Doctor of Philosoph
Effective structural impact detection and localization using convolutional neural network and Bayesian information fusion with limited sensors
No full textDue to their unpredictable nature, many impact events (e.g., overheight vehicles striking on bridges) go unnoticed or get reported many hours later. However, they can induce structural failures or hidden damage that accelerates the structure's long-term degradation. Therefore, prompt impact detection and localization strategies are essential for early warning of impact events and rapid maintenance of structures. Most existing impact detection strategies are developed for aircraft composite panels utilizing high-rate synchronized measurement from densely deployed sensors. Limited efforts have been made for infrastructure or human habitats which generally require large-scale but low-rate measurement. In particular, due to harsh environments (e.g., deep space habitats under meteoroids), structural impact localization must be robust to limited sensors (e.g., sensor damage during impacts) and multi-source errors (e.g., measurement errors). In this study, an effective impact detection and localization strategy is proposed using a limited number of vibration measurements, especially in harsh environments (e.g. in deep space). Convolutional neural networks are trained for each sensor node and are fused using Bayesian theory to improve the accuracy of impact localization. Special considerations are paid to evaluate the effect of both measurement error and modeling error in the analysis. The proposed strategy is illustrated using 1D structure, and further validated in 3D geodesic dome structure numerically. The results demonstrate that it can detect and localize impact events accurately and robustly on structures.Ministry of Education (MOE)Nanyang Technological UniversityThe authors gratefully acknowledge the support of this research by a Space Technology Research Institutes Grant (No. 80NSSC19K1076) from NASA’s Space Technology Research Grants Program, the start-up grant at Nanyang Technological University, Singapore (03INS001210C120), and the Ministry of Education Tier 1 Grants, Singapore (No. RG121/21)
Exploration of a new load detect method for wireless power transfer system—third harmonic analysis
No full textWireless power transfer (WPT) is an emerging energy transmission method that can transmit energy omnidirectionally or directionally. WPT systems mainly use electromagnetic fields and operate through two main technologies: inductive coupling and resonant inductive coupling, and are widely used in various applications ranging from small electronic devices to electric vehicle charging. The development of efficient and safe WPT technology has the potential to revolutionize the industry by providing a more convenient, flexible and safer way to transmit energy. At present, directional power transmission has become a hot topic of research due to its high efficiency. To achieve directional energy transmission, it becomes crucial to accurately detect the energy demand of the load and its relative position.
In this paper, a load detection method based on third harmonic analysis is designed. By analyzing the third harmonic of the primary voltage and current, the load resistance and position information can be accurately obtained. A two-dimensional WPT system is established in ANSYS, and the inductance values of different positions of the receiving coil are obtained by simulation. In addition, a simulation model is constructed in MATLAB to verify the feasibility of the third harmonic analysis method. The experimental results show that in most cases, it is feasible to use third harmonic analysis for load detection of WPT system, and the results have high accuracy.Master's degre
Building trustworthy AI from small DNNs to large language models: a software engineering perspective
No full textAs Artificial Intelligence (AI) software becomes increasingly prevalent across various industries, concerns about its trustworthiness and reliability have come to the forefront. Although the trustworthiness of traditional software is regulated by Software Engineering (SE) practices, these practices have not been well integrated into AI model development due to the significant differences between traditional software development and AI model development. Inspired by this, we aim to systematically address trustworthiness by regulating the AI development process through the lens of SE practices. Specifically, we are inspired by the regulation of traditional software, focusing on the key phases in software regulation: software development, execution, and testing. We identify corresponding phases in AI model development: training, inference, and testing. These phases are crucial for ensuring the trustworthiness and reliability of AI models. My study aims to improve these phases to enhance the trustworthiness of AI models. Our primary approach to regulating AI model development mirrors traditional software practices. It involves first debugging these phases and then implementing repairs. Moreover, large language models (LLMs) are revolutionizing the software industry. Thus, in this thesis, I explore the debugging and repairing of AI software from three phases (i.e., training, inference, and testing), focusing on both small Deep Neural Networks (DNNs) and LLMs.Doctor of Philosoph
Optimizing dielectric, mechanical, and thermal properties of epoxy resin through molecular design for multifunctional performance
No full textEpoxy resins are widely used as dielectric materials in electrical and electronic systems. However, the trend of miniaturization of electronic devices and increasing power output of electrical equipment have created new challenges for dielectric materials, necessitating low dielectric constants, high breakdown strength, and high electrical resistivity. This study introduces three molecular modifications to epoxy resin systems using facile synthesis procedures, including modifiers with bulky groups and crosslinking potential to reduce the dielectric constant while enhancing mechanical and thermal reliability, along with deep traps to increase breakdown strength. The modified epoxy resins exhibit significant improvements. Notably, epoxy/amine resin grafted with only 0.5 wt% maleic anhydride demonstrates a 30% decrease in dielectric constant, a 17-fold increase in volume resistivity, an increase in dielectric breakdown strength from 61.5 to 73.4 kV mm-1, and a rise in tensile strength from 69.7 to 75.4 MPa. Other modifiers also show enhancements in dielectric, mechanical, thermal, and water uptake properties. Fourier-transform infrared spectroscopy (FTIR), scanning electron microscopy (SEM), and energy-dispersive X-ray spectroscopy (EDX) are employed to reveal the chemical structure of the modified epoxy resin and the distribution of modifiers. Results confirm successful grafting and exceptional dispersion without agglomeration. This study demonstrates that small amounts of chemical modifiers can significantly enhance epoxy resin performance. The resulting materials can meet the requirements for next-generation dielectric materials while maintaining low production costs.Energy Market Authority (EMA)Nanyang Technological UniversityNational Research Foundation (NRF)Submitted/Accepted versionThis research is supported by SP Group, the National Research Foundation, Singapore, the Energy Market Authority, under its Energy Programme (EMA-EP010-SNJL-002) and Nanyang Technological University
Unravelling the role of filler surface wettability in long-term mechanical and dielectric properties of epoxy resin composites under hygrothermal aging
No full textEpoxy resin (EP) incorporating inorganic fillers has garnered significant attention in the electrical and electronic industries due to its enhanced dielectric and mechanical properties, but its long-term performance under harsh conditions remains a critical concern. This study investigates the effects of filler surface wettability on the durability of EP-SiO2 composites. Micro-sized SiO2 with hydrophilic (HP) and hydrophobic (HB) surfaces are prepared via surface treatment, before they are incorporated into epoxy resin and subjected to hygrothermal aging at 95 °C and 95 % relative humidity for up to 1200 h. Comprehensive characterizations of wettability, microstructure, mechanical properties, and dielectric performance are conducted. Results show that the composite with hydrophilic fillers, HP-SiO2-EP, exhibits superior dispersion and interfacial adhesion compared to its hydrophobic counterpart, HB-SiO2-EP. Consequently, HP-SiO2-EP demonstrates higher initial tensile strength, Young's modulus, and dielectric breakdown strength. Finite element simulations reveal the breakdown mechanism, highlighting that the hydrophobic SiO2 filler with interfacial defects results in earlier mechanical and dielectric failure. Furthermore, HP-SiO2-EP shows better resistance to hygrothermal aging compared to HB-SiO2-EP, with smaller increases in dielectric constant (+13 % vs. +28 %) and dielectric loss (+234 % vs. +311 %), as well as lower decrease in volume resistivity (-89 % vs. -93 %). This study provides valuable insights into the relationship between filler surface wettability and long-term composite performance, contributing to the design of more reliable materials for advanced dielectric applications.Energy Market Authority (EMA)Ministry of Education (MOE)National Research Foundation (NRF)Submitted/Accepted versionThis research is supported by SP Group, the National Research Foundation, Singapore, the Energy Market Authority, under its Energy Programme (EMA-EP010-SNJL-002) and Ministry of Education, Singapore (RG7/24)
Data efficient training for egocentric vision-based action recognition
No full textWe investigate the application of semi-supervised learning in egocentric action anticipation to tackle the issue of limited labeled data. Leveraging both fully labeled and pseudo-labeled data for training can effectively improve model performance, especially when fully labeled data is scarce. We implement this strategy using two advanced transformer-based models, the Memory-and-Anticipation Transformer (MAT) and the Anticipative Feature Fusion Transformer (AFFT), both of which are tailored for capturing intricate temporal dependencies within egocentric video data. Experimental evaluations on the Epic-Kitchens-100 and EGTEA Gaze+ datasets reveal that the semi-supervised approach yields notable improvements in action anticipation accuracy compared to models trained exclusively on limited labeled data. Importantly, performance gains are most significant under highly constrained data settings, emphasizing the practicality of semi-supervised learning in scenarios where labeled data is limited or costly to obtain. This study highlights the promise of integrating semi-supervised learning with specialized models to advance action anticipation capabilities in egocentric video tasks.Master's degre
Whither US-China relations under Trump 2.0?
No full textDonald Trump’s second presidency renewed the focus on US-China relations long strained by trade disputes, China’s military growth, Taiwan, and the South China Sea. While Trump’s return may bring about escalations in tensions, his unpredictability and expanded powers could worsen ties or create unexpected opportunities for improvement.Published versio
Deception detection in videos
No full textThis study explores a multi-modal approach for deception detection in videos, classi-
fying videos as deceptive or truthful by using four modalities: video, audio, transcript,
and micro-expressions. The video modality captures body language and gestures, while
audio is used to analyze vocal variations such as pitch and stutters. Transcript data
analyses linguistic patterns and semantic choices, while micro-expressions are used
to detect subtle facial cues that may indicate underlying emotions. To integrate these
diverse data sources, we investigate the use of techniques such as early and late fusion,
alongside neural approaches such as multi-layer perceptrons and convolutional neural
networks.Bachelor's degre
Enhancing the quality of tuition services for all
No full textTuition services has been increasingly popular due to the demand for a better education outside of school. Many websites that provide tuition services are in the market now such as MindFlex, SmileTutor and ChampionTutor. However, these existing solutions are not adequate in creating a convenient environment and good quality experience for users.
This project aims to create an improved version of tuition websites that will cater to the needs of both teachers and students by incorporating new advanced features. This new system will allow separate login for teachers and students, creating a more personalised user experience. It will also have a chatbot powered by Artificial Intelligence (AI) to allow users to ask any questions. Personalised content will be available for users to browse and sign up for. The website also aims to make learning more interesting and enhance the quality of tutoring experience through a gamification effect.
The platform will be built using MERN Stack to ensure a reliable and scalable web application. Surveys have been conducted, and the results show that so far, the functionalities of the website have improved the optimization of interactions between users as well as proven to be user-friendly (Appendix A).Bachelor's degre