82837 research outputs found
Sort by
Understanding RAG Systems Performance by Profiling Key Factors
No full textThe launch of ChatGPT in 2022 brought Large Language Models (LLMs) into the spotlight, establishing them as powerful tools across various fields. To further augment and enhance the performance of LLM in knowledge-instense tasks, RetrievalAugmented Generation (RAG) has emerged as a key technique, significantly enhancing LLM performance across diverse applications. While RAG systems powered by LLMs achieve remarkable results, they face persistent challenges in stability and reliability due to their intricate design. Developing a RAG system involves numerous design decisions, such as retriever selection, query optimization, retrieved material curation, and prompt configurations, creating a vast design space that is impractical to explore exhaustively. These complexities pose substantial challenges for developers in constructing, maintaining, and optimizing effective RAG systems.
In this work, we introduce a novel system factor analysis framework for understanding RAG system behavior. Rather than evaluating every design decision combination, we abstract them into observable system factors that provide a structured and interpretable lens for analysis. Through extensive experiments on three code datasets, three QA datasets, and two LLMs, we examine three critical factors: retrieval recall, document type, and prompt techniques. Furthermore, we explore the possibility of utilizing perplexity as a proxy metric for monitoring system factors in RAG systems, providing a real-time and low-cost way to interpret RAG system behavior and identify optimization directions.
Our study provide a new framework to understand RAG system behavior. By analyzing how these system factors influence RAG systems, we provide key insights into their characteristic and derive multiple implications for building more reliable and effective designs. Our exploration of perplexity points to a promising approach for monitoring RAG systems in certain contexts. We hope this foundational work inspires further advancements in developing and sustaining intelligent, LLM-driven systems with greater efficiency and reliability
Stability Response of Double-overhanging Steel Girders
No full textOverhanging steel girders are key components of Gerber roof framing systems, which are widely used in large single-storey steel buildings. These girders extend beyond and over the columns as cantilevers and support intermediate drop-in segments. Their continuity introduces negative moments into the system, reducing the magnitudes of positive moment and creating a more balanced moment distribution. This results in an efficient structural system, allowing lighter girders to carry the same loads as compared to simply supported girders. Additional advantages include simple and repeatable connections, and reduced deflections owing to the continuity at the column locations. However, current steel design standards provide limited guidance for Gerber systems, leading to inconsistent applications of design requirements. Such inconsistencies have contributed to multiple collapses throughout North America, highlighting the need for a comprehensive and reliable design method to enable safe and efficient designs of overhanging girders, which can be particularly susceptible to lateral–torsional buckling. To improve the understanding of the stability response of double overhanging girders, an experimental testing program was conducted. Nine W460x97 ASTM A992 steel girders, each with 1.83 m cantilevers and a 9.14 m back span, were tested under various loading and restraint conditions. This study examined the effects of lateral bracing, bending moment distribution, initial geometric imperfections, and residual stresses on girder capacities and deflections. The experimental resistances were compared with predictions from three potential design approaches for overhanging girders. The results provide new insights into the stability behaviour of overhanging girders and support the development of a more reliable and efficient design procedure for Gerber roof framing systems
Crafting a Hex Solver with Connection-Search and Expected-Work-Search
No full textHex is the 2-player alternate-turn perfect-information board game created by Piet Hein in 1942. The board is an m×n array of Hexagonal cells that forms a parallelogram, or a rhombus when m equals n. Each player owns two opposing edges of the
board. On a turn, a player colors an empty cell. The winner is whoever joins their two edges with a path of their color.
Connection Search refers to any Hex-specific method to find connections. Threat Pattern Search, proposed by Jack van Rijswijck in 2000, finds cells belonging to opponent threats. Hierarchical Search, proposed by Vadim Anshelevich in 2002, finds point-to-point connections.
Expected Work Search (EWS) is the 2-player game tree search algorithm created by Owen Randall et alia in 2024. Combining features of Monte Carlo Tree Search and Proof Number Search, EWS guides search by considering both expected win rate and expected proof tree size.
In this thesis, we show how to craft a simple Hex solver based on Connection Search and EWS. We start with a minimax solver that prunes moves based on move ordering, inferior move detection, and safe cell-to-cell connection strategies. We measure the importance of these features and add them to a game-agnostic version of EWS. The result is a 1,000-line Python program that solves 6×6 Hex states
Insight into government, November 28, 2025
No full textAlberta's independent newsletter on government & politics
Mechanistic Insights into Protease Function and Inhibition in Infectious Diseases
No full textProteases, also referred to as peptidases, regulate many essential cellular processes, including protein turnover, cellular differentiation and apoptosis. While they play critical roles in maintaining homeostasis in the human proteome, proteases also serve as key determinants of virulence in bacterial and viral pathogens. As a result, protease inhibitors have emerged as crucial therapeutic agents for combating infectious diseases. This thesis explores the mechanism and inhibition of viral and bacterial proteases as a strategy to target pathogen viability and pathogenicity.
For viral infections, coronaviruses remain a persistent global health threat due to their high transmissibility, mild-to-severe symptoms, and zoonotic spillover potential. During the coronavirus lifecycle, the main protease (Mpro) plays a pivotal role in cleaving the viral polyprotein for viral replication. In domestic cats, coronavirus infections can lead to fatal outcomes. Structural and biochemical studies of feline infectious peritonitis virus (FIPV) Mpro, a close homolog of SARS-CoV-2 Mpro, were conducted to characterize the mechanism of inhibition by GC376, a pre-clinical antiviral candidate. High-resolution structural analysis revealed key interactions responsible for inhibition, addressing gaps in prior studies. Additionally, derivative compounds of GC376 demonstrated enhanced potency in vitro, though further characterization of their off-target effects is required before advancing to in vivo studies.
For bacterial infections, the emergence of antimicrobial resistance (AMR) has continued to diminish the efficacy of traditional antibiotics, necessitating the identification of novel therapeutic targets. The rhomboid protease GlpG in Escherichia coli was investigated for its role in bacterial colonization and virulence. Using a proteomic approach accompanied with functional assays, this work identifies FimA, the structural subunit of type 1 pili, as a substrate of GlpG. Loss of GlpG function led to FimA aggregation in the periplasm, impairing pilus formation and reducing E. coli adhesion and biofilm formation—key steps in bacterial pathogenicity. Furthermore, inhibition of GlpG was found to disrupt pili assembly, highlighting its potential as a therapeutic target to mitigate E. coli infections, including AMR strains.
By elucidating the mechanisms of viral and bacterial protease inhibition, this thesis contributes to the development of targeted therapeutics against infectious diseases. The findings underscore the significance of proteases in pathogen survival and highlight their potential as viable drug targets for both viral and bacterial infections
Toward Efficient Reinforcement Learning Under Non-Stationarity
No full textNon-stationarity in reinforcement learning (RL) arises naturally from value bootstrapping, policy iterations, and the resulting shifts in data distributions.
These shifts lead to catastrophic forgetting, unstable training, and inefficient learning, which are further exacerbated by limited resources. This thesis addresses some of these challenges by developing novel algorithmic and architectural methods that reduce catastrophic forgetting, enhance learning efficiency, and improve training robustness under non-stationarity.
First, we propose memory-efficient RL algorithms that reduce reliance on large replay buffers by consolidating knowledge to mitigate forgetting, achieving strong performance within constrained memory budgets.
Second, we investigate the role of neural network architecture in reducing forgetting and introduce elephant activation functions, which induce both sparse activations and gradients. This architectural modification significantly enhances resistance to forgetting and improves learning performance.
Third, we explore the potential of learning optimizers tailored for RL, addressing challenges such as non-independent and identically distributed update gradients with high bias and variance. We present an effective meta-training framework that enables learned optimizers to generalize across diverse tasks.
Finally, we present a novel method to exploit reward functions more effectively by incorporating reward gradients into policy gradient methods, without explicit dynamics modeling.
This approach improves sample efficiency and performance, complementing the broader goal of efficient RL.
Collectively, these contributions deepen the understanding of RL under non-stationary and resource-constrained conditions, paving the way toward more adaptable, efficient, and practical RL algorithms
Peropyrene: Synthesis, Derivatization, and Analysis of Optical Properties
No full textThe synthesis of polycyclic aromatic hydrocarbons (PAHs) is of great interest in a variety of academic and industrial fields. One in particular is the field of organic electronics, which aims to take advantage of the optical and electronic properties of PAHs to replace the inorganic materials currently in use in common electronic devices. The use of bottom-up synthetic techniques allows for precise control of the shape, size, and chemical/physical properties of target molecules, and this is a method used by the Chalifoux group to synthesize a wide variety of PAHs. This thesis focuses on the synthesis and late-stage functionalization of peropyrene, using both well-established and novel synthetic methods. Peropyrene was synthesized using several key reactions, including Sonogashira and Suzuki cross-couplings, as well as InCl3-catalyzed alkyne benzannulation. The method for functionalizing the 9 position of peropyrene using iridium-catalyzed C–H activation was optimized, and the borylated product was transformed into several other highly versatile functional groups. An iodo derivative was synthesized from the borylated product in one step, and this iodo derivative was used for a small scope of Sonogashira cross-couplings with different acetylene derivatives. To see if extending the conjugation of peropyrene would affect its optical properties, solution and solid-state UV-Vis and emission data was collected. Finally, we hypothesized that a meta-coupled peropyrene dimer might have some applications as a singlet-fission material. This dimer was synthesized, however solubility issues with the final product made full characterization and analysis difficult
Learning Survival Prediction Models that Estimate Individual Treatment Effect from Observational Data
No full textA fundamental goal of causal inference is to determine whether, and to what extent, a change in one variable influences another. In precision medicine, this translates to estimating individual treatment effects (ITE) — how a patient’s expected outcome would differ under alternative treatments — to answer counterfactual questions such as: Would a patient with coronary artery disease (CAD) have survived longer if they had undergone coronary artery bypass grafting (CABG) instead of percutaneous coronary intervention (PCI)? Addressing these questions is critical for tailoring personalized treatment recommendations rather than relying on average population effects. However, this task is fundamentally challenging because the training data contain outcomes only for the treatment each patient actually received, but not under the alternatives, forcing reliance on estimating potential outcomes. Additionally, the data often originate from observational studies exhibiting selection bias and censoring, which must be properly accounted for to ensure reliable predictions.
Here, we investigate whether survival models trained on a real-world dataset of CAD patients treated with either PCI or CABG, can identify the best treatment for each specific patient. Furthermore, we developed a novel method that effectively improves individual treatment effect estimation from observational data in the presence of censoring, while accounting for selection bias, thus enabling reliable personalized treatment recommendations based on predicted survival benefits.
The first part of this thesis explores the application of survival analysis techniques to the CAD dataset, benchmarking several models for predicting individual survival distributions (ISDs) and evaluating their accuracy. Additionally, we investigated whether early predictions using only pre-treatment data at the time of diagnosis could match the accuracy of post-treatment predictions incorporating procedural information. The results of this study suggest that treatment planning and risk stratification could begin at the time of diagnosis, rather than waiting for procedural details to become available. We also performed statistical analyses comparing predicted ISDs of different treatments for each patient, as well as summary scores obtained from the ISDs, to see if the observed difference in the potential outcomes are statistically significant, a requirement for reliable personalized treatment recommendation.
In the second part, we propose a novel method that integrates counterfactual representation learning with survival modeling to estimate ITEs from censored observational data. This method extends Counterfactual Regression Networks (CFRNet) by incorporating a survival model, Multi-Task Logistic Regression (MTLR), to align treated and control groups within a shared representation. This alignment transforms the observational data to resemble a randomized controlled trial (RCT), consequently mitigating selection bias while accounting for censoring. Since real-world observational data lacks counterfactual outcomes, we evaluate our method on synthetic and semi-synthetic datasets using both survival prediction and causal inference metrics. Our results demonstrate that our approach outperforms standard (Neural) MTLR in factual prediction and significantly improved counterfactual estimates. Moreover, our approach achieves better counterfactual predictions than BITES — a Cox Proportional Hazards–based competitive baseline. These results highlight the potential of integrating causal inference with survival analysis for more reliable personalized treatment recommendations
