21090 research outputs found
Sort by
Learning to Reach Goals from Suboptimal Demonstrations via World Models
No full textA central challenge for training autonomous agents is the scarcity of high-quality and long-horizon demonstrations. Unlike fields such as natural language or computer vision—where abundant internet data exists—many robotics and decision-making domains lack large, diverse, and high-quality datasets. One underutilized resource is leveraging suboptimal demonstrations, which are easier to collect and potentially more abundant.
This limitation is particularly pronounced in goal-conditioned reinforcement learning (GCRL), where agents must learn to reach diverse goal states from limited demonstrations. While methods such as contrastive reinforcement learning (CRL) show promising scaling behavior when given access to abundant and high-quality training demonstrations, they struggle when demonstrations are suboptimal. In particular, when training demonstrations are short or exploratory, CRL struggles to generalize beyond the training demonstrations, and the resulting policy exhibits lower success rates.
To overcome this, we explore the use of self-supervised representation learning to extract general-purpose representations from demonstrations. The intuition is that if an agent can first learn robust representations of environment dynamics—without relying on demonstration optimality—it can then use these representations to guide reinforcement learning more effectively. Such representations can serve as a bridge between noisy demonstrations and goal-directed control, allowing policies to learn faster.
In this thesis, we propose World Model Contrastive Reinforcement Learning (WM-CRL), which augments CRL with representations from a world model (WM). The world model is trained to anticipate future state embeddings from past state–action pairs, thereby encoding the dynamics of the environment. As the world model aims to only learn environment dynamics, it can leverage both high and low quality demonstrations. By integrating these world model embeddings into CRL’s framework, it can help CRL more easily comprehend the environment dynamics and select actions that more effectively achieve its goals.
We evaluate WM-CRL on tasks from the OGBench benchmark. We explore performance on multiple locomotion and manipulation environments and multiple datasets varying in quality.
Our results show that WM-CRL can substantially improve performance over CRL in suboptimal-data settings, such as stitching short trajectories or learning from exploratory behavior. However, we find our method offers limited benefit when abundant expert demonstrations are available. Ablation studies further reveal that success depends critically on the stability of world model training and on how its embeddings are integrated into the agent’s architecture
Characterization and Comparison of Flammability Properties and Trace Emissions of Select Native and Invasive Canadian Wildland Fire Fuels
No full textFire has played an integral role in the evolution, formation, and sustainability of North American forest ecosystems. Historically, Indigenous peoples have employed fire as a deliberate land management tool to maintain forest health, shape landscapes, and achieve early industrial objectives. With the landing of European settlers, and changes in governmental policy, the use of fire as a land/fuel management tool was greatly diminished. In addition to the suppression of fire as a tool, the intentional and accidental introduction of non-native plant species to Canadian forest ecosystems has dramatically altered its structure from the 17th century through to today. In addition, emissions of CO₂ and other greenhouse gases have been rapidly increasing since industrialization, which has warmed the planet, resulting in extreme weather events like droughts and storms that occur at increasing frequency and severity. This has culminated in wildfire conditions that are drastically different to those that shaped the historical evolution of Canadian forests. Key changes in forest fuels include larger spatial distributions of fuel types and moisture content, which affect fire growth and development. Over the past few decades it has become evident that understanding these factors of fuel types, moisture content, and fire growth and development are critical to improve performance of predictive models, as well as our overall understanding of how to combat and minimize the damage caused by these severe wildfire events. Assessment of wildfires has generally taken two approaches: the first being a largescale analysis of a real wildfire event, which characterizes total emissions and bulk burning behaviour, and the second being small-scale studies that often focus only on one specific fire performance metric or a limited set of emissions. While both approaches have yielded significant data in terms of bulk fire performance metrics and separate emissions data, this separation has led to a dearth of integrated, detailed, and comparative data. This comparative data is critically important because the lack of species-specific flammability metrics and associated detailed emissions data under varying conditions hinders the accurate prediction of fire behaviour and the development of effective land management strategies. Furthermore, the absence of data explicitly linking exposure conditions to trace emission profiles (the toxic fraction of smoke) leads to misestimates in both emission inventories and air quality models, potentially compromising environmental safety assessments. In this research, a pair of native species and a pair of invasive species are tested at small scale for their flammability properties, major, and trace emissions. Testing was conducted under two different levels of fuel moisture and two different radiation exposures, with twelve replicates per condition. The native species are trembling aspen and ironwood while the invasive species are buckthorn and barberry. All four species were tested under reference conditions (35 kW m⁻² incident heat flux, and naturally dry conditions), buckthorn and trembling aspen were tested under elevated heat flux (50 kW m⁻² incident heat flux, and naturally dry conditions), and barberry and ironwood were tested at elevated moisture conditions (35 kW m⁻² incident heat flux, and field-tested moisture conditions). Across the tests, flammability properties, such as ignition delay time and heat release rate, were compared as well as real-time concentrations of CO₂, CO, and VOCs. In addition to these three gases, thermal desorption tubes were employed to sample the smoke plume at three phases during a test – pyrolysis, open flaming, and smouldering – and were analyzed using GC-MS to identify and group key emissions, then develop a qualitative sensitivity of trace emissions to species and burning conditions. To properly frame the discussion surrounding the production of trace emissions, the lignocellulosic compositions (% cellulose, % hemicellulose, and % lignin) and the apparent activation energy of each of the species was determined using thermogravimetric analysis. Finally, inductively coupled plasma-optical emission spectroscopy and X-ray diffraction were employed to identify and quantify metallic emission differences in the post-burn particulate matter and the fire smoke plume. A broad summary of the results shows that species composition (lignocellulosic makeup) and intrinsic physical characteristics (sample piece sizes and packing geometry) are the dominant factors driving differences in fire performance and flammability under reference conditions. When exposed to a higher heat flux, the external energy largely overcame the impacts of geometry, allowing compositional differences to become the sole dominant factor dictating distinct species responses in peak heat release rate and emissions. The exposure to the increased heat flux also greatly reduced the ignition delay time and increased the heat release rates for both native and invasive species. Conversely, increased fuel moisture content led to a clear and consequential shift toward less efficient, incomplete combustion
processes, resulting in substantial increases ignition delay time, reductions in heat release rate and increases in CO and VOC emission factors during the pyrolysis, flaming, and smouldering phases. Thermogravimetric analysis confirmed a compositional-kinetic relationship where the apparent activation energy varied by up to 24% across species. This kinetic variation, coupled with data from thermal desorption-gas chromatography–mass spectrometry, highlighted the dependent nature of trace species production on the specific species composition and apparent activation energy. Different species produced distinct groups of trace emissions and showed differing responses to both elevated heat flux (where some experienced volatile suppression and others persistent intermediates) and varying moisture conditions (where smouldering emissions were dramatically amplified or altered)
Atomistic Modelling and ReaxFF Parameter Optimization for Ionic Liquid Electrolyte
No full textThe optimization of organic carbonate-based electrolytes, such as ethylene carbonate (EC) and
propylene carbonate (PC), was a pivotal enabler for graphite anode materials in the mid-1990s and
remains at the heart of modern Li-ion battery (LIB) technology. With battery R&D publications
growing 4.5 times faster than general literature between 2010 and 2017 (Li et al., 2018), current
research prioritizes electrode and electrolyte improvements to enhance energy capacity, cycling rates,
and safety. However, future advancements rely heavily on the digitalization of materials science.
Recent industry roadmaps indicate a critical global need for integrating multi-sourced, multi-fidelity
data streams—combining experimental and computational data—to holistically analyze cell
performance and safety (Batteries Europe Secretariat, 2023).
In this framework, this thesis investigates an organic liquid electrolyte with an ionic liquid additive
using atomistic and molecular simulations. Initial molecular topology, equilibration, and thermalization
were established using Generalized Amber Force Field (GAFF) parameters. Subsequently, the Reactive
Force Field (ReaxFF) was employed to simulate the reactive electrolyte environment. To optimize
ReaxFF parameters for this specific system, Plane-wave Density Functional Theory (DFT) electronic
calculations were performed to derive energy baselines. A custom-developed Python library was
created to generate a comprehensive training dataset, comprising bond lengths, 3-body angles, 4-body
dihedrals, partial atomic charges, interatomic forces, and reaction enthalpies. Molecular dynamics
simulations revealed that the ionic liquid additive improves electrolyte properties by
altering the solvation structure and acting as a Li-salt stabilizer. Furthermore, the weak cation-anion ligand
interactions introduced by the additive were found to enhance Li-ion diffusion
Introducing heterogeneity into coupled social-climate models
No full textAnthropogenic climate change is possibly the greatest global problem we face today. However efforts to mitigate climate change are insufficient to keep warming within the safe limit of 1.5 degrees Celsius, despite the increasing availability and affordability of clean energy. Resistance to mitigation arises from social and geopolitical factors. However, the role of social dynamics in mitigation is not well understood as these have been treated as extrinsic to most existing models of climate change. Recently coupled social-climate models have modelled two-way feedback between social systems and climate change and found social parameters to be comparable to geophysical parameters in their influence of climate change outcomes. However, most of these models assume society to be homogeneous or contain stylized representations of heterogeneity. In this thesis, we construct a coupled social-climate model representing five world regions, parameterized with empirical survey and socioeconomic data, and use evolutionary game theory to model the evolution of mitigation support in each region. We find that social learning and norms can cause mitigation support to collapse or spread, and that opinions in one region are affected by opinions in other parts of the world, purely through feedback from the global warming. Social processes influence the peak temperature anomaly by several degrees Celsius. We next look at the roles of social and climate change impact heterogeneity in shaping social-climate outcomes by including location-specific warming and vulnerability to impacts in our model. We find that heterogeneity, on the whole, leads to worse global warming outcomes, with heterogeneity in impacts increasing the peak temperature by 0.2 degrees Celsius. We also identify a bifurcation in social-climate outcomes, and find that a drop in vulnerability to impacts in one region can tip the social-climate system into a slow-mitigation, high-temperature outcome. Finally, we couple our model of social-climate dynamics to a game theoretical representation of government decision-making on climate change mitigation. We find that if governments factor in support for mitigation evolving over time, it could lead to more regions choosing to mitigate at Nash equilibria, depending on mitigation targets considered. Social processes alone cannot create conditions for a coordination game as long as regions are coupled through temperature impacts alone. This thesis presents a framework for representing heterogeneity in coupled social-climate models, identifies states that do not arise in homogenous settings and, therefore, provides insight into ways to overcome social barriers to mitigation
Integrating Symbolic Reasoning into Large Language Models
No full textLarge language models (LLMs) face fundamental challenges in symbolic reasoning, struggling with tasks requiring precise rule-following, logical consistency, and manipulation of structured representations. This thesis introduces a comprehensive neurosymbolic framework that addresses these limitations by integrating Vector Symbolic Algebras (VSAs) directly into the computational flow of transformer-based language models. Our core method encodes LLM hidden states into compositional neurosymbolic vectors, enabling symbolic algorithms to operate within a high-dimensional vector space before decoding results back into the neural network's processing pipeline.
We demonstrate that LLMs naturally develop internally separable representations for symbolic concepts, which our linear and transformer-based encoders can extract with high fidelity. On mathematical reasoning tasks, our approach achieves 88.6\% lower cross-entropy loss and solves 15.4 times more problems correctly compared to chain-of-thought prompting and LoRA fine-tuning, while preserving performance on non-mathematical tasks through selective intervention.
Beyond arithmetic, we extend this framework to three applications. First, we enable language-only models to perform visual question answering by encoding segmented images as queryable VSA representations, achieving 92% accuracy without requiring multimodal architectures. Second, we demonstrate environment navigation where LLMs use spatial semantic pointers to interpret and act upon grid-based worlds according to natural language instructions. Third, we address the context length limitations of LLMs by compressing reasoning histories into VSA representations, maintaining performance on iterative problem-solving tasks while avoiding quadratic scaling costs.
Our results establish VSA-based neurosymbolic integration as a practical approach for augmenting neural language models with symbolic reasoning capabilities, providing both theoretical insights into LLM representations and practical improvements across diverse reasoning tasks. This work contributes to the broader goal of creating AI systems that combine the flexibility of neural networks with the precision and interpretability of symbolic computation. Code and data are available at https://github.com/vdhanraj/Neurosymbolic-LLM
REMind: A Robot Role-Playing Game To Promote Bystander Intervention
No full textPeer bullying is a pervasive social problem, with bystanders' inaction being a critical challenge despite widespread disapproval of bullying. Effective intervention strategies must move beyond explanation-based instruction to facilitate embodied situated learning. This dissertation explores how social robots can serve as mediators for applied drama to foster prosocial bystander intervention in the context of peer bullying. It introduces Robot-Mediated Applied Drama (RMAD): an innovative framework that integrates drama-based pedagogy with social robotics to create safe, reflective, and embodied learning experiences.
Using a Research through Design (RtD) methodology, this work advances through an iterative sequence of design studies that culminate in the development and evaluation of REMind (short for Robots Empowering Minds): a mixed-reality role-playing game where children engage in dramatized bullying scenarios performed by social robots. In REMind, three robots enact a conflict involving a bully, a victim, and a passive bystander. Players are invited to assume control of robotic avatar, reflect on the unfolding narrative, and improvise an intervention by using the robot as a proxy in order to change the story’s outcome. Through this structure, children rehearse bystander intervention strategies within a psychologically safe, yet emotionally engaging environment.
The iterative design process of REMind unfolded across complementary empirical inquiries. A crowdsourced feasibility study first established that observers perceive aggression toward robots as morally wrong, validating the viability of using robots in the intervention. A narrative co-design study with children revealed storytelling patterns such as preferences for emotionally expressive and customizable robot characters. Interviews with teachers grounded the design in classroom realities, identifying gaps in existing programs. A game design focus group study further examined what makes educational robot role-play games pleasurable for children, leading to identifying concrete design elements that informed REMind’s interactive components such as core mechanics, use of tangible props, world aesthetics and narrative structure.
This dissertation presents the resulting artifact, REMind, as a system consisting of five interconnected components: Learning Goals, Mechanics, Narrative, Technology, and Aesthetics. The learning goals were defined through consultation with subject-matter experts to ensure grounding in evidence-based best practices.
By deliberate aligning the game pleasures identified in prior studies with the learning objectives, REMind introduces a suite of game mechanics that scaffold socio-emotional skills (such as robot-mediated spect-actorship or "puppet mode" for moral intervention, interpretation of immersive affective displays for empathy-training and perspective taking, and custom-made logic-gate puzzles for moral reasoning). Narrative design is scaffolded by borrowing a five-step cognitive model of bystander intervention from social psychology. The technical implementation is realized through StorySync, a novel spreadsheet-based scripting toolkit developed to synchronize multimodal cues (including multiple robots, graphical interfaces, ambient lighting, and sound) and manage narrative branching for live interactive robot drama. Finally, the aesthetic elements leverages emotional design, ambient cues, and digital scenography to create an emotionally resonant learning experience. This concrete high-fidelity prototype serves as a proof of concept for RMAD.
This research contributes a theoretical and practical foundation for designing robot-mediated experiential learning systems, offering RMAD as a new direction for social robotics and educational technology. It further illustrates how embodied storytelling and interactive systems design might cultivate reflective, prosocial action in a complex domain of social-emotional learning. More broadly, it advocates for a shift in Human-Robot Interaction (HRI) research toward systems thinking, positioning game design as a powerful systems lens for creating and analyzing holistic user experiences
The salty truth behind gassy ponds: stratification and greenhouse gas emissions in urban stormwater ponds in southern Ontario
No full textUrban stormwater management ponds are established end-of-pipe components of green stormwater management infrastructure in North America. The main aim of this infrastructure is to protect our cities and homes from storm flooding and to clean the water before introducing it to protected ecosystems. While created for stormwater management, stormwater ponds provide key ecosystem services in urban areas, such as storing carbon, preserving biodiversity, and enhancing the connection to nature for the surrounding communities. Regardless of these services, stormwater ponds also accumulate large loads of organic matter, nutrients, and contaminants from their catchment and undergo a range of aerobic to anaerobic microbial processes that create and release significant amounts of greenhouse gases (GHGs) such as methane (CH4), carbon dioxide (CO2), and nitrous oxide (N2O). Pond design and catchment inputs may affect the processes and the GHG exchanges from the ponds. Therefore, this study aims to understand the processes that drive GHG emissions under the different design and contaminant conditions of depth and road salt concentration in four ponds over a study period from June - November and April - May the following year with no sampling over the winter months. The high salt ponds had greater stratification induced by road salt density gradient impact then their low salt counterparts, which was reflected in water specific conductivity and dissolved oxygen (DO), the stratification indexes calculated from the water column density gradients of each sample location, and the redox water quality sampling profiles. The CH4 emissions of the high salt ponds were also higher than their low salt counterparts, with the shallow ponds having higher emissions then the deep ponds, while the CO2 and N2O emissions were not driven by depth or road salt. A redundancy analysis showed that there were six significant predictors for the GHG emissions: specific conductivity, temperature, pH, dissolved CH₄ concentration, DO, and pond depth. CO₂ emissions were associated with deeper, cooler, low-conductivity conditions, whereas CH₄ diffusive emission and N₂O emissions were associated with high-conductivity, low-oxygen waters and ebullitive CH₄ emission aligned with warm, higher pH environments. These findings highlight the complex and dynamic role of stormwater management infrastructure in urban GHG emissions and the importance of understanding the processes that govern them
Advancing Semi-Supervised Domain Adaptive Semantic Segmentation Through Effective Source Integration Strategies
No full textSemantic segmentation is a highly valuable visual recognition task with applications across fields such as medical imaging, remote sensing, and manufacturing. However, training segmentation models is challenging because it requires large-scale, densely labeled data specific to the target. Semi-supervised learning (SSL) addresses this challenge by leveraging unlabeled data alongside limited labeled data, reducing the reliance on fully labeled datasets. Semi-supervised domain adaptation (SSDA) further mitigates this issue by incorporating labeled data from a source domain alongside minimally labeled target data.
While existing SSDA methods often underperform compared to fully supervised approaches, recent SSL methods that utilize foundation models achieve near fully supervised performance. Given the strength of current SSL methods using foundation models, this thesis investigates effective strategies for integrating source-domain data from a different distribution into existing pipelines to improve segmentation performance. First, we explore a simple source transfer mechanism that merges target and source data into a single unified labeled set for SSL pipelines. Our analysis demonstrates the accuracy benefits of this setup while also highlighting some downsides, particularly in terms of training efficiency.
We also examine the use of ensembling SSL and SSDA models to enhance target-domain performance. This ensemble combines a model trained solely on target data with a source-transferred SSDA model. We find that ensembling can improve performance in certain cases but is less effective in others, and training efficiency remains suboptimal due to the need to train two models.
Given the training inefficiencies of simple source transfer and ensembling, we propose a dual-curriculum source integration strategy to address and improve these limitations. This approach consists of two complementary learning strategies: curriculum retrieval, which progressively samples source examples from easy to hard, and curriculum pasting, which increases the diversity of target-labeled data.
Across our experiments, we compare against and outperform state-of-the-art SSL and SSDA methods on a variety of benchmarks, including synthetic-to-real and real-to-real scenarios. Our findings highlight the benefits of effective source data integration into modern SSL pipelines for boosting segmentation performance, opening a new avenue for label-efficient semantic segmentation
NAVIGATING SETTLEMENT AND WELL-BEING: EXPERIENCES OF NEWCOMER WEST AFRICAN IMMIGRANTS IN ONTARIO
No full textBackground:
Canada has become a popular destination for immigrants, with a significant proportion arriving through economic immigration pathways. Despite the growing population, African economic immigrants remain understudied and are often grouped with other immigrant subgroups whose experiences may differ significantly from theirs. Although many arrive with strong educational and professional qualifications, African newcomers frequently encounter structural barriers and cultural transitions that shape their well-being, settlement experiences, and help-seeking behaviours. Understanding these experiences is important in improving culturally centered support.
Objective:
This study examined how settlement stressors, cultural perspectives, and changes in sociocultural realities shape West African economic immigrants’ settlement experiences, understanding of well-being, and their engagement with support services in Ontario.
Methodology:
This study employed a qualitative research design using semi-structured, in-depth interviews with nine West African economic immigrants who had lived in Ontario for between nine months and four years. Data were analyzed sequentially using Braun and Clarke’s six-step thematic analysis. Interpretation was conducted inductively and informed by Ager and Strang’s Integration Framework and Berry’s Acculturative Stress Theory.
Results:
Inductive coding identified six themes that captured the participants’ settlement journeys, identity negotiations, and well-being experiences. These themes were organized into three overarching domains: (1) Starting Over, which described participants’ lives before immigrating, emotional transitions, and early settlement stressors; (2) Identity and Systems Navigation, which highlighted cultural adjustments, racialization, experiences of ‘sudden’ Blackness, employment barriers, and housing challenges; and (3) Resilience and Well-being, which reflected key coping strategies, including faith, community support, and cultural maintenance.
Conclusion:
The findings highlighted that well-being among West African economic immigrants is multidimensional and shaped by the interaction of cultural identity, settlement challenges, racialization, and structural inequities. Overall, greater attention is needed to center the experiences of economic immigrants to develop meaningful and culturally responsive approaches to settlement support and integration
Balancing female basketball players’ career progression with family planning decisions
No full textAbstract
Background: Female professional athletes face unique challenges, including lack of funding for league improvements or player salaries, fewer opportunities in terms of exposure to or abundance of professional leagues, and gender norms that limit their engagement in professional sport careers. These limitations are exacerbated when childbearing during the peak years of their career comes into consideration. The goal of this research is to better understand the impact of family planning and pregnancy on career progression of professional female basketball players.
Research Questions: This thesis examined: What impact, if any, do female basketball players believe pregnancy, giving birth, and parenting may have on their career progression? Specifically, I explored (a) What potential implications on physical performance exist because of pregnancy? (b) What financial changes do athletes anticipate pregnancy, childbirth, and the postpartum period could bring to a career in sport? (c) What supports are necessary to help female athletes balance pregnancy and motherhood with a career in sport?
Methods: This study employs a qualitative research design. Narrative inquiry was used to examine how athletes navigate decisions regarding pregnancy, childbearing, and the career progression. The study population included nine professional athletes who are considering or who have experienced childbearing, and who have or had a basketball career. Participants were recruited through personal social media accounts. Individuals were eligible for this study if they self-identified as a professional basketball player and felt they could speak on pregnancy or motherhood in sport through personal experiences. Semi-structured, individual interviews lasting approximately 45-60 minutes explored participants’ accounts regarding the factors influencing their pregnancy decisions and the effects childbearing may have on their careers. Narrative thematic analysis was used to capture common themes across interviews.
Findings: Three stories were created from a compilation of participants’ accounts at three stages of the decision-making process to have children. Five participants did not have children at the time of interviews, and four participants were mothers of one or more children. First, a professional basketball player before pregnancy and motherhood, a professional basketball player after pregnancy while still competing, and lastly, a retired professional basketball player who waited until after their career was over to have children. These three stories demonstrate multiple stages of this decision and how the participants navigate the decision-making experience. These three stories also highlight four main themes within the analysis. The themes highlighted are financial insecurity and structural constraints that exist within professional women’s basketball, global mobility in sport, the body as a site of uncertainty because of pregnancy, and lastly, the stigma surrounding pregnancy and motherhood in professional sport. Many participants experienced difficulty or conflict in making decisions about pregnancy and childbearing and participants often found balancing motherhood and professional basketball challenging. Lastly, their perspectives on pregnancy and motherhood in sport was largely influenced by their personal situations and experiences and therefore is different for every athlete.
Conclusions: The findings from this study contribute to understanding the unique challenges female athletes might face when making decisions about pregnancy and childbearing. The findings can also be leveraged to advocate for improved support systems and practices in professional sports to ensure female athletes who choose to become pregnant and give birth are supported in maintaining their athletic career. Ultimately, this research highlights the need for further exploration into the intersection of gender, sport, and reproductive choices