DSpace@RPI (Rensselaer Polytechnic Institute)
Not a member yet
6809 research outputs found
Sort by
Reflections From Former EICs: 40 Years of IEEE Intelligent Systems.
No full textThe year 2025 marks the 40th anniversary of IEEE Intelligent Systems, a significant milestone in the magazine’s remarkable evolution. With this issue, we celebrate four decades of the magazine’s insightful contributions to the artificial intelligence (AI) community. Past issues, articles, and summaries are conveniently aggregated at the IEEE Computer Society Digital Library and IEEE Xplore. This anniversary offers a moment for reflection, recollection, and celebration. We invited five former editors-in-chief (EICs)—Daniel Edmund O’Leary, James Hendler, Daniel Zeng, V.S. Subrahmanian, and San Murugesan—to reflect on the magazine’s history, accomplishments, and future and share their memories. Here are their reflections
A knowledge augmented probabilistic framework for 3d human reconstruction from monocular cameras
No full textDecember 2024School of EngineeringMonocular 3D human reconstruction aims to recover human body and hand configurations from images captured by a single RGB camera. It is an important topic in computer vision with broad applications in human behavior analysis, robotics, and many other fields. Recently, deep learning has enabled promising 3D human reconstruction performance, even in real-time. Nonetheless, existing methods suffer from several limitations. First, the deep 3D human reconstruction models are data-driven; their performance highly depends on the quality and quantity of training data. Second, being deterministic, they typically cannot capture uncertainties in input data and learned models, and hence cannot accurately quantify their reconstruction accuracy. Finally, most approaches perform frame-based static reconstruction, ignoring underlying human dynamics. To address these limitations, in this thesis, we develop a knowledge-driven probabilistic approach for data-efficient, generalizable, and robust 3D human reconstruction from monocular cameras. To improve data efficiency and generalization performance, we propose integrating prior knowledge into model training. Instead of learning prior knowledge from Motion Capture (MoCap) data, we systematically exploit different types of well-established generic knowledge that govern the behaviors and properties of human bodies and hands. Specifically, we exploit spatial body and hand knowledge, including anatomy, biomechanics, and physics. We effectively incorporate this knowledge through differentiable training loss functions, allowing for 3D human reconstruction without any 3D annotations. Moreover, to effectively account for uncertainties in both input data and trained models, we introduce novel probabilistic frameworks. We study the impacts of uncertainties on reconstruction accuracy and further leverage the captured uncertainties to improve 3D reconstruction accuracy and robustness. Finally, we exploit temporal physics knowledge to further improve 3D human reconstruction from monocular videos. For 3D body and motion recovery, we leverage theoretical dynamics and develop a new physics-based reconstruction model that produces not only body reconstruction but also physically plausible body motion and joint force estimates. For hand motion reconstruction, we introduce a novel hand motion refinement framework based on combining a diffusion model with intuitive hand dynamics. This framework produces physically plausible hand motion and is robust to degraded image observations.Ph
Variable selection and manipulation with missing data
No full textDecember 2024School of EngineeringBoth causal modeling and associative feature selection aim to identify essential relationships among the set of modeled variables, albeit with different objectives. While causal modeling focuses on functional relationships, associative feature selection focuses on statistical relationships. In practice, inferring causal or associational relations often has to be done from a dataset with missing entries under the potential bias missing data introduces. Traditionally, missingness has been attributed to benign data collection processes, but as datasets are increasingly curated from diverse sources, including untrusted parties, maliciously engineered missingness has become a likely threat. In turn, to make reliable inferences, a practitioner has to understand how the methods used to extract these causal or associational relationships are affected by benign and adversarial missingness. This dissertation addresses these challenges in three parts. First, we examine the impact of benign missing data on the model-X knockoffs framework, a recent method that provides false discovery rate (FDR) control across a broad range of feature selection techniques. We identify how the distribution shift resulting from imputing the missing entries or dropping partially observed data points interferes with the model-X knockoffs’ FDR guarantees. Next, we introduce sufficient conditions under which imputation using the generative model originally intended for FDR calibration can preserve all assumptions of the model-X framework. Second, we study the effects of adversarial missing data on causal structural learning from observational data. We introduce the adversarial missingness treat model, where an attacker selectively omits data entries. Under this threat model, we show an adversary can asymptotically render a corrupted causal model an optimal solution by concealing a subset of the features in certain observations. We also propose learning-based attacks that are effective with finite data and show that they can successfully obscure adversarially targeted causal relationships in various experimental setups. Third, we extend our study of adversarial missingness to associative learning tasks through a bi-level optimization approach. To tailor attacks to standard missing data handling methods, we develop differentiable approximations for three widely used techniques: mean imputation, regression-based imputation, and complete-case analysis. Our results demonstrate that these attacks can effectively manipulate generalized linear models, altering p-values from significant to insignificant by omitting less than 20% of targeted features.Ph
Synthesis and properties of cation exchange membrane with different fixed group and counter ions
No full textAugust2025School of ScienceWith increasing emphasis on low-carbon economies and sustainable energy systems, electrochemical energy technologies—such as fuel cells, water electrolyzers, and redox flow batteries—have garnered significant attention for their roles in clean energy conversion and storage. A key component in these technologies is ion exchange membrane (IEM), including cation and anion exchange membranes, which serves a vital function in regulating ion transport and maintaining system efficiency.Structurally, a typical IEM comprises a polymeric backbone, ionically charged tethered groups, and associated counterions. These components collectively govern the membrane’s physicochemical properties, including ionic conductivity, mechanical stability, and ion selectivity—parameters that are highly dependent on the specific application and operating environment. Despite extensive research efforts, most studies on cation exchange membranes (CEMs) have focused predominantly on sulfonic acid functional groups, owing to their strong acidic nature and high ionic conductivity in hydrated conditions. In contrast, alternative fixed groups such as phosphonic and carboxylic acids have received comparatively limited attentionIn this work, a series of biphenyl-based polymers bearing distinct fixed ionic groups—sulfonic acid, phosphonic acid, and carboxylic acid—were synthesized via post-functionalization strategies. Membranes were fabricated by solvent casting and subsequently characterized for their mechanical properties using dynamic mechanical analysis. Despite sharing an identical polymer backbone, the polymer membranes exhibited significantly different mechanical responses, underscoring the influence of fixed group chemistry on the polymer network. Additionally, ionic diffusion coefficients and permeabilities were determined for various cations (H⁺, Li⁺, Na⁺, and K⁺). The results indicate that ionic transport properties are primarily governed by (1) the hydration radius of the counter-ions and (2) the binding energy between the mobile ions and fixed groups.In addition, a series of biphenyl-phosphonate/sulfonate blend membranes were fabricated to further explore their potential as high temperature PEMs. These blend membranes exhibited promising proton conductivity under both low and high relative humidity conditions. Notably, the membrane containing 10 wt% biphenyl-phosphonate and 90 wt% biphenyl-sulfonate demonstrated the highest conductivity under low humidity, surpassing that of both the commercial Nafion membrane and the pure biphenyl-sulfonate membrane. Furthermore, the blend membrane showed outstanding mechanical properties, suggesting its suitability for high-temperature fuel cell applications.Ph
Image-driven fact-checking of ai generated chest radiology reports
No full textAugust2025School of EngineeringWith the developments in radiology artificial intelligence (AI), many researchers have turnedto the problem of automated reporting of imaging studies. The goal of such work is to
produce a preliminary read of imaging studies in locations such as emergency rooms where
a radiologist may not be readily available, or to present a preliminary structured report
to radiologists to reduce their dictation workload. An automatically produced structured
report could also be more consistent and easier to read, leading to improved accuracy and
lower overall costs of radiology reads in clinical workflows.
Among the imaging areas where this has been found most useful are chest X-rays,
which are the most common imaging modality read by radiologists in hospitals and tele-
radiology practices today. With the recent rise of generative AI, a number of researchers and
corporations are attempting to generate preliminary reports for chest X-ray images thanks
to the availability of relatively large datasets such as MIMIC and CheXpert that come with
their companion reports for training large vision-language models (VLMs). These newly
emerged VLMs can generate longer and more natural sentences when prompted with good
radiology-specific linguistic cues. However, despite the powerful language generation capabil-
ities, ensuring there are no hallucinations, incorrect mentions of findings or their descriptions,
has been difficult for these models limiting their clinical applicability. While methods for
hallucination removal and fact-checking exist for large language models, with strategies such
as direct policy optimization (DPO) or proximal policy optimization (PPO), and reward
models, they are mostly applicable during training or fine-tuning of the models. On the
other hand, methods that check facts during inference time often consult external general
knowledge or detect errors through analysis of produced text either by themselves or through
an LLM serving as a judge. In radiology report generation, however, neither is possible since
the report has to be specific to the patient and consistent with the evidence seen in the
imaging. Since the automated reporting LLMs themselves have hallucinations, there are no
teacher LLMs that are good enough to correct automatically generated radiology reports.
Further, they may not be able to corroborate their deductions with the patient-specific im-
age. Finally, any fact checking should be agnostic to the radiology report generation tool
to give versatility of use during clinical deployment where different choices of vendors may be prevalent with separate evolving capabilities over time. Thus, there is a need to develop
an independent fact-checking method for use during clinical inference to bootstrap radiology
report generation and increase their adoption in clinical workflows.
This Master’s thesis investigates a hypothesis that it is possible to develop such inde-
pendent discriminative neural networks as fact-checking models for use during inference to
detect and correct errors in automatically generated reports. The key idea explored in the
thesis is that by creating a synthetic dataset of real and fake findings derived from ground
truth reports and pairing them with the corresponding chest X-ray images, a fact-checking
classifier could be trained to distinguish between real/correct description of findings and
incorrect description of findings when they are paired with the corresponding images. Such
an independently developed classifier can then be used to detect and correct errors in the
reports generated by automated radiology reporting tools.
To proceed with the verification of the hypothesis, the thesis is divided into 4 investi-
gations. First, by examining several radiology reporting methods, we analyze the types of
errors made by the report generators to conclude four major error types such as irrelevant
predictions, polarity reversal or omissions, incorrect location predictions and other types such
as incorrect severity assessments. We then simulate the errors to create a large synthetic
dataset by perturbing findings and their locations in ground truth reports reflecting real and
fake findings-location pairs with images. We then proceed to build a discriminative classifier
to detect the errors and remove the finding errors in reports using two different methods,
one that is based on the findings alone and the other that captures their spatial locations.
Finally, we develop methods to correct the automated report while still ensuring language
correctness by careful prompting of a large language model using information derived from
the fact checking model.
Throughout, we conduct experiments with multiple benchmark datasets and conduct
ablation experiments to select relevant architectural configurations and document the overall
improvement in the quality of the report by the use of our fact-checking model to detect
and correct errors. A novel measure was developed for assessing the report correctness
leveraging both clinical accuracy and phrase grounding accuracy. Explainable visualizations
were generated to show the deviation of the reported findings from predicted findings and
their locations generated by the fact-checking model.
The overall results indicated that it was possible to develop a fact-checking model using an independently collected dataset of real and fake findings to simulate the errors made by
report generators. The resulting fact-checking model was over 90% accurate as tested on
multiple benchmark datasets and led to improvement in the quality of the automatically
generated reports in the range of 7-29%. A high degree of concordance was found between
the use of our fact-checking model and ground truth for verification of automated reports
leading us to also conclude that the fact-checking model has the potential to serve as a
surrogate ground truth during clinical inference. This proves further utility of our model
as an additional validation checkpoint in making AI models robust and ready for clinical workflows.M
Multi-scale simulation and model order reduction for the radiative transfer equation
No full textAugust2025School of ScienceThe radiative transfer equation (RTE) models the propagation of radiation through a medium; it has applications in areas such as astrophysics, remote sensing, optical tomography, and radiative transfer. In applications such as uncertainty quantification and design optimization, the parametric RTE arises. Two computational challenges are considered here in solving the RTE. First, the RTE is multi-scale in nature due to the varying magnitude of the material properties throughout the spatial domain. Second, the unknown function in the RTE is defined in a high-dimensional phase space; simulating the RTE often requires one to solve a large algebraic system. This computational cost is further amplified when the parametric RTE is solved at many parameter values. In this thesis, we develop numerical schemes to solve the RTE and overcome these challenges. To address the multi-scale nature of the RTE, we consider a dimensionless form of the equation under a diffusive scaling. In the scattering dominated regime, the equation becomes stiff; this poses an additional challenge in designing appropriate time discretizations. In the first part of this thesis, we develop high-order asymptotic preserving (AP) schemes that are based on the micro-macro decomposition of the model and use implicit-explicit backward differentiation formula (IMEX-BDF) methods in time combined with discontinuous Galerkin (DG) methods in space and the discrete ordinates method in velocity/angle. AP schemes are advantageous for solving multi-scale equations as they correctly capture the behavior of the equations under different scales. In particular, our schemes follow from previous works that develop AP schemes with implict-explicit Runge-Kutta (IMEX-RK) methods in time combined with our spatial and velocity discretizations. Implicit-explicit (IMEX) methods are designed to solve ordinary differential equations (ODEs) containing stiff and non-stiff terms. IMEX-RK methods are a particular class of IMEX methods that use Runge-Kutta (RK) methods as the time integrator. In RK, the right-hand side of the ODE must be evaluated multiple times over one time step. In contrast, linear multistep methods such as backward differentiation formula (BDF) methods only need to evaluate the right-hand side once over one time step. This motivates us to develop AP schemes that use IMEX-BDF methods in time and systematically investigate the time discretization. We form three families of schemes that each use a particular choice of explicit and implicit terms for the time discretization and investigate their stability, accuracy, and AP property. The schemes are demonstrated to be stable and high-order accurate for the RTE under a broad range of scales. In particular, our proposed schemes compute high-resolution solutions faster than IMEX-RK schemes when the formal orders of accuracy are second or third order. In the second part of this thesis, we design reduced order models (ROMs) based on reduced basis methods (RBM) to efficiently solve the parametric steady-state RTE. Reduced order modeling seeks to build a surrogate model that computes accurate solutions to (non-)parametric partial differential equations (PDEs) at a much lower cost than more expensive full order models (FOMs). As a projection-based ROM, RBM involves the construction of a (low-dimensional) reduced basis space to represent the solution manifold over the parameter space of the PDE. A greedy procedure is used to iteratively build the reduced basis space. Because this procedure only requires the computation of the minimum number of FOM solutions, RBM has an advantage over other projection-based ROMs such as proper orthogonal decomposition (POD) that potentially require the computation of a large number of FOM solutions. Despite this, there has been limited work in developing RBM-based ROMs to solve the parametric RTE. Our work is the first to conduct a thorough formulation, investigation, and analysis of RBM-based ROMs to solve the parametric steady-state RTE. We develop four ROMs that are derived from the choice of one of two reduced basis projections combined with one of two possible error indicators. Implementation strategies are carefully designed to enhance the efficiency and robustness of our ROMs. The ROMs compute accurate solutions to the parametric RTE at a large collection of parameter values more efficiently than the FOM.Ph
"Apples and Oranges" - Evaluating Reaction Time Measures as a Paradigm to Contrast Expert vs. Novice Performance in Complex, Dynamic Task Environments
No full textPrevious research has effectively employed the fast-paced action puzzle video-game Tetris for understanding the acquisition of extreme expertise in complex, dynamic environments. A common approach when contrasting expert to novice performance has been the dissection of their interactions with the environment into disjoint sub-tasks – such as reaction time (RT), measured by the input latency to new events on screen. The crucial, underlying assumption to this paradigm is task consistency at all levels of expertise. Using data collected from participants of the Tetris World Championship 2019 and from novices in our lab, we show that this assumption does not hold. While for novices the RT task type remains the same across all conditions, for experts - depending on environmental parameters - RT task type undergoes a shift and under specific conditions does not represent an RT task anymore. Thus, expert vs. novice sub-task comparison may not be a valid paradigm
Interpretable transfer learning: understanding and controlling knowledge transfer
No full textMay2025School of ScienceTransfer learning involves leveraging the knowledge gained while solving one problem and applying it to a different but related problem, thus facilitating the adaptation of learned patterns and representations. This approach is particularly beneficial when labeled data is scarce or training resources are limited. Over the past decade, transfer learning has emerged as a critical technique in the field of machine learning, revolutionizing how models are trained and deployed across various domains. Approaches such as fine-tuning pretrained models, representation transfer, and domain adaptation have enabled models to leverage knowledge learned from large-scale datasets and transfer it to new, related tasks with limited labeled data. However, the interpretability of the transferred knowledge remains a challenge in transfer learning. While pretrained models often achieve impressive performance gains, understanding how and why these models make specific predictions is often non-trivial. This thesis seeks to further our understanding of transfer learning by investigating the knowledge transferred between source and target domains. Previous research on interpretable transfer learning has focused on empirical evaluations of network architectures that lead to better transfer, as opposed to understanding what knowledge enables positive versus negative transfer of knowledge. Furthermore, transfer learning has predominantly functioned as a tool for enhancing performance in target domains, overlooking the potential harm of propagating undesirable knowledge encoded in source models to downstream tasks. To this end, we address three research questions surrounding interpretable transfer learning: Can we interpret what, where, and how the knowledge is transferred from a source domain to a target domain? Can we mitigate the transfer of undesirable knowledge to downstream tasks? Can we automatically identify and transfer common concepts or attributes that are helpful to the target task? For the first research question, we designed and implemented Auto-Transfer (AT), a framework that automatically learns to route source representations to appropriate target representations, following which they are combined in meaningful ways to produce accurate target models. We demonstrated upwards of 5% accuracy improvements compared with the state-of-the-art knowledge transfer methods on several benchmark datasets. We qualitatively analyze the goodness of our transfer scheme by showing individual examples of the essential features using visual explanation methods. We also observed that our improvement over other methods is higher for smaller target datasets, making it an effective tool for small data applications that may benefit from transfer learning. For the second research question, we proposed a novel approach for suppressing the transfer of user-determined semantic concepts (viz. color, glasses, etc.) in intermediate source representations to target tasks without retraining the source model, which can otherwise be expensive or even infeasible. Notably, we tackled a bigger challenge in the input data as a given intermediate source representation is biased towards the source task, thus further entangling the desired concepts. We evaluated our approach both qualitatively and quantitatively in the visual domain and demonstrated that our approach successfully suppresses user-determined concepts without altering other concepts. Lastly, we explored the automatic identification of beneficial concepts for the target task, using examples from the biomedical domain. We introduced Conceptual Counterfactual Explanation (CoCoX), a method that integrates conceptual and counterfactual explanations to pinpoint the most relevant medical concepts for a black-box chest X-ray classifier. Furthermore, we enhanced the joint embedding space of biomedical foundation models with textual concepts, achieving performance improvements of over 5\% across various downstream tasks from diverse biomedical domains. Overall, through this thesis, we developed methods to support the interpretability of knowledge transferred between source and target domains, mitigate the transfer of undesirable knowledge, and improve performance on resource-constrained tasks. As the field of transfer learning continues to evolve, achieving a balance between performance and interpretability remains a crucial area of focus for advancing the robustness and reliability of machine learning models across diverse real-world application domains.Ph
On the use of fluorescence resonance energy transfer in amyloid fibril detection and protein conformational studies
No full textMay2025School of ScienceThe phenomenon known as fluorescence resonance energy transfer (FRET) has become an extremely powerful and versatile tool that is widely-used in biomolecular sciences to report on various interactions, conformational changes, and much more. This work is comprised of two independent projects that are connected through the use of FRET. First, we describe the development of a FRET-based universal amyloid detection platform harnessing the intrinsic affinity of fluorescent proteins (FPs) for the general amyloid core structure. Amyloid fibrils are a type of protein aggregate characterized by a common core structure and extreme stability. These aggregates are widely known for their pathogenic roles, as amyloid deposition is associated with numerous human diseases, but also serve many important functional roles across all domains of life. Being able to detect the presence of amyloid is central to studying their roles in both normal biological processes they are involved in, as well as the disease states they are found in. Proof of concept for our quantitative amyloid sensor platform is demonstrated using a model fibril system, PAPf39, and the subsequent application in detecting other amyloid systems is assessed. We observe differences in the sensitivity of the FRET sensor towards these different fibrils and attempt to elucidate the underlying causes. The second portion of this work describes our investigation into the effects of pressure and temperature on proteins lacking tertiary structure. The study of biological macromolecules under pressure is guided by the notion that the large majority of the Earth’s biomass exists under high hydrostatic pressure. However, high pressure is known to disrupt biological processes of organisms living at ambient pressure, and also has significant effects on biomolecular stability and interactions. These points together highlight the lack of understanding we have of the ways in which piezophilic organisms are adapted to their native environments. The study of high pressure effects on isolated proteins has widely shown that pressure denatures globular proteins primarily through the release of void spaces present in the folded structure in order to achieve a smaller volume. However, there has not been significant study of the effects of pressure on secondary structural elements at the core of these tertiary structures. Additionally, it is now well-known that intrinsically disordered proteins (IDPs) perform important biological functions, despite their lack of stable structure, and, yet, the effects of pressure on their conformational ensembles have been largely unstudied. We use FRET, among other biophysical methods, to analyze the combined pressure and temperature dependence of α-helical structure, as well as the dimensions of coiled and intrinsically disordered proteins.Ph
Investigating cellular dysfunction in injury-induced neurological and musculoskeletal disorders
No full textMay2025School of EngineeringThe incidence and severity of musculoskeletal and neurological disorders – which has been shown to be interconnected through an abundance of clinical studies – increases with age and injury. With this, regenerative medicine approaches must first understand the mechanisms behind which the pathological states occur. This brings a specific focus on spinal cord injury (SCI), where its aftermath is associated with increased risks of developing Alzheimer’s disease (AD) and osteoporosis. The interplay between the three becomes clearer when viewed in the context of the commonalities found in cellular dysfunction, specifically through mass cell deaths, chronic inflammation, neural cell over-reactivity, and altered regeneration capabilities. However, the specific interactions involving SCI, neurological dysfunction, and bone health decline remains unclear. Thus, the studies presented herein will focus on clarifying the dysregulated cellular response in pathology and regenerative approach through (1) investigating the effects of an elevated pressure condition after impact in the context of SCI and increased intraspinal pressure (ISP), (2) demonstrate the direct regulating effect of sympathetic dysfunction in altered osteogenic capabilities, and (3) assess the applicability of shape memory polymer scaffolds in bone regeneration.Ph