675 research outputs found
Latent Diffusion Models for Structural Component Design
No full textRecent advances in generative modeling, namely Diffusion models, have revolutionized generative modeling, enabling high-quality image generation tailored to user needs. This paper proposes a framework for the generative design of structural components. Specifically, we employ a Latent Diffusion model to generate potential designs of a component that can satisfy a set of problem-specific loading conditions. One of the distinct advantages our approach offers over other generative approaches, such as generative adversarial networks (GANs), is that it permits the editing of existing designs. We train our model using a dataset of geometries obtained from structural topology optimization utilizing the SIMP algorithm. Consequently, our framework generates inherently near-optimal designs. Our work presents quantitative results that support the structural performance of the generated designs and the variability in potential candidate designs. Furthermore, we provide evidence of the scalability of our framework by operating over voxel domains with resolutions varying from 323 to 1283. Our framework can be used as a starting point for generating novel near-optimal designs similar to topology-optimized designs.This is a preprint from Herron, Ethan, Jaydeep Rade, Anushrut Jignasu, Baskar Ganapathysubramanian, Aditya Balu, Soumik Sarkar, and Adarsh Krishnamurthy. "Latent Diffusion Models for Structural Component Design." arXiv preprint arXiv:2309.11601 (2023). doi: https://doi.org/10.48550/arXiv.2309.11601. Published as Herron, Ethan, Jaydeep Rade, Anushrut Jignasu, Baskar Ganapathysubramanian, Aditya Balu, Soumik Sarkar, and Adarsh Krishnamurthy. "Latent Diffusion Models for Structural Component Design." Computer-Aided Design 171 (2024): 103707.
doi: https://doi.org/10.1016/j.cad.2024.103707
Physics-aware machine learning surrogates for real-time manufacturing digital twin
No full textIn this Manufacturing Blue Sky idea, we envision a Cyber Adaptive Manufacturing Intelligent System (CyAMIS, pronounced Siamese) that integrates concepts from the emerging area of physics-aware machine learning (ML) to formulate, develop, and deploy a near-real-time Siamese (digital) twin to reliably and efficiently achieve exceptional part quality and desired material properties in additive manufacturing processes. We believe such a real-time digital twin framework to be the future of modern additive manufacturing systems, ultimately leading to Manufacturing 5.0 systems.This is a manuscript of the article published as Balu, Aditya, Soumik Sarkar, Baskar Ganapathysubramanian, and Adarsh Krishnamurthy. "Physics-aware machine learning surrogates for real-time manufacturing digital twin." Manufacturing Letters 34 (2022): 71-74. doi: https://doi.org/10.1016/j.mfglet.2022.08.013
Distributed Online Non-convex Optimization with Composite Regret
No full textRegret has been widely adopted as the metric of choice for evaluating the performance of online optimization algorithms for distributed, multi-agent systems. However, data/model variations associated with agents can significantly impact decisions and requires consensus among agents. Moreover, most existing works have focused on developing approaches for (either strongly or non-strongly) convex losses, and very few results have been obtained regarding regret bounds in distributed online optimization for general non-convex losses. To address these two issues, we propose a novel composite regret with a new network regret-based metric to evaluate distributed online optimization algorithms. We concretely define static and dynamic forms of the composite regret. By leveraging the dynamic form of our composite regret, we develop a consensus-based online normalized gradient (CONGD) approach for pseudo-convex losses, and it provably shows a sublinear behavior relating to a regularity term for the path variation of the optimizer. For general non-convex losses, we first shed light on the regret for the setting of distributed online non-convex learning based on recent advances such that no deterministic algorithm can achieve the sublinear regret. We then develop the distributed online non-convex optimization with composite regret (DINOCO) without access to the gradients, depending on an offline optimization oracle. DINOCO is shown to achieve sublinear regret; to our knowledge, this is the first regret bound for general distributed online non-convex learning.This is a preprint from Jiang, Zhanhong, Aditya Balu, Xian Yeow Lee, Young M. Lee, Chinmay Hegde, and Soumik Sarkar. "Distributed Online Non-convex Optimization with Composite Regret." arXiv preprint arXiv:2209.10105 (2022). doi: https://doi.org/10.48550/arXiv.2209.10105. Published as Jiang, Zhanhong, Aditya Balu, Xian Yeow Lee, Young M. Lee, Chinmay Hegde, and Soumik Sarkar. "Distributed online non-convex optimization with composite regret." In 2022 58th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp. 1-8. IEEE, 2022. doi: https://doi.org/10.1109/Allerton49937.2022.9929356
Decentralized Deep Learning Using Momentum-Accelerated Consensus
No full textWe consider the problem of decentralized deep learning where multiple agents collaborate to learn from a distributed dataset. While there exist several decentralized deep learning approaches, the majority consider a central parameter-server topology for aggregating the model parameters from the agents. However, such a topology may be inapplicable in networked systems such as ad-hoc mobile networks, field robotics, and power network systems where direct communication with the central parameter server may be inefficient. In this context, we propose and analyze a novel decentralized deep learning algorithm where the agents interact over a fixed communication topology (without a central server). Our algorithm is based on the heavy-ball acceleration method used in gradient-based optimization. We propose a novel consensus protocol where each agent shares with its neighbors its model parameters as well as gradient-momentum values during the optimization process. We consider both strongly convex and non-convex objective functions and theoretically analyze our algorithm's performance. We present several empirical comparisons with competing decentralized learning methods to demonstrate the efficacy of our approach under different communication topologies.This is a preprint from Balu, Aditya, Zhanhong Jiang, Sin Yong Tan, Chinmay Hedge, Young M. Lee, and Soumik Sarkar. "Decentralized Deep Learning using Momentum-Accelerated Consensus." arXiv preprint arXiv:2010.11166 (2020). doi: https://doi.org/10.48550/arXiv.2010.11166. Published as Balu, Aditya, Zhanhong Jiang, Sin Yong Tan, Chinmay Hedge, Young M. Lee, and Soumik Sarkar. "Decentralized deep learning using momentum-accelerated consensus." In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3675-3679. IEEE, 2021. doi: https://doi.org/10.1109/ICASSP39728.2021.9414564
NeuFENet: Neural Finite Element Solutions with Theoretical Bounds for Parametric PDEs
No full textWe consider a mesh-based approach for training a neural network to produce field predictions of solutions to parametric partial differential equations (PDEs). This approach contrasts current approaches for "neural PDE solvers" that employ collocation-based methods to make point-wise predictions of solutions to PDEs. This approach has the advantage of naturally enforcing different boundary conditions as well as ease of invoking well-developed PDE theory -- including analysis of numerical stability and convergence -- to obtain capacity bounds for our proposed neural networks in discretized domains. We explore our mesh-based strategy, called NeuFENet, using a weighted Galerkin loss function based on the Finite Element Method (FEM) on a parametric elliptic PDE. The weighted Galerkin loss (FEM loss) is similar to an energy functional that produces improved solutions, satisfies a priori mesh convergence, and can model Dirichlet and Neumann boundary conditions. We prove theoretically, and illustrate with experiments, convergence results analogous to mesh convergence analysis deployed in finite element solutions to PDEs. These results suggest that a mesh-based neural network approach serves as a promising approach for solving parametric PDEs with theoretical bounds.This is a pre-print of the article Khara, Biswajit, Aditya Balu, Ameya Joshi, Soumik Sarkar, Chinmay Hegde, Adarsh Krishnamurthy, and Baskar Ganapathysubramanian. "NeuFENet: Neural Finite Element Solutions with Theoretical Bounds for Parametric PDEs." arXiv preprint arXiv:2110.01601 (2021). Copyright 2021 The Authors. Posted with permission.
Published as Khara, Biswajit, Aditya Balu, Ameya Joshi, Soumik Sarkar, Chinmay Hegde, Adarsh Krishnamurthy, and Baskar Ganapathysubramanian. "Neufenet: Neural finite element solutions with theoretical bounds for parametric pdes." Engineering with Computers (2024): 1-23. doi: https://doi.org/10.1007/s00366-024-01955-7
THB‑Diff: a GPU‑accelerated diferentiable programming framework for THB‑splines
No full textWe have developed a differentiable programming framework for truncated hierarchical B-splines (THB-splines), which can be used for several applications in geometry modeling, such as surface fitting and deformable image registration, and can be easily integrated with geometric deep learning frameworks. Differentiable programming is a novel paradigm that enables an algorithm to be differentiated via automatic differentiation, i.e., using automatic differentiation to compute the derivatives of its outputs with respect to its inputs or parameters. Differentiable programming has been used extensively in machine learning for obtaining gradients required in optimization algorithms such as stochastic gradient descent (SGD). While incorporating differentiable programming with traditional functions is straightforward, it is challenging when the functions are complex, such as splines. In this work, we extend the differentiable programming paradigm to THB-splines. THB-splines offer an efficient approach for complex surface fitting by utilizing a hierarchical tensor structure of B-splines, enabling local adaptive refinement. However, this approach brings challenges, such as a larger computational overhead and the non-trivial implementation of automatic differentiation and parallel evaluation algorithms. We use custom kernel functions for GPU acceleration in forward and backward evaluation that are necessary for differentiable programming of THB-splines. Our approach not only improves computational efficiency but also significantly enhances the speed of surface evaluation compared to previous methods. Our differentiable THB-splines framework facilitates faster and more accurate surface modeling with local refinement, with several applications in CAD and isogeometric analysis.This article is published as Moola, Ajith, Aditya Balu, Adarsh Krishnamurthy, and Aishwarya Pawar. "THB-Diff: a GPU-accelerated differentiable programming framework for THB-splines." Engineering with Computers (2023): 1-17. doi: https://doi.org/10.1007/s00366-023-01929-1. © The Author(s) 2023. This open access article is licensed under a Creative Commons Attribution 4.0. (http://creativecommons.org/licenses/by/4.0/
MDPGT: Momentum-Based Decentralized Policy Gradient Tracking
No full textWe propose a novel policy gradient method for multi-agent reinforcement learning, which leverages two different variance-reduction techniques and does not require large batches over iterations. Specifically, we propose a momentum-based decentralized policy gradient tracking (MDPGT) where a new momentum-based variance reduction technique is used to approximate the local policy gradient surrogate with importance sampling, and an intermediate parameter is adopted to track two consecutive policy gradient surrogates. Moreover, MDPGT provably achieves the best available sample complexity of O(N−1ϵ−3) for converging to an ϵ-stationary point of the global average of N local performance functions (possibly nonconcave). This outperforms the state-of-the-art sample complexity in decentralized model-free reinforcement learning, and when initialized with a single trajectory, the sample complexity matches those obtained by the existing decentralized policy gradient methods. We further validate the theoretical claim for the Gaussian policy function. When the required error tolerance ϵ is small enough, MDPGT leads to a linear speed up, which has been previously established in decentralized stochastic optimization, but not for reinforcement learning. Lastly, we provide empirical results on a multi-agent reinforcement learning benchmark environment to support our theoretical findings.This is a preprint from Jiang, Zhanhong, Xian Yeow Lee, Sin Yong Tan, Kai Liang Tan, Aditya Balu, Young M. Lee, Chinmay Hegde, and Soumik Sarkar. "MDPGT: Momentum-based Decentralized Policy Gradient Tracking." arXiv preprint arXiv:2112.02813 (2021). doi: https://doi.org/10.48550/arXiv.2112.02813. Published as Jiang, Zhanhong, Xian Yeow Lee, Sin Yong Tan, Kai Liang Tan, Aditya Balu, Young M. Lee, Chinmay Hegde, and Soumik Sarkar. "MDPGT: momentum-based decentralized policy gradient tracking." In Proceedings of the AAAI conference on artificial intelligence, vol. 36, no. 9, pp. 9377-9385. 2022. doi: https://doi.org/10.1609/aaai.v36i9.21169
Learning and Visualizing Localized Geometric Features Using 3D-CNN: An Application to Manufacturability Analysis of Drilled Holes
No full text3D Convolutional Neural Networks (3D-CNN) have been used for object recognition based on the voxelized shape of an object. However, interpreting the decision making process of these 3D-CNNs is still an infeasible task. In this paper, we present a unique 3D-CNN based Gradient-weighted Class Activation Mapping method (3D-GradCAM) for visual explanations of the distinct local geometric features of interest within an object. To enable efficient learning of 3D geometries, we augment the voxel data with surface normals of the object boundary. We then train a 3D-CNN with this augmented data and identify the local features critical for decision-making using 3D GradCAM. An application of this feature identification framework is to recognize difficult-to-manufacture drilled hole features in a complex CAD geometry. The framework can be extended to identify difficult-to-manufacture features at multiple spatial scales leading to a real-time design for manufacturability decision support system.This is a proceeding preprint from Ghadai, Sambit, Aditya Balu, Adarsh Krishnamurthy, and Soumik Sarkar. "Learning and visualizing localized geometric features using 3d-cnn: An application to manufacturability analysis of drilled holes." arXiv preprint arXiv:1711.04851 (2017). doi: https://doi.org/10.48550/arXiv.1711.04851. Copyright 2017 The Authors
On Consensus-Optimality Trade-offs in Collaborative Deep Learning
No full textIn distributed machine learning, where agents collaboratively learn from diverse private data sets, there is a fundamental tension between consensus and optimality. In this paper, we build on recent algorithmic progresses in distributed deep learning to explore various consensus-optimality trade-offs over a fixed communication topology. First, we propose the incremental consensus-based distributed stochastic gradient descent (i-CDSGD) algorithm, which involves multiple consensus steps (where each agent communicates information with its neighbors) within each SGD iteration. Second, we propose the generalized consensus-based distributed SGD (g-CDSGD) algorithm that enables us to navigate the full spectrum from complete consensus (all agents agree) to complete disagreement (each agent converges to individual model parameters). We analytically establish convergence of the proposed algorithms for strongly convex and nonconvex objective functions; we also analyze the momentum variants of the algorithms for the strongly convex case. We support our algorithms via numerical experiments, and demonstrate significant improvements over existing methods for collaborative deep learning.This article is published as Jiang, Zhanhong, Aditya Balu, Chinmay Hegde, and Soumik Sarkar. "On consensus-optimality trade-offs in collaborative deep learning." Frontiers in artificial intelligence 4 (2021): 573731. doi: https://doi.org/10.3389/frai.2021.573731
Differentiable Programming for Piecewise Polynomial Functions
No full textWe introduce a new, principled approach to extend gradient-based optimization to piecewise smooth models, such as k-histograms, splines, and segmentation maps. We derive an accurate form of the weak Jacobian of such functions and show that it exhibits a block-sparse structure that can be computed implicitly and efficiently. We show that using the redesigned Jacobian leads to improved performance in applications such as denoising with piecewise polynomial regression models, datafree generative model training, and image segmentation.This proceeding is published as Cho, Minsu, Ameya Joshi, Xian Yeow Lee, Aditya Balu, Adarsh Krishnamurthy, Baskar Ganapathysubramanian, Soumik Sarkar, and Chinmay Hegde. "Differentiable Programming for Piecewise Polynomial Functions." NeurIPS Thirty-fourth Annual Conference on Neural Information Processing Systems. Learning Meets Combinatorial Algorithms (LMCA): Workshop at NeurIPS 2020. December 6-12, 2020. Posted with permission.</p
- …
