1,721,034 research outputs found

    Online algorithm design via smoothing with application to online experiment selection

    No full text
    Thesis (Ph.D.)--University of Washington, 2017-08In this dissertation, we present the results of our research on three topics, namely, the design and analysis of online convex optimization algorithms, convergence rate analysis of proximal gradient homotopy algorithm for structured convex problems, and application of computational methods for study of brain cells in the visual cortex of the primate brain. In our work on online optimization, we have developed a systematic approach with a clear connection to regret minimization for design and worst case analysis of online optimization algorithms. We apply this approach to online experiment design problems. Our results on the convergence rate analysis of proximal gradient homotopy algorithm extends the linear convergence rate of this algorithm from l1l_1 norm to a general class of norms called decomposable norms. Our results on the clustering of cells in the visual cortical area V4 reveal new categories of neurons in this area. We discuss how online adaptive algorithms can be utilized for classification of these neurons in closed loop neurophysiological experiments

    The Diminishing Returns (DR) Property and Its Applications in Machine Learning

    Full text link
    Thesis (Ph.D.)--University of Washington, 2023Numerous tasks in machine learning involve objective functions that exhibit a Diminishing Returns (DR) property, i.e., the marginal gain of increasing the input gets smaller as the input gets larger. For instance, in document summarization, the goal is to select a small subset of sentences that represent the entirety of the document. As the summary gets larger, the additional benefit of adding a sentence to the summary diminishes. In this dissertation, we focus on the class of set functions and continuous functions that exhibit the DR property. These functions are called submodular set functions and continuous DR-submodular functions respectively. This dissertation presents several contributions to various online and offline maximization problems in machine learning where the utility functions satisfy the DR property, with the main themes organized into three parts: (i) study of online DR-submodular maximization under online budget constraints and designing primal-dual algorithms that not only perform well in terms of the utility, but they also satisfy the online constraints; (ii) characterization of the class of strongly DR-submodular functions and providing techniques for offline and online maximization of these functions that utilize the additional structure to obtain improved convergence rates and regret guarantees respectively; and (iii) study of offline and online submodular set function maximization problems under social and economic considerations (e.g., privacy and strategic behavior) and designing differentially private and incentive-compatible algorithms for these problems

    Graph Design via Convex Optimization: Online and Distributed Perspectives

    No full text
    Thesis (Ph.D.)--University of Washington, 2017-06Network and graph have long been natural abstraction of relations in a variety of applications, e.g. transportation, power system, social network, communication, electrical circuit, etc. As a large number of computation and optimization problems are naturally defined on graphs, graph structures not only enable important properties of these problems, but also leads to highly efficient distributed and online algorithms. For example, graph separability enables the parallelism for computation and operation as well as limits the size of local problems. More interestingly, graphs can be defined and constructed in order to take best advantage of those problem properties. This dissertation focuses on graph structure and design in newly proposed optimization problems, which establish a bridge between graph properties and optimization problem properties. We first study a new optimization problem called \emph{Geodesic Distance Maximization Problem (GDMP)}. Given a graph with fixed edge weights, finding the shortest path, also known as the geodesic, between two nodes is a well-studied network flow problem. We introduce the Geodesic Distance Maximization Problem (GDMP): the problem of finding the edge weights that maximize the length of the geodesic subject to convex constraints on the weights. We show that GDMP is a convex optimization problem for a wide class of flow costs, and provide a physical interpretation using the dual. We present applications of the GDMP in various fields, including optical lens design, network interdiction, and resource allocation in the control of forest fires. We develop an Alternating Direction Method of Multipliers (ADMM) by exploiting specific problem structures to solve large-scale GDMP, and demonstrate its effectiveness in numerical examples. We then turn our attention to distributed optimization on graph with only local communication. Distributed optimization arises in a variety of applications, e.g. distributed tracking and localization, estimation problems in sensor networks, multi-agent coordination. Distributed optimization aims to optimize a global objective function formed by summation of coupled local functions over a graph via only local communication and computation. We developed a weighted proximal ADMM for distributed optimization using graph structure. This fully distributed, single-loop algorithm allows simultaneous updates and can be viewed as a generalization of existing algorithms. More importantly, we achieve faster convergence by jointly designing graph weights and algorithm parameters. Finally, we propose a new problem on networks called \emph{Online Network Formation Problem}: starting with a base graph and a set of candidate edges, at each round of the game, player one first chooses a candidate edge and reveals it to player two, then player two decides whether to accept it; player two can only accept limited number of edges and make online decisions with the goal to achieve the best properties of the synthesized network. The network properties considered include the number of spanning trees, algebraic connectivity and total effective resistance. These network formation games arise in a variety of cooperative multiagent systems. We propose a primal-dual algorithm framework for the general online network formation game, and analyze the algorithm performance by the competitive ratio and regret

    Learning structured matrices in high dimensions: Low rank estimation and structured graphical models

    Full text link
    Thesis (Ph.D.)--University of Washington, 2015The topic of learning matrix structures in the emph{high-dimensional statistical setting} has received a lot of attention in machine learning, statistics and signal processing literature. High dimensional setting refers to problems that have more parameters to estimate than samples or measurements. Examples of problems that fall in this area include matrix factorization, matrix rank minimization, and graphical model estimation. These problems arise in many applications including collaborative filtering, system identification, and learning gene-regulatory networks. The focus of this thesis is on the algorithmic and theoretical aspects of learning matrix structures in the high dimensional setting with emphasis on the two problems of matrix rank minimization and graphical model estimation. We first consider the problem of reconstructing a low-rank matrix given a few linear measurements. This problem is known to be NP-hard. We propose a family of iterative reweighted algorithms that reconstruct the low-rank matrix efficiently and accurately. We also give recovery guarantees for these algorithms under suitable assumptions on the measurement maps. Finally, we also apply our algorithms to the problems of system identification and matrix completion. Our extensive numerical experiments illustrate that our algorithms perform better than state of the art algorithms on both synthetic and real data when the rank of the true underlying matrix is unknown. We next consider the problem of learning structured Gaussian graphical models in the high dimensional setting. This problem has a lot of applications ranging from biological modeling to social networks. While existing literature focuses on learning Gaussian graphical models using a simple sparsity regularizer (e.g. the graphical lasso formulation), applications often present prior knowledge that require more structured regularizers. Examples of networks that occur in applications include scale-free networks (where a few nodes have a high degree), community networks (where the connections occur in small dense communities), etc. We propose the structured graphical lasso formulation, a framework that generalizes graphical lasso to learning graphical models with arbitrary user-specified structures. We continue by considering a special case of the structured graphical lasso that pertains to learning graphical model with a few hub nodes. We propose an ADMM algorithm to solve this formulation. Empirical studies indicate the superiority of hub graphical lasso over graphical lasso and other traditional methods in learning graphs with few hubs. We also give model selection consistency results for recovering graphs with few hubs. When the number of hubs to be learned is O(1), hub graphical lasso requires far fewer samples than graphical lasso to learn the right graphical model. We move on to study the problem of estimating multiple Gaussian graphical models where information is shared between the graphical models. We propose different formulations that capture different ways in which information is shared between the networks. Our empirical results indicate that our approach does better than existing approaches in estimating gene-regulatory networks with shared information. We switch gears by focusing on methods to reduce computational time for our models. We note that the proposed ADMM algorithm for hub graphical lasso and other formulations, though effective, involves a singular value decomposition in each iteration whose computational cost can be prohibitive for large problem sizes. To alleviate the computational bottle neck, we first provide algorithm independent approaches (based on screening rules) to speed up our proposed convex formulations. These screening rules are used to break down the problem into independent sub-problems (where possible), yielding significant speed-ups in computation. Typical algorithms for learning sparse graphical models require a Cholesky computation to check for positive definiteness of the iterates. We propose a block-coordinate descent algorithm with step-size selection rules that avoid expensive Cholesky checks and only rely on matrix vector multiplies in each iteration. Our empirical results show significant speed ups over the SVD based ADMM algorithm

    Nonconvex Optimization and Model Representation with Applications in Control Theory and Machine Learning

    No full text
    Thesis (Ph.D.)--University of Washington, 2022In control and machine learning, the primary goal is to learn the models that make predictions or decisions and act in the world. This thesis covers two important aspects for control theory and machine learning: the model structure that allows low training and generalization error with few samples (i.e., low sample complexity), and convergence guarantees for first-order optimization algorithms for nonconvex optimization. If the model and the training algorithm apply the knowledge of the structure of data (such as sparsity, low-rankness, etc.), the model can be learned with low sample complexity. We present two results, the Hankel nuclear norm regularization method for learning a low order system, and the overparameterized representation for linear meta-learning. We study dynamical system identification in the first result. We assume the true system order is low. A low system order means that the state can be represented by a low dimensional vector, and the system corresponds to a low rank Hankel matrix. The low-rankness is known to be encouraged by nuclear norm regularized estimator in matrix completion theory. We apply a nuclear norm regularized estimator for Hankel matrix, and show that it requires fewer samples than the ordinary least squares estimator. We study linear meta-learning in the second part. The meta-learning algorithm contains two steps: learning a large model in representation learning stage, and fine tuning the model in few-shot learning stage. The few-shot dataset contains few samples, and to avoid overfitting, we need a fine-tuning algorithm that uses the information from representation learning. We generalize the subspace-based model in prior arts to Gaussian model, and describe the overparameterized meta-learning procedure. We show that the feature-task alignment reduces the sample complexity in representation learning, and the optimal task representation is overparameterized. First order optimization methods such as gradient based method, is widely used in machine learning thanks to its simplicity for implementation and fast convergence. However, the objective function in machine learning can be nonconvex, and the first order method has only the theoretical guarantee that it converges to a stationary point, rather than a local/global minimum. We dive into more refined analysis of the convergence guarantee, and present two results, the convergence of perturbed gradient descent approach to a local minimum on Riemannian manifold, and a unified global convergence result of policy gradient descent for linear system control problems. We study how Riemannian gradient converges to an approximate local minimum in the first part. While it is well-known that the perturbed gradient descent escapes saddle points in Euclidean space, less is known about the concrete convergence rate when we apply Riemannian gradient descent on the manifold. In the first result, we show that the perturbed Riemannian gradient descent converges to an approximate local minimum and reveal the relation between convergence rate and the manifold curvature. We study the policy gradient descent applied in control in the second part. Many control problems are revisited under the context of the recent boom in reinforcement learning (RL), however, there is a gap between the RL and control methodology: The policy gradient in RL applies first-order method on nonconvex landscape, and it is hard to show they converge to global minimum, while control theory invents reparameterization that makes the problem convex and they are proven to find the globally optimal controller in polynomial time. Targeting on interpreting the success of the nonconvex method, in the second result, we connect the nonconvex policy gradient descent applied for a collection of control problems with their convex parameterization, and propose a unified proof for the global convergence of policy gradient descent

    Online Decision Making: DR-Submodular Objectives and Stochastic Linear Constraints

    No full text
    Thesis (Master's)--University of Washington, 2020In this thesis, we consider online continuous DR-submodular maximization with linear stochastic long-term constraints. Compared to the prior work on online submodular maximization \cite{chenOnlineContinuousSubmodular2018}, our setting introduces the extra complication of stochastic linear constraint functions generated at each round and are independent and identically distributed (i.i.d). To be precise, at step t{1,,T}t\in\{1,\dots,T\}, a DR-submodular utility function ft()f_t(\cdot) and a constraint vector ptp_t, i.i.d. generated from an unknown distribution with mean pp, are revealed after committing to an action xtx_t and we aim to maximize the overall utility while the expected cumulative resource consumption t=1Tp,xt\sum_{t=1}^T \langle p,x_t\rangle is below a fixed budget BTB_T. Stochastic long-term constraints arise naturally in applications where there is a limited budget or resource available and resource consumption at each step is governed by stochastically time-varying environments. We propose the Online Lagrangian Frank-Wolfe (OLFW) algorithm to solve this class of online problems. We analyze the performance of the OLFW algorithm and we obtain sub-linear regret bounds as well as sub-linear cumulative constraint violation bounds, both in expectation and with high probability

    Exploration and Primal-dual Methods in Bandits and Reinforcement Learning

    Full text link
    Thesis (Ph.D.)--University of Washington, 2025Sequential decision-making, which encompasses both bandit problems and reinforcement learning, forms the foundation of intelligent systems across diverse applications, from adaptive recommendation systems to autonomous robotics. This thesis addresses two fundamental challenges in building reliable, sample-efficient agents that operate robustly in dynamic, complex environments: efficient exploration in non-stationary or structurally complex settings, and the design of appropriate objective functions when multiple approximation layers are inevitable. Regarding the efficient exploration, we develop the first robust pure exploration algorithm for both stationary and non-stationary linear bandits, achieving strong performance in benign settings while maintaining robustness to environmental changes. For single-step congestion games, we exploit the structure of this special class of games to develop the first algorithms for Nash equilibrium learning under various feedback models. For tabular reinforcement learning, we propose the first near-optimal randomized exploration algorithm that nearly matches the fundamental lower bound. Regarding the objective design, we analyze learning objectives through the lens of duality between value learning and policy learning. In an online selective sampling problem for linear bandits, we characterize an optimal ellipsoid-based selection rule through primal-dual analysis. For approximate policy optimization, we propose using dual Bregman divergence instead of the common Euclidean norm to measure similarity in dual space, resulting in the first policy optimization framework with both fast theoretical convergence and superior practical performance. Collectively, these contributions advance the theoretical frontier of exploration and objective design, close several open complexity gaps, and provide practical algorithms validated on robotic control benchmarks. They offer a principled route towards agents that learn robustly and act reliably in dynamic, high-dimensional environments
    corecore