1,720,995 research outputs found
Regularized learning schemes in feature Banach spaces
No full textThis paper proposes a unified framework for the investigation of constrained learning theory in reflexive Banach spaces of features via regularized empirical risk minimization. The focus is placed on Tikhonov-like regularization with totally convex functions. This broad class of regularizers provides a flexible model for various priors on the features, including, in particular, hard constraints and powers of Banach norms. In such context, the main results establish a new general form of the representer theorem and the consistency of the corresponding learning schemes under general conditions on the loss function, the geometry of the feature space, and the modulus of total convexity of the regularizer. In addition, the proposed analysis gives new insight into basic tools such as reproducing Banach spaces, feature maps, and universality. Even when specialized to Hilbert spaces, this framework yields new results that extend the state of the art
A strongly convergent reflection method for finding the projection onto the intersection of two closed convex sets in a Hilbert space
No full textA new iterative method for finding the projection onto the intersection of two closed convex sets in a Hilbert space is presented. It is a Haugazeau-like modification of a recently proposed averaged alternating reflections method which produces a strongly convergent sequence
Finding best approximation pairs relative to two closed convex sets in Hilbert spaces
No full textWe consider the problem of finding a best approximation pair, i.e., two points which achieve the minimum distance between two closed convex sets in a Hilbert space. When the sets intersect, the method under consideration, termed AAR for averaged alternating reflections, is a special instance of an algorithm due to Lions and Mercier for finding a zero of the sum of two maximal monotone operators. We investigate systematically the asymptotic behavior of AAR in the general case when the sets do not necessarily intersect and show that the method produces best approximation pairs provided they exist. Finitely many sets are handled in a product space, in which case the AAR method is shown to coincide with a special case of Spingarn's method of partial inverses
On the structure of some phase retrieval algorithms
No full textThe state of the art for solving the phase retrieval problem in two dimensions relies heavily on the algorithms proposed by Gerchbercy, Saxton, and Fienup. Despite the widespread use of these algorithms, current mathematical theory cannot explain their remarkable success. It is already known that the Gerchberg-Saxton algorithm is a nonconvex version of method of alternating projections. In this paper, we show that two other prominent phase retrieval methods also have well known counterparts in the world of convex optimization algorithms: Fienup's basic input-output algorithm corresponds to Dykstra's algorithm, and Fienup's hybrid input-output algorithm can be viewed as an instance of the Douglas-Rachford algorithm. This work provides a theoretical framework to better understand and, potentially, improve existing phase recovery algorithms
Fixed-Point Algorithms for Inverse Problems in Science and Engineering
No full text"Fixed-Point Algorithms for Inverse Problems in Science and Engineering" presents some of the most recent work from top-notch researchers studying projection and other first-order fixed-point algorithms in several areas of mathematics and the applied sciences. The material presented provides a survey of the state-of-the-art theory and practice in fixed-point algorithms, identifying emerging problems driven by applications, and discussing new approaches for solving these problems. This book incorporates diverse perspectives from broad-ranging areas of research including, variational
Phase retrieval, error reduction algorithm, and fienup variants: a view from convex optimization
No full textThe phase retrieval problem is of paramount importance in various areas of applied physics and engineering. The state of the art for solving this problem in two dimensions relies heavily on the pioneering work of Gerchberg, Saxton, and Fienup. Despite the widespread use of the algorithms proposed by these three researchers, current mathematical theory cannot explain their remarkable success. Nevertheless, great insight can be gained into the behavior, the shortcomings, and the performance of these algorithms from their possible counterparts in convex optimization theory. An important step in this direction was made two decades ago when the error reduction algorithm was identified as a nonconvex alternating projection algorithm. The purpose of this paper is to formulate the phase retrieval problem with mathematical care and to establish new connections between well established numerical phase retrieval schemes and classical convex optimization methods. Specifically, it is shown that Fienup’s basic inputoutput algorithm corresponds to Dykstra’s algorithm, and that Fienup’s hybrid input-output algorithm can be viewed as an instance of the Douglas-Rachford algorithm. This work provides a theoretical framework to better understand and, potentially, improve existing phase recovery algorithms. 1
Consistent learning by composite proximal thresholding
No full textWe investigate the modeling and the numerical solution of machine learning problems with prediction functions which are linear combinations of elements of a possibly infinite dictionary of functions. We propose a novel flexible composite regularization model, which makes it possible to incorporate various priors on the coefficients of the prediction function, including sparsity and hard constraints. We show that the estimators obtained by minimizing the regularized empirical risk are consistent in a statistical sense, and we design an error-tolerant composite proximal thresholding algorithm for computing such estimators. New results on the asymptotic behavior of the proximal forward–backward splitting method are derived and exploited to establish the convergence properties of the proposed algorithm. In particular, our method features a o(1 / m) convergence rate in objective values
- …
