1,720,981 research outputs found
Riemannian multiplicative update for sparse simplex constraint using oblique rotation manifold
No full textWe propose a new manifold optimization method to solve low-rank problems with sparse simplex constraints (variables are simultaneous nonnegativity, sparsity, and sum-to-1) that are beneficial in applications. The proposed approach exploits oblique rotation manifolds, rewrite the problem, and introduce a new Riemannian optimization method. Experiments on synthetic datasets compared to the standard Euclidean method show the effectiveness of the proposed method
Chordal-NMF with Riemannian Multiplicative Update
No full textNonnegative Matrix Factorization (NMF) is the problem of approximating a given nonnegative matrix M through the conic combination of two nonnegative low-rank matrices W and H. Traditionally NMF is tackled by optimizing a specific objective function evaluating the quality of the approximation. This assessment is often done based on the Frobenius norm. In this study, we argue that the Frobenius norm as the "point-to-point" distance may not always be appropriate. Due to the nonnegative combination resulting in a polyhedral cone, this conic perspective of NMF may not naturally align with conventional point-to-point distance measures. Hence, a ray-to-ray chordal distance is proposed as an alternative way of measuring the discrepancy between M and WH. This measure is related to the Euclidean distance on the unit sphere, motivating us to employ nonsmooth manifold optimization approaches.We apply Riemannian optimization technique to solve chordal-NMF by casting it on a manifold. Unlike existing works on Riemannian optimization that require the manifold to be smooth, the nonnegativity in chordal-NMF is a non-differentiable manifold. We propose a Riemannian Multiplicative Update (RMU) that preserves the convergence properties of Riemannian gradient descent without breaking the smoothness condition on the manifold.We showcase the effectiveness of the Chordal-NMF on synthetic datasets as well as real-world multispectral images
A new Riemannian Multiplicative Updates for Chordal Nonnegative Matrix Factorization
No full textNonnegative Matrix Factorization (NMF) is the problem of approximating a given nonnegative matrix M through the product of two nonnegative low-rank matrices W and H. Traditionally NMF is tackled by optimizing a specific objective function evaluating the quality of the approximation. This assessment is often done based on the Frobenius norm (F-norm). In this work, we argue that the F-norm, as the “point-to-point” distance, may not always be appropriate. Viewing from the perspective of cone, NMF may not naturally align with F-norm. So, a ray-to-ray chordal distance is proposed as an alternative way of measuring the quality of the approximation. As this measure corresponds to the Euclidean distance on the sphere, it motivates the use of manifold optimization techniques. We apply Riemannian optimization technique to solve chordal-NMF by casting it on a manifold. Unlike works on Riemannian optimization that require the manifold to be smooth, the nonnegativity in chordal-NMF defines a non-differentiable manifold. We propose a Riemannian Multiplicative Update (RMU), and showcase the effectiveness of the chordal-NMF on synthetic and real-world datasets
A new Riemannian Multiplicative Updates for Chordal Nonnegative Matrix factorization
No full textNonnegative Matrix Factorization (NMF) is the problem of approximating a given nonnegative matrix M through the product of two nonnegative low-rank matrices W and H. Traditionally NMF is tackled by optimizing a specific objective function evaluating the quality of the approximation. This assessment is often done based on the Frobenius norm (F-norm). In this work, we argue that the F-norm, as the “point-to-point” distance, may not always be appropriate. Viewing from the perspective of cone, NMF may not naturally align with F-norm. So, a ray-to-ray chordal distance is proposed as an alternative way of measuring the quality of the approximation. As this measure corresponds to the Euclidean distance on the sphere, it motivates the use of manifold optimization techniques. We apply Riemannian optimization technique to solve chordal-NMF by casting it on a manifold. Unlike works on Riemannian optimization that require the manifold to be smooth, the nonnegativity in chordal-NMF defines a non-differentiable manifold. We propose a Riemannian Multiplicative Update (RMU), and showcase the effectiveness of the chordal-NMF on synthetic and real-world datasets
Inexact higher-order proximal algorithms for tensor factorization
No full textIn the last decades, Matrix Factorization (MF) models and their multilinear extension-Tensor Factorization (TF) models have been shown to be powerful tools for high dimensional data analysis and features extraction. Computing MF's or TF's are commonly achieved by solving a constrained optimization subproblem on each block of variables, where the subproblems usually have a huge problem size that one has to rely on First-order Methods (FoM), i.e., gradient-based optimization methods. In this work, we consider Higher-order Methods (HoM), which are based on higher-order derivatives of the objective function. Compared to FoM, HoM are faster both in theory and practice. However, HoM has a higher per-iteration cost than FoM. Based on the recent development of efficient and implementable HoM, we consider higher-order proximal point methods within the BLUM framework which is potentially tractable for large-scale problems. For the newly proposed HoM, we introduce the appropriate objective functions, derive the algorithm, and show experimentally that the drop in the number of iterations with respect to their per-iteration cost make these HoM-based algorithms attractive for computing MF's and TF's
MGProx: a nonsmooth multigrid proximal gradient method with adaptive restriction for strongly convex optimization
No full textWe study the combination of proximal gradient descent with multigrid for solving a class of possibly nonsmooth strongly convex optimization problems. We propose a multigrid proximal gradient method called MGProx, which accelerates the proximal gradient method by multigrid, based on using hierarchical information of the optimization problem. MGProx applies a newly introduced adaptive restriction operator to simplify the Minkowski sum of subdifferentials of the nondifferentiable objective function across different levels. We provide a theoretical characterization of MGProx. First we show that the MGProx update operator exhibits a fixed-point property. Next, we show that the coarse correction is a descent direction for the fine variable of the original fine level problem in the general nonsmooth case. Last, under some assumptions we provide the convergence rate for the algorithm. In the numerical tests on the elastic obstacle problem, which is an example of a nonsmooth convex optimization problem where the multigrid method can be applied, we show that MGProx has a faster convergence speed than competing methods
Optimal network pricing with oblivious users: a new model and algorithm
No full textTraffic modeling is important in modern society. In this work we propose a new model on the optimal network pricing (Onp) with the assumption of oblivious users, in which the users remain oblivious to real-time traffic conditions and others' behavior. Inspired by works on transportation research and network pricing for selfish traffic, we mathematically derive and prove a new formulation of Onp with decision-dependent modeling that relax certain existing modeling constraints in the literature. Then, we express the Onp formulation as a constrained nonconvex stochastic quadratic program with uncertainty, and we propose an efficient algorithm to solve the problem, utilizing graph theory, sparse linear algebra and stochastic approximation. Lastly, we showcase the effectiveness of the proposed algorithm and the usefulness of the new Onp formulation. The proposed algorithm achieves a 5x speedup by exploiting the sparsity structure of the model
Binno: a 1st-order method for Bi-level nonconvex nonsmooth optimization for matrix factorizations
No full textIn this work, we develop a method for nonconvex, nonsmooth bi-level optimization and we introduce Binno, a first order method that leverages proximal constructions together with carefully designed descent conditions and variational analysis. Within this framework, Binno provably enforces a descent property for the overall objective surrogate associated with the bi-level problem. Each iteration performs blockwise proximal-gradient updates for the upper and the lower problems separately and then forms a calibrated, block-diagonal convex combination of the two tentative iterates. A linesearch selects the combination weights to enforce simultaneous descent of both level-wise objectives, and we establish conditions guaranteeing the existence of such weights together with descent directions induced by the associated proximal-gradient maps. We also apply Binno in the context of sparse low-rank factorization, where the upper level uses elementwise penalties and the lower level uses nuclear norms, coupled via a Frobenius data term. We test Binno on synthetic matrix and a real traffic-video dataset, attaining lower relative reconstruction error and higher peak signal-to-noise ratio than some standard methods
Accelerating nonnegative matrix factorization algorithms using extrapolation
No full textWe propose a general framework to accelerate significantly the algorithms for nonnegative matrix factorization (NMF). This framework is inspired from the extrapolation scheme used to accelerate gradient methods in convex optimization and from the method of parallel tangents. However, the use of extrapolation in the context of the exact coordinate descent algorithms tackling the nonconvex NMF problems is novel. We illustrate the performance of this approach on two state-of-the-art NMF algorithms: accelerated hierarchical alternating least squares and alternating nonnegative least squares, using synthetic, image, and document data sets
Algorithms and comparisons of nonnegative matrix factorizations with volume regularization for hyperspectral unmixing
No full textIn this paper, we consider nonnegative matrix factorization (NMF) with a regularization that promotes small volume of the convex hull spanned by the basis matrix. We present highly efficient algorithms for three different volume regularizers, and compare them on endmember recovery in hyperspectral unmixing. The NMF algorithms developed in this paper are shown to outperform the state-of-The-Art volume-regularized NMF methods, and produce meaningful decompositions on real-world hyperspectral images in situations where endmembers are highly mixed (no pure pixels). Furthermore, our extensive numerical experiments show that when the data is highly separable, meaning that there are data points close to the true endmembers, and there are a few endmembers, the regularizer based on the determinant of the Gramian produces the best results in most cases. For data that is less separable and/or contains more endmembers, the regularizer based on the logarithm of the determinant of the Gramian performs best in general.</p
- …
