1,721,035 research outputs found
Nonlinear optimization and support vector machines
No full textSupport Vector Machine (SVM) is one of the most important class of machine learning models and algorithms, and has been successfully applied in various fields. Nonlinear optimization plays a crucial role in SVM methodology, both in defining the machine learning models and in designing convergent and efficient algorithms for large-scale training problems. In this paper we present the convex programming problems underlying SVM focusing on supervised binary classification. We analyze the most important and used optimization methods for SVM training problems, and we discuss how the properties of these problems can be incorporated in designing useful algorithms
Exploiting derivative-free local searches in DIRECT-type algorithms for global optimization
Full text linkIn this paper we consider bound constrained global optimization problems where first-order derivatives of the objective function can be neither computed nor
approximated explicitly. For the solution of such problems the DIRECT algorithm has been proposed which has a good ability to locate promising regions of the feasible domain and convergence properties based on the generation of a dense set of points over the feasible domain. However, the efficiency of DIRECT deteriorates as the dimension and the ill-conditioning of the objective function increase. To overcome these limits, we propose DIRECT-type algorithms enriched by the efficient use of derivative-free local searches combined with nonlinear transformations of the feasible domain and, possibly, of the objective function. We report extensive numerical results both on test problems from the literature and on an application in structural
proteomics
Three methods for robust grading
No full textWe propose three strategies by which a professor of a university course can assign final letter grades tak- ing into account the natural uncertainty in students’ individual assignment and final numerical grades. The first strategy formalizes a common technique that identifies large gaps in the final numerical grades. For the second and third strategies, we introduce the notion of a borderline student , that is, a student who is close to, but below, the breakpoint for the next highest letter grade. Using mixed-integer linear programming and a tailor-made branch-and-bound algorithm, we choose the letter-grade breakpoints to minimize the number of borderline students. In particular, the second strategy treats the uncertainty implicitly and minimizes the number of borderline students, while the third strategy uses a robust- optimization approach to minimize the maximum number of borderline students that could occur based on an explicit uncertainty set. We compare the three strategies on realistic instances and identify over- all trends as well as some interesting exceptions. While no strategy appears best in all cases, each can be computed in a reasonable amount of time for a moderately sized course. Moreover, they collectively provide the professor important insight into how uncertainty affects the assignment of final letter grades
Improving non-intrusive load disaggregation through an attention-based deep neural network
Full text linkEnergy disaggregation, known in the literature as Non-Intrusive Load Monitoring (NILM), is the task of inferring the power demand of the individual appliances given the aggregate power demand recorded by a single smart meter which monitors multiple appliances. In this paper, we propose a deep neural network that combines a regression subnetwork with a classification subnetwork for solving the NILM problem. Specifically, we improve the generalization capability of the overall architecture by including an encoder–decoder with a tailored attention mechanism in the regression subnetwork. The attention mechanism is inspired by the temporal attention that has been successfully applied in neural machine translation, text summarization, and speech recognition. The experiments conducted on two publicly available datasets—REDD and UK-DALE—show that our proposed deep neural network outperforms the state-of-the-art in all the considered experimental conditions. We also show that modeling attention translates into the network’s ability to correctly detect the turning on or off an appliance and to locate signal sections with high power consumption, which are of extreme interest in the field of energy disaggregation
An optimization-based method for feature ranking in nonlinear regression problems
No full textIn this brief, we consider the feature ranking problem, where, given a set of training instances, the task is to associate a score with the features in order to assess their relevance. Feature ranking is a very important tool for decision support systems, and may be used as an auxiliary step of feature selection to reduce the high dimensionality of real-world data. We focus on regression problems by assuming that the process underlying the generated data can be approximated by a continuous function (for instance, a feedforward neural network). We formally state the notion of relevance of a feature by introducing a minimum zero-norm inversion problem of a neural network, which is a nonsmooth, constrained optimization problem. We employ a concave approximation of the zero-norm function, and we define a smooth, global optimization problem to be solved in order to assess the relevance of the features. We present the new feature ranking method based on the solution of instances of the global optimization problem depending on the available training data. Computational experiments on both artificial and real data sets are performed, and point out that the proposed feature ranking method is a valid alternative to existing methods in terms of effectiveness. The obtained results also show that the method is costly in terms of CPU time, and this may be a limitation in the solution of large-dimensional problems
A new branch-and-bound algorithm for standard quadratic programming problems
Full text linkIn this paper we propose convex and LP bounds for standard quadratic programming (StQP) problems and employ them within a branch-and-bound approach. We first compare different bounding strategies for StQPs in terms both of the quality of the bound and of the computation times. It turns out that the polyhedral bounding strategy is the best one to be used within a branch-and-bound scheme. Indeed, it guarantees a good quality of the bound at the expense of a very limited computation time. The proposed branch-and-bound algorithm performs an implicit enumeration of all the KKT (stationary) points of the problem. We compare different branching strategies exploiting the structure of the problem. Numerical results on randomly generated problems (with varying density of the underlying convexity graph) are reported which show the effectiveness of the proposed approach, in particular in limiting the growth of the number of nodes in the branch-and-bound tree as the density of the underlying graph increases
A derivative based algorithm for a particular class of mixed variable optimization problems
No full textIn this article, we consider a particular class of nonlinear mixed variable optimization problems where the structure
and the number of variables of the problem depend on the values of some discrete variables. The peculiarity of this
class is that, for fixed values of the integer variables, the corresponding continuous optimization problem contains no
constraints and a large number of variables. For such a class of problems we propose two minimization algorithms
and prove their global convergence properties
3β,5α,6β-Trihydroxylated sterols with a saturated nucleus from two populations of the marine sponge Cliona copiosa
No full textNonlinear optimization and support vector machines
Full text linkSupport Vector Machine (SVM) is one of the most important class of
machine learning models and algorithms, and has been successfully applied in various
fields. Nonlinear optimization plays a crucial role in SVM methodology, both in defining
the machine learning models and in designing convergent and efficient algorithms
for large-scale training problems. In this paper we present the convex programming
problems underlying SVM focusing on supervised binary classification. We analyze
the most important and used optimization methods for SVM training problems, and
we discuss how the properties of these problems can be incorporated in designing
useful algorithms
A computational study on QP problems with general linear constraints
Full text linkIn this paper we consider Quadratic Programming (QP) problems with general linear constraints. We show, through a computational investigation, that a careful selection of a suitable reformulation of such problems, together with the related relaxation, and an intensive application of bound tightening are simple but very effective ingredients in order to make a standard branch and bound approach very competitive and in some cases able to outperform even well known commercial solvers
- …
