18,989 research outputs found
Locally linear approximation for Kernel methods : the Railway Kernel
No full textIn this paper we present a new kernel, the Railway Kernel, that works properly for
general (nonlinear) classification problems, with the interesting property that acts
locally as a linear kernel. In this way, we avoid potential problems due to the use of a
general purpose kernel, like the RBF kernel, as the high dimension of the induced
feature space. As a consequence, following our methodology the number of support
vectors is much lower and, therefore, the generalization capability of the proposed
kernel is higher than the obtained using RBF kernels. Experimental work is shown to
support the theoretical issues
A survey of kernel and spectral methods for clustering
No full textClustering algorithms are a useful tool to explore data structures and have been employed in many disciplines. The focus of this paper is the partitioning clustering problem with a special interest in two recent approaches: kernel and spectral methods. The aim of this paper is to present a survey of kernel and spectral clustering methods, two approaches able to produce nonlinear separating hypersurfaces between clusters. The presented kernel clustering methods are the kernel version of many classical clustering algorithms, e.g., K-means, SOM and neural gas. Spectral clustering arise from concepts in spectral graph theory and the clustering problem is configured as a graph cut problem where an appropriate objective function has to be optimized. An explicit proof of the fact that these two paradigms have the same objective is reported since it has been proven that these two seemingly different approaches have the same mathematical foundation. Besides, fuzzy kernel clustering methods are presented as extensions of kernel K-means clustering algorithm. (C) 2007 Pattem Recognition Society. Published by Elsevier Ltd. All rights reserved
Recommended from our members
Sparse kernel modelling: a unified approach
No full textA unified approach is proposed for sparse kernel data modelling that includes regression and classification as well as probability density function estimation. The orthogonal-least-squares forward selection method based on the leave-one-out test criteria is presented within this unified data-modelling framework to construct sparse kernel models that generalise well. Examples from regression, classification and density estimation applications are used to illustrate the effectiveness of this generic sparse kernel data modelling approach
Nonlinear Knowledge in Kernel Approximation
No full textPrior knowledge over arbitrary general sets is
incorporated into nonlinear kernel approximation problems in
the form of linear constraints in a linear program. The key
tool in this incorporation is a theorem of the alternative for
convex functions that converts nonlinear prior knowledge implications
into linear inequalities without the need to kernelize
these implications. Effectiveness of the proposed formulation is
demonstrated on two synthetic examples and an important lymph
node metastasis prediction problem. All these problems exhibit
marked improvements upon the introduction of prior knowledge
over nonlinear kernel approximation approaches that do not
utilize such knowledge
Approximate inference of the bandwidth in multivariate kernel density estimation
No full textKernel density estimation is a popular and widely used non-parametric method for data-driven density estimation. Its appeal lies in its simplicity and ease of implementation, as well as its strong asymptotic results regarding its convergence to the true data distribution. However, a major difficulty is the setting of the bandwidth, particularly in high dimensions and with limited amount of data. An approximate Bayesian method is proposed, based on the Expectation–Propagation algorithm with a likelihood obtained from a leave-one-out cross validation approach. The proposed method yields an iterative procedure to approximate the posterior distribution of the inverse bandwidth. The approximate posterior can be used to estimate the model evidence for selecting the structure of the bandwidth and approach online learning. Extensive experimental validation shows that the proposed method is competitive in terms of performance with state-of-the-art plug-in methods
Maximum kernel likelihood estimation
No full textWe introduce an estimator for the population mean based on maximizing likelihoods formed by parameterizing a kernel density estimate. Due to these origins, we have dubbed the estimator the maximum kernel likelihood estimate (mkle). A speedy computational method to compute the mkle based on binning is implemented in a simulation study which shows that the mkle at an optimal bandwidth is decidedly superior in terms of efficiency to the sample mean and other measures of location for heavy tailed symmetric distributions. An empirical rule and a computational method to estimate this optimal bandwidth are developed and used to construct bootstrap confidence intervals for the population mean. We show that the intervals have approximately nominal coverage and have significantly smaller average width than the standard t and z intervals. Lastly, we develop some mathematical properties for a very close approximation to the mkle called the kernel mean. In particular, we demonstrate that the kernel mean is indeed unbiased for the population mean for symmetric distributions
Comportamento estocástico do algoritmo kernel least-mean-square
No full textTese (doutorado) - Universidade Federal de Santa Catarina, Centro Tecnológico. Programa de Pós-Graduação em Engenharia Elétrica.Algoritmos baseados em kernel têm-se tornado populares no processamento não-linear de sinais. O processamento não-linear aplicado sobre um sinal pode ser modelado como um processamento linear aplicado a um sinal transformado para um espaço de Hilbert com kernels reprodutivos (RKHS). A operação linear no espaço transformado pode ser implementada com baixa complexidade e pode ser melhor estudada e projetada. O algoritmo Kernel Least-Mean-Squares (KLMS) é um algoritmo popular em filtragem adaptativa não-linear devido à sua simplicidade e robustez. Implementações práticas desse algoritmo requerem um modelo de ordem finita do processamento não-linear, o que modifica o comportamento do algoritmo em relação ao LMS simplesmente mapeado para o RKHS. Essa modificação leva à necessidade de novos modelos analíticos para o comportamento do algoritmo. O desempenho do algoritmo é função do passo de convergência e dos parâmetros do kernel empregado. Este trabalho estuda o comportamento do KLMS em regimes transitório e permanente para entradas Gaussianas e um modelo de não-linearidade de ordem finita. Dois kernels são considerados; o Gaussiano e o Polinomial. Derivamos modelos analíticos recursivos para os comportamentos do vetor médio de erros nos coeficientes e do erro quadrático médio de estimação. As previsões do modelo mostram excelente acordo com simulações de Monte Carlo no transitório e no regime permanente. Isso permite a determinação explícita das condições para a estabilidade, e permite escolher os parâmetros do algoritmo a fim de obter um desempenho desejado. Exemplos de projeto são apresentados para o kernel Gaussiano e para o kernel Polinomial de segundo grau de forma a validar a análise teórica e ilustrar sua aplicação.Kernel-based algorithms have become popular in nonlinear signal processing. A nonlinear processing can be modeled as a linear processing applied to a signal transformed to a reproducing kernel Hilbert space (RKHS). The linear operation in the transformed space can be implemented with low computational complexity and can be more easily studied and designed. The Kernel Least-Mean-Squares (KLMS) is a popular algorithm in nonlinear adaptive filtering due to its simplicity and robustness. Practical implementations of this algorithm require a finite order model for the nonlinear processing. This modifies the algorithm behavior as compared to the LMS simply mapped to the RKHS. This modification leads to the need for new analytical models for the algorithm behavior. The algorithm behavior is a function of both the step size and the kernel parameters. This work studies the KLMS algorithm behavior in transient and in steady-state for Gaussian inputs and for a finite order nonlinearity model. Two kernels are considered; the Gaussian and the Polinomial. We derive analytical models for the behavior of both the mean weight error vector and the mean-square estimation error. The model predictions show excellent agreement with Monte Carlo simulations at both the transient and the steady-state. This allows the explicit determination of the stability limits and to design the algorithm parameters to obtain a desired performance. Design examples are presented for the Gaussian and for the second degree Polinomial kernels to validate the analysis and to illustrate its application
Kernel principal component analysis (KPCA) for the de-noising of communication signals
No full textThis paper is concerned with the problem of de-noising for non-linear signals. Principal Component Analysis (PCA) cannot be applied to non-linear signals however it is known that using kernel functions, a non-linear signal can be transformed into a linear signal in a higher dimensional space. In that feature space, a linear algorithm can be applied to a non-linear problem. It is proposed that using the principal components extracted from this feature space, the signal can be de-noised in its input space
kernlab - An S4 Package for Kernel Methods in R
No full textkernlab is an extensible package for kernel-based machine learning methods in R. It takes advantage of R's new S4 ob ject model and provides a framework for creating and using kernel-based algorithms. The package contains dot product primitives (kernels), implementations of support vector machines and the relevance vector machine, Gaussian processes, a ranking algorithm, kernel PCA, kernel CCA, and a spectral clustering algorithm. Moreover it provides a general purpose quadratic programming solver, and an incomplete Cholesky decomposition method.
- …
