1,720,983 research outputs found
Wavelet-like representation of multivariate periodic functions on scattered data by periodic translation networks
No full textThesis (M.S.)
California State University, Los Angeles, 2012Committee members: Hrushikesh N Mhaskar, Michael Hoffman, Gerald Beer, Grant FraserApproximation theory, Local smoothness, Low pass filter, Periodic translation network, Scattered data, Summability kernelThis thesis is intended to study and develop the machinery of analysis necessary to perform good approximations of multivariate periodic functions defined using partial information, with which one can retrieve local information about their smoothness. The techniques combine aspects of harmonic analysis, approximation theory, and learning theory. Pioneering work in this field by Mhaskar and his collaborators over the past two decades has led to wavelet-like expansions where the terms are defined using global information such as Fourier coefficients, and yet can reveal local smooth- ness of a target function. New techniques allow explicit constructions of periodic translation networks with a priori performance guarantees, without invoking further optimization routines. The novel contribution of this thesis is such a construction in which the terms are defined using scattered data of the target function, and the local smoothness of the target function is characterized by the local behaviour of the constituent networks
Wavelet-like representation of multivariate periodic functions on scattered data by periodic translation networks
No full textThis thesis is intended to study and develop the machinery of analysis necessary to perform good approximations of multivariate periodic functions defined using partial information, with which one can retrieve local information about their smoothness. The techniques combine aspects of harmonic analysis, approximation theory, and learning theory. Pioneering work in this field by Mhaskar and his collaborators over the past two decades has led to wavelet-like expansions where the terms are defined using global information such as Fourier coefficients, and yet can reveal local smooth- ness of a target function. New techniques allow explicit constructions of periodic translation networks with a priori performance guarantees, without invoking further optimization routines. The novel contribution of this thesis is such a construction in which the terms are defined using scattered data of the target function, and the local smoothness of the target function is characterized by the local behaviour of the constituent networks.Committee members: Hrushikesh N Mhaskar, Michael Hoffman, Gerald Beer, Grant FraserThesis (M.S.) California State University, Los Angeles, 2012Approximation theory, Local smoothness, Low pass filter, Periodic translation network, Scattered data, Summability kerne
Deep vs. shallow networks : An approximation theory perspective
Get PDFThe paper briefly reviews several recent results on hierarchical architectures for learning from examples, that may formally explain the conditions under which Deep Convolutional Neural Networks perform much better in function approximation problems than shallow, one-hidden layer architectures. The paper announces new results for a non-smooth activation function – the ReLU function – used in present-day neural networks, as well as for the Gaussian networks. We propose a new definition of relative dimension to encapsulate different notions of sparsity of a function class that can possibly be exploited by deep networks but not by shallow ones to drastically reduce the complexity required for approximation and learning.This work was supported by the Center for Brains, Minds and Machines (CBMM), funded by NSF STC award CCF – 1231216
Approximation by non-symmetric networks for cross-domain learning
Get PDFFor the past 30 years or so, machine learning has stimulated a great deal of
research in the study of approximation capabilities (expressive power) of a
multitude of processes, such as approximation by shallow or deep neural
networks, radial basis function networks, and a variety of kernel based
methods. Motivated by applications such as invariant learning, transfer
learning, and synthetic aperture radar imaging, we initiate in this paper a
general approach to study the approximation capabilities of kernel based
networks using non-symmetric kernels. While singular value decomposition is a
natural instinct to study such kernels, we consider a more general approach to
include the use of a family of kernels, such as generalized translation
networks (which include neural networks and translation invariant kernels as
special cases) and rotated zonal function kernels. Naturally, unlike
traditional kernel based approximation, we cannot require the kernels to be
positive definite. In particular, we obtain estimates on the accuracy of
uniform approximation of functions in a ()-Sobolev class by ReLU
networks when is not necessarily an integer. Our general results apply to
the approximation of functions with small smoothness compared to the dimension
of the input space
Dimension independent bounds for approximation of smooth functions on metric spaces
No full textNon UBCUnreviewedAuthor affiliation: Claremont Graduate UniversityFacult
Globally accessible finite element method based web-solver for the vibrational Schro??dinger equation and its application to HC???O carbon chain free dadical, ZnCl?????? and hydroxyacetaldehyde
No full textIncludes bibliographical references (pages 173-188).A software package was developed for the ab initio solution of the multidimensional, adiabatic vibrational Schro??dinger equation (VSE), using the nite element method (FEM). The package is extended and improved with a point-wise PES evaluation as well as a revised routine for computation of the Wilson G matrix on scattered surfaces. The weak formulation of the VSE requires the potential energy surface (PES) to be evaluated at speci c locations based on the FEM discretization. Recent developments in automated PES generation result in two different interpolation schemes available in the package: second degree interpolating moving least squares and Kriging interpolation with variogram parameter tuning and optimized cluster size. A significant improvement has been made to the Wilson G matrix evaluation, where rotational discrepancies for planar and symmetrical molecules are removed before imposing the Eckart conditions. The new algorithm is adapted to scattered surfaces, which assures no information is lost during the transformation from Cartesian to internal coordinates. The entire package has been fully automated and made available as a web-based, platform-independent solver. In addition to the above mentioned improvements the solver was used to perform vibrational analyses on two molecular systems: the lowest excited state ??II[subscript g] of ZnCl?????? and the X??A ?? ground state of HC???O carbon chain free radical. Both species exhibit a strongly anharmonic PES, with two non-equivalent minima separated by a small energy barrier, which makes them ideal candidates for FEM-based probing. The solver has also been used for explicit quantum mechanical solution of the partition function of hydroxyacetaldehyde internal rotors
Theory I: Why and When Can Deep Networks Avoid the Curse of Dimensionality?
No full textThe paper characterizes classes of functions for which deep learning can be exponentially better than shallow learning. Deep convolutional networks are a
special case of these conditions, though weight sharing is not the main reason for their exponential advantage
Why and when can deep-but not shallow-networks avoid the curse of dimensionality: A review
Get PDFThe paper reviews and extends an emerging body of theoretical results on deep learning including the conditions under which it can be exponentially better than shallow learning. A class of deep convolutional networks represent an important special case of these conditions, though weight sharing is not the main reason for their exponential advantage. Implications of a few key theorems are discussed, together with new results, open problems and conjectures.McGovern Institute for Brain Research at MIT. Center for Brains, Minds, and MachinesNational Science Foundation (U.S.) (STC award CCF (No. 1231216))United States. Army Research Office (No. W911NF-15-1-0385
- …
