1,721,018 research outputs found
Prediction of protein β-residue contacts by Markov logic networks with grounding-specific weights
No full textMotivation: Accurate prediction of contacts between β-strand residues can significantly contribute towards ab initio prediction of the 3D structure of many proteins. Contacts in the same protein are highly interdependent. Therefore, significant improvements can be expected by applying statistical relational learners that overcome the usual machine learning assumption that examples are independent and identically distributed. Furthermore, the dependencies among β-residue contacts are subject to strong regularities, many of which are known a priori. In this article, we take advantage of Markov logic, a statistical relational learning framework that is able to capture dependencies between contacts, and constrain the solution according to domain knowledge expressed by means of weighted rules in a logical language. Results: We introduce a novel hybrid architecture based on neural and Markov logic networks with grounding-specific weights. On a non-redundant dataset, our method achieves 44.9% F1 measure, with 47.3% precision and 42.7% recall, which is significantly better (P < 0.01) than previously reported performance obtained by 2D recursive neural networks. Our approach also significantly improves the number of chains for which β-strands are nearly perfectly paired (36% of the chains are predicted with F1 ≥ 70% on coarse map). It also outperforms more general contact predictors on recent CASP 2008 targets. © The Author 2009. Published by Oxford University Press. All rights reserved
Learning without Local Minima in Radial Basis Function Networks
No full textLearning from examples plays a central role in artificial neural networks. The success of many learning schemes is not guaranteed, however, since algorithms like backpropagation may get stuck in local minima, thus providing suboptimal solutions. For feedforward networks, the theoretical results reported in past literature show that optimal learning can be achieved provided that certain conditions on the network and the learning environment are met. A similar investigation is put forward in this paper for the case of networks using radial basis functions (RBF). The analysis proposed in for multilayer networks is extended naturally under the assumption that the patterns of the learning environment are separable by hyperspheres. In that case, we prove that the attached cost function is local minima free with respect to all the weights. This provides us with some theoretical foundations for a massive application of RBF in pattern recognition
Learning in Multilayered Networks Used as Autoassociators
No full textGradient descent learning algorithms may get stuck in local minima, thus making the learning suboptimal. In this paper, we focus attention on multilayered networks used as autoassociators and show some relationships with classical linear autoassociators. In addition, using the theoretical framework of our previous research, we derive a condition which is met at the end of the learning process and show that this condition has a very intriguing geometrical meaning in the pattern space
Learning Efficiently with Neural Networks: A Theoretical Comparison between Structured and Flat Representations
No full textWe are interested in the relationship between learning efficiency and representation in the case of supervised neural networks for pattern classification trained by continuous error minimization techniques, such as gradient descent. In particular, we focus our attention on a recently introduced architecture called recursive neural network (RNN) which is able to learn class membership of patterns represented as labeled directed ordered acyclic graphs (DOAG). RNNs offer several benefits compared to feedforward and recurrent networks for sequences. However, how RNNs compare to these models in terms of learning efficiency still needs investigation. In this paper we give a theoretical answer by giving a set of results concerning the shape of the error surface and critically discussing the implications of these results on the relative difficulty of learning with different data representations. The message of this paper is that, whenever structured representations are available, they should be preferred to ``flat'' (array based) representations because they are likely to simplify learning in terms of time complexity
Image Document Categorization using Hidden Tree-Markov Models Structured Representations
No full textUnified integration of explicit knowledge and learning by example in recurrent networks
No full textWe propose a novel unified approach for integrating explicit knowledge and learning by example in recurrent networks. The explicit knowledge is represented by automaton rules, which are directly injected into the connections of a network. This can be accomplished by using a technique based on linear programming, instead of learning from random initial weights. Learning is conceived as a refinement process and is mainly responsible for uncertain information management. We present preliminary results for problems of automatic speech recognition
Inductive Inference of Regular Grammars Using Recurrent Networks: A Critical Analysis
No full textMany researchers have recently explored the use of recurrent networks for the inductive inference of regular grammars from positive and negative examples [5, 9, 11] with very promising results. In this paper, we give a set of weight constraints guaranteeing that a recurrent network behave as an automaton and show that the measure of this admissible set decreases progressively as the network dimension increases, thus suggesting that automata behavior becomes more and more unlikely for "large" networks. As a result, problems of inductive inference of regular grammars from "long" strings are likely not to be afforded effectively with "large" networks. We suggest looking for more valuable approaches based on the divide et impera paradigm that allow us to limit the network dimensions [3]. 1 Introduction Recently, many researchers have used recurrent neural networks for performing inductive inference of regular grammars with very promising results [5, 9, 11]
Machine Learning for the Internet Part 1 — Guest Editors’ Editorial
No full textThe World Wide Web has been at the center of a revolution in how algorithms are designed with massive amounts of data in mind. The essence of this revo- lution is conceptually very simple: real-world massive data sets are, more often than not, highly structured and regular. Regularities can be used in two com- plementary ways. First, systematic regularities within massive data sets can be used to craft algorithms that are potentially suboptimal in the worst-case, but highly effective for expected cases. Second, nonsystematic regularities—those that are too subtle to be encoded within an algorithm—can be discovered by automated methods so that the solutions are actually determined by the un- derlying data. In both cases, the existence of enormous problem instances that arise from a highly regular source is key to building more effective methods
- …
