1,721,023 research outputs found
Regularizing deep networks with prior knowledge: A constraint-based approach
Full text linkDeep Learning architectures can develop feature representations and classification models in an integrated way during training. This joint learning process requires large networks with many parameters, and it is successful when a large amount of training data is available. Instead of making the learner develop its entire understanding of the world from scratch from the input examples, the injection of prior knowledge into the learner seems to be a principled way to reduce the amount of require training data, as the learner does not need to induce the rules from the data. This paper presents a general framework to integrate arbitrary prior knowledge into learning. The domain knowledge is provided as a collection of first-order logic (FOL) clauses, where each task to be learned corresponds to a predicate in the knowledge base. The logic statements are translated into a set of differentiable constraints, which can be integrated into the learning process to distill the knowledge into the network, or used during inference to enforce the consistency of the predictions with the prior knowledge. The experimental results have been carried out on multiple image datasets and show that the integration of the prior knowledge boosts the accuracy of several state-of-the-art deep architectures on image classification tasks
Learning in Variable-Dimensional Spaces
No full textThis paper proposes a unified approach to learning in environments in which patterns can be represented in variable-dimension domains, which nicely includes the case in which there are missing features. The proposal is based on the representation of the environment by pointwise constraints that are shown to model naturally pattern relationships that come out in problems of information retrieval, computer vision, and related fields. The given interpretation of learning leads to capturing the truly different aspects of similarity coming from the content at different dimensions and the pattern links. It turns out that functions that process real-valued features and functions that operate on symbolic entities are learned within a unified framework of regularization that can also be expressed using the kernel machines mathematical and algorithmic apparatus. Interestingly, in the extreme cases in which only the content or only the links are available, our theory returns classic kernel machines or graph regularization, respectively. We show experimental results that provide clear evidence of the remarkable improvements that are obtained when both types of similarities are exploited on artificial and real-world benchmarks
Image Document Categorization using Hidden Tree-Markov Models Structured Representations
No full textLearning Similarities for Text Documents using Neural Networks
No full textLearning algorithms for neural networks follow either a supervised or a unsupervised scheme. In the first case a teacher provides a target for each pattern in the learning set, while in the second one no target is provided and the learning process aims at fitting the network to the data distribu- tion. In this paper we propose a learning algorithm which, somehow, lies in between these two schemes. The supervisor does not provide a specific target for each example but he specifies a set of relationships among pairs of input patterns. The neural network is trained to map the examples into points of the output space which meet the topological constraints related to the relationships provided by the supervisor. This algorithm is applied to realize an adaptive dimensionality reduction for the vector space representation of text documents. We present a set of experimental results showing that the algorithm is capable of exploiting the semantic relationships implicitly contained in the user’s feedback
Relational reasoning networks
Full text linkNeural-symbolic methods integrate neural architectures, knowledge representation and reasoning. However, they have struggled with both the intrinsic uncertainty of the observations and scaling to real-world applications. This paper presents Relational Reasoning Networks (R2N), a novel end-to-end model that performs relational reasoning in the latent space of a deep learner architecture, where the representations of constants, ground atoms and their manipulations are learned in an integrated fashion. Unlike flat architectures such as Knowledge Graph Embedders, which can only represent relations between entities, R2Ns define an additional computational structure, accounting for higher-level relations among the ground atoms. The considered relations can be explicitly known, like the ones defined by logic formulas, or defined as unconstrained correlations among groups of ground atoms. R2Ns can be applied to purely symbolic tasks or as a neural-symbolic platform to integrate learning and reasoning in heterogeneous problems with entities represented both symbolically and feature-based. The proposed model overtakes the limitations of previous neural-symbolic methods that have been either limited in terms of scalability or expressivity. The proposed methodology is shown to achieve state-of-the-art results in different experimental settings
- …
