A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Novel approach trains LLMs for inductive reasoning using probabilistic programs.
The dominant paradigm for relation prediction in knowledge graphs involves learning and operating on latent representations (i.e., embeddings) of entities and relations. However, these embedding-based methods do not explicitly capture the compositional logical rules underlying the knowledge graph, and they are limited …
Improved neural networks for relational reasoning by projecting high-dimensional data to low-dimensional manifolds.
Abstractor enhances Transformers for relational reasoning, improving sample efficiency and performance.
Looped transformers outperform standard transformers in complex reasoning tasks due to a specific loss landscape geometry.
While current deep learning systems excel at tasks such as object classification, language processing, and gameplay, few can construct or modify a complex system such as a tower of blocks. We hypothesize that what these systems lack is a "relational inductive bias": a capacity for reasoning about inter-object relations…
We propose the Neural Logic Machine (NLM), a neural-symbolic architecture for both inductive learning and logic reasoning. NLMs exploit the power of both neural networks---as function approximators, and logic programming---as a symbolic processor for objects with properties, relations, logic connectives, and quantifier…
Causal reasoning has been an indispensable capability for humans and other intelligent animals to interact with the physical world. In this work, we propose to endow an artificial agent with the capability of causal reasoning for completing goal-directed tasks. We develop learning-based approaches to inducing causal kn…
Endowing robots with human-like physical reasoning abilities remains challenging. We argue that existing methods often disregard spatio-temporal relations and by using Graph Neural Networks (GNNs) that incorporate a relational inductive bias, we can shift the learning process towards exploiting relations. In this work,…
In this paper we introduce a new inductive bias for capsule networks and call networks that use this prior -capsule networks. Our inductive bias that is inspired by TE neurons of the inferior temporal cortex increases the adversarial robustness and the explainability of capsule networks. A theoretical framework with…
Graph neural networks improve combinatorial optimization by leveraging inductive bias.
This abstract extends on the previous work (arXiv:1407.2646, arXiv:1606.00075) on program induction using probabilistic programming. It describes possible further steps to extend that work, such that, ultimately, automatic probabilistic program synthesis can generalise over any reasonable set of inputs and outputs, in …
Study shows different trajectory prediction models generalize better under OoD conditions.
We introduce the -simplicial Transformer, an extension of the Transformer which includes a form of higher-dimensional attention generalising the dot-product attention, and uses this attention to update entity representations with tensor products of value vectors. We show that this architecture is a useful inductive …
Discovering causal relationships is a hard task, often hindered by the need for intervention, and often requiring large amounts of data to resolve statistical uncertainty. However, humans quickly arrive at useful causal relationships. One possible reason is that humans extrapolate from past experience to new, unseen si…
Attention mechanisms have been boosting the performance of deep learning models on a wide range of applications, ranging from speech understanding to program induction. However, despite experiments from psychology which suggest that attention plays an essential role in visual reasoning, the full potential of attention …
As an efficient and scalable graph neural network, GraphSAGE has enabled an inductive capability for inferring unseen nodes or graphs by aggregating subsampled local neighborhoods and by learning in a mini-batch gradient descent fashion. The neighborhood sampling used in GraphSAGE is effective in order to improve compu…
Artificial intelligence (AI) has undergone a renaissance recently, making major progress in key domains such as vision, language, control, and decision-making. This has been due, in part, to cheap data and cheap compute resources, which have fit the natural strengths of deep learning. However, many defining characteris…
We introduce an approach for deep reinforcement learning (RL) that improves upon the efficiency, generalization capacity, and interpretability of conventional approaches through structured perception and relational reasoning. It uses self-attention to iteratively reason about the relations between entities in a scene a…
Paper proposes a novel RRL framework that learns from images and incorporates expert knowledge.
Unsupervised learning models can produce accurate but misleading predictions.
Study logical generalization in GNNs using a new benchmark.
Machine learning algorithms often contain many hyperparameters (HPs) whose values affect the predictive performance of the induced models in intricate ways. Due to the high number of possibilities for these HP configurations and their complex interactions, it is common to use optimization techniques to find settings th…
Study on NNs for forecasting time series with novel control variable combinations.
The recent success of natural language understanding (NLU) systems has been troubled by results highlighting the failure of these models to generalize in a systematic and robust way. In this work, we introduce a diagnostic benchmark suite, named CLUTRR, to clarify some key issues related to the robustness and systemati…
This paper explores the limits of Transformers in learning new patterns from scratch.
New approach uses neural networks to learn program structure and parameters.
CSML learns causal structures for few-shot learning.
Paper explores how knowledge distillation transfers inductive biases between models.
Interpolated-MLPs control inductive bias for better performance in low-compute tasks.
New method quantifies inductive bias for machine learning tasks.
The paper presents a method to compute trusted confidence bounds for LECs in CPS.
The paper uses transfinite induction to prove existence in analysis.
We propose a neural network approach to price EU call options that significantly outperforms some existing pricing models and comes with guarantees that its predictions are economically reasonable. To achieve this, we introduce a class of gated neural networks that automatically learn to divide-and-conquer the problem …
One-layer transformers can't solve induction heads task efficiently.
OTI extends OTP for inductive semi-supervised learning.
SpatialSim benchmarks machine learning in recognizing object spatial configurations.
Transformers learn to generalize unseen tasks by composing self-attention layers.
Enhances graph neural networks with structural message-passing for better generalization.
Strong inductive biases prevent harmless interpolation in overparameterized models.
Deep ResNets favor low bottleneck rank with proper hyperparameters.
GQML uses symmetries from representation theory to improve quantum machine learning.
Existing model-based reinforcement learning methods often study perception modeling and decision making separately. We introduce joint Perception and Control as Inference (PCI), a general framework to combine perception and control for partially observable environments through Bayesian inference. Based on the fact that…
Bayesian probability theory is one of the most successful frameworks to model reasoning under uncertainty. Its defining property is the interpretation of probabilities as degrees of belief in propositions about the state of the world relative to an inquiring subject. This essay examines the notion of subjectivity by dr…
This research formalizes inductive generalization and proposes a new learning paradigm called Inductive Learning.
We introduce several methods to define the self-inductance of a single loop as the regularization of divergent integrals which we obtain by applying Neumann (or Weber) formula for the mutual inductance of a pair of loops to the case when two loops are identical.
New method learns disentangled discrete representations using categorical variational autoencoders.
We introduce the notion of large scale inductive dimension for asymptotic resemblance spaces. We prove that the large scale inductive dimension and the asymptotic dimensiongrad are equal in the class of r-convex metric spaces. This class contains the class of all geodesic metric spaces and all finitely generated groups…