arXiv research

Transformer models can solve complex math problems with less data.

NeSS combines neural and symbolic approaches for better compositional generalization.

CSML learns causal structures for few-shot learning.

How can neural networks perform so well on compositional tasks even though they lack explicit compositional representations? We use a novel analysis technique called ROLE to show that recurrent neural networks perform well on such tasks by converging to solutions which implicitly represent symbolic structure. This meth…

Improves neural network search in combinatorial spaces of mathematical symbols.

Many real-world domains can be expressed as graphs and, more generally, as multi-relational knowledge graphs. Though reasoning and learning with knowledge graphs has traditionally been addressed by symbolic approaches, recent methods in (deep) representation learning has shown promising results for specialized tasks su…

Reinforcement learning and symbolic planning have both been used to build intelligent autonomous agents. Reinforcement learning relies on learning from interactions with real world, which often requires an unfeasibly large amount of experience. Symbolic planning relies on manually crafted symbolic knowledge, which may …

Neuro-symbolic agent learns systematic generalisation from formal instructions.

Extraction of symbolic information from signals is an active field of research enabling numerous applications especially in the Musical Information Retrieval domain. This complex task, that is also related to other topics such as pitch extraction or instrument recognition, is a demanding subject that gave birth to nume…

Generative Neuro-Symbolic model learns from raw data with rich conceptual representations.

Paper closes neural-symbolic learning loop with grammar model and back-search algorithm.

We present the perceptor gradients algorithm -- a novel approach to learning symbolic representations based on the idea of decomposing an agent's policy into i) a perceptor network extracting symbols from raw observation data and ii) a task encoding program which maps the input symbols to output actions. We show that t…

S2KAN integrates symbolic primitives into neural network activations for improved interpretability.

Successful human-robot cooperation hinges on each agent's ability to process and exchange information about the shared environment and the task at hand. Human communication is primarily based on symbolic abstractions of object properties, rather than precise quantitative measures. A comprehensive robotic framework thus…

A new method for spotting symbols in CAD images reduces annotation costs and improves accuracy.

ISR creates analytical relationships from data via invertible maps.

Efficiently searches through Gaussian process kernels using symbolic representation and Bayesian optimization.

We propose the Neural Logic Machine (NLM), a neural-symbolic architecture for both inductive learning and logic reasoning. NLMs exploit the power of both neural networks---as function approximators, and logic programming---as a symbolic processor for objects with properties, relations, logic connectives, and quantifier…

PySR method automates discovering equations from data in chaotic dynamics and epidemics.

Study shows how transformers classify symbols without naming them, proving a margin-versus-collision criterion.

This work analyzes tree-based methods from a ranking perspective, providing insights and new statistics.

Symbolic regression is a powerful technique that can discover analytical equations that describe data, which can lead to explainable models and generalizability outside of the training data set. In contrast, neural networks have achieved amazing levels of accuracy on image recognition and natural language processing ta…

Discovering the underlying mathematical expressions describing a dataset is a core challenge for artificial intelligence. This is the problem of $\textit{symbolic regression}$. Despite recent advances in training neural networks to solve complex tasks, deep learning approaches to symbolic regression are underexplored. …

DEFINED improves wireless symbol detection with limited pilot data.

QABBA improves time series storage efficiency while preserving shape information.

FIGARO generates symbolic music with fine-grained control.

New algorithm improves interpretability in sequence classification.

Transformers learn multi-step reasoning through gradient descent.

CHOOSE enhances shallow Transformers for wireless symbol detection.

Achieving machine intelligence requires a smooth integration of perception and reasoning, yet models developed to date tend to specialize in one or the other; sophisticated manipulation of symbols acquired from rich perceptual spaces has so far proved elusive. Consider a visual arithmetic task, where the goal is to car…

AI agents beat previous best on NetHack, but symbolic bots still outperform.

DEFINED uses decision feedback ICL to detect symbols with minimal pilot data.

Unified framework for hybrid learning and optimization via active inference.

Neural Logic Reasoning integrates deep learning and symbolic logic for better prediction tasks.

Recurrent Neural Networks (RNN) are a type of statistical model designed to handle sequential data. The model reads a sequence one symbol at a time. Each symbol is processed based on information collected from the previous symbols. With existing RNN architectures, each symbol is processed using only information from th…

Relational data mining is becoming ubiquitous in many fields of study. It offers insights into behaviour of complex, real-world systems which cannot be modeled directly using propositional learning. We propose Symbolic Graph Embedding (SGE), an algorithm aimed to learn symbolic node representations. Built on the ideas …

Neural programming involves training neural networks to learn programs, mathematics, or logic from data. Previous works have failed to achieve good generalization performance, especially on problems and programs with high complexity or on large domains. This is because they mostly rely either on black-box function eval…

A framework isolates VQA reasoning from perception for better model evaluation.

VaSST uses soft symbolic trees for probabilistic symbolic regression.

Deep Reinforcement Learning (deep RL) has made several breakthroughs in recent years in applications ranging from complex control tasks in unmanned vehicles to game playing. Despite their success, deep RL still lacks several important capacities of human intelligence, such as transfer learning, abstraction and interpre…

We propose a generalization of neural network sequence models. Instead of predicting one symbol at a time, our multi-scale model makes predictions over multiple, potentially overlapping multi-symbol tokens. A variation of the byte-pair encoding (BPE) compression algorithm is used to learn the dictionary of tokens that …

Study on multitask learning performance factors.

Research on style transfer and domain translation has clearly demonstrated the ability of deep learning-based algorithms to manipulate images in terms of artistic style. More recently, several attempts have been made to extend such approaches to music (both symbolic and audio) in order to enable transforming musical st…

SPPL simplifies probabilistic programming for exact inference.

Transformer learns to infer partial MDPs for efficient in-context adaptation and exploration.

We describe dimensionally constrained symbolic regression which has been developed for mass measurement in certain classes of events in high-energy physics (HEP). With symbolic regression, we can derive equations that are well known in HEP. However, in problems with large number of variables, we find that by constraini…

Neural networks using numerous text data have been successfully applied to a variety of tasks. While massive text data is usually compressed using techniques such as grammar compression, almost all of the previous machine learning methods assume already decompressed sequence data as their input. In this paper, we propo…

For an arbitrary Riemannian manifold $X$ and Hermitian vector bundles $E$ and $F$ over $X$ we define the notion of the normal symbol of a pseudodifferential operator $P$ from $E$ to $F$. The normal symbol of $P$ is a certain smooth function from the cotangent bundle $T^*X$ to the homomorphism bundle $Hom (E,F)$ and dep…