Holographic Invariant Storage uses vector architectures to ensure LLM safety at design time.
problem Mitigating context drift in large language models (LLMs) during deployment.
method Introduces Holographic Invariant Storage (HIS) protocol that combines known properties of bipolar Vector Symbolic Architectures into a design-time safety contract.
result Closed-form guarantees for single-signal recovery fidelity, continuous-noise robustness, and multi-signal capacity degradation are provided and validated.
Explores tensor products in hyperdimensional computing.
problem Understanding tensor products in hyperdimensional computing.
method Generalized results from graph embeddings to vector symbolic architectures and hyperdimensional computing.
result Tensor product is the most general and expressive representation with errorless unbinding and detection.
Paper presents a novel neural network for MIMO symbol detection.
problem Handling a variable number of users in MIMO systems.
method Recurrent and permutation equivariant neural network architecture with iterative decoding.
result The neural detector outperforms existing methods in accuracy and efficiency.
ISR creates analytical relationships from data via invertible maps.
problem Creating analytical relationships from datasets.
method Combines INNs and EQL, using invertible maps and sparsity promoting regularization.
result ISR can serve as a normalizing flow for density estimation and solve inverse problems.
New characterization of vector bundles using Lie algebras of symbols.
problem Characterizing vector bundles using algebraic structures.
method Analyzing Lie algebras of symbols of linear operators on vector bundles.
result Improved Lie algebraic characterization of vector bundles.
The paper classifies symbols of differential operators on vector bundles.
problem Classifying symbols of linear differential operators on vector bundles.
method Associated tuples of linear operators to non-degenerate symbols and used C. Procesi's results to find rational invariants and equivalence criteria.
result Generators for rational invariants and a criterion for symbol equivalence.
In this paper, we propose a new Recurrent Neural Network (RNN) architecture. The novelty is simple: We use diagonal recurrent matrices instead of full. This results in better test likelihood and faster convergence compared to regular full RNNs in most of our experiments. We show the benefits of using diagonal recurrent…
SATNet solves the Symbol Grounding Problem, enabling self-supervised learning.
problem Mapping visual inputs to symbolic variables without explicit supervision.
method Self-supervised pre-training pipeline and proofreading method.
result SATNet achieves full accuracy with no label leakage, surpassing state-of-the-art.
SE-RRMs solve structured problems like Sudoku and ARC-AGI by enforcing symbol equivariance.
problem Structured reasoning problems like Sudoku and ARC-AGI.
method Symbol-equivariant recurrent reasoning models enforcing permutation equivariance.
result SE-RRMs outperform prior RRMs on 9x9 Sudoku and generalize to larger and smaller instances.
Arguments in favor of injecting symbolic knowledge into neural architectures abound. When done right, constraining a sub-symbolic model can substantially improve its performance and sample complexity and prevent it from predicting invalid configurations. Focusing on deep probabilistic (logical) graphical models -- i.e.…
Symbolic regression is a powerful technique that can discover analytical equations that describe data, which can lead to explainable models and generalizability outside of the training data set. In contrast, neural networks have achieved amazing levels of accuracy on image recognition and natural language processing ta…
How can neural networks perform so well on compositional tasks even though they lack explicit compositional representations? We use a novel analysis technique called ROLE to show that recurrent neural networks perform well on such tasks by converging to solutions which implicitly represent symbolic structure. This meth…
The paper characterizes vector bundles and differential operators using Lie algebras and their symbols.
problem Characterizing vector bundles and differential operators using algebraic methods.
method Lie-algebraic characterization of vector bundles and differential operators.
result The Lie algebras P(E,M) and S(P(E,M)) characterize vector bundles and their smooth sections. In order to build efficient deep recurrent neural architectures, it is essential to analyze the complexityof long distance dependencies (LDDs) of the dataset being modeled. In this paper, we presentdetailed analysis of the dependency decay curve exhibited by various datasets. The datasets sampledfrom a similar process …
The paper extends Lie bracket to noncommutative geometry using differential operators.
problem Generalizing Lie bracket to noncommutative geometry.
method Antisymmetrizing compositions of vector fields and treating symbols of differential operators.
result Provided necessary and sufficient conditions for jet modules to represent differential operators.
Recurrent Neural Networks (RNN) are a type of statistical model designed to handle sequential data. The model reads a sequence one symbol at a time. Each symbol is processed based on information collected from the previous symbols. With existing RNN architectures, each symbol is processed using only information from th…
Develops global pseudo-differential calculus on homogeneous vector bundles.
problem Global theory of subelliptic pseudo-differential operators on homogeneous vector bundles.
method Global symbolic calculus, complex functional calculus, Hörmander system of vector-fields.
result Global pseudo-differential calculus on homogeneous vector bundles.
Conventional embedding methods directly associate each symbol with a continuous embedding vector, which is equivalent to applying a linear transformation based on a "one-hot" encoding of the discrete symbols. Despite its simplicity, such approach yields the number of parameters that grows linearly with the vocabulary s…
R2T hybrid model improves robust regression for asymmetric noise.
problem Least-squares regression fails with asymmetric structured noise.
method Transformer encoder, compression NN, fixed symbolic equation.
result Median regression MSE of 6e-6 to 3.5e-5 on synthetic data.
For an arbitrary Riemannian manifold X and Hermitian vector bundles E and F over X we define the notion of the normal symbol of a pseudodifferential operator P from E to F. The normal symbol of P is a certain smooth function from the cotangent bundle T∗X to the homomorphism bundle Hom(E,F) and dep…
A new method for spotting symbols in CAD images reduces annotation costs and improves accuracy.
problem Challenging task of labeling symbols from CAD drawings.
method Pixel-wise point location via Progressive Gaussian Kernels (PGK) and local offset.
result The proposed method achieves good generalization on real-world CAD images.
Proves a Gel'fand-Kolmogoroff type result for vector bundles and polynomial functions.
problem Characterizing vector bundles and polynomial functions using differential operators.
method Analyzes the associative structure of symbols of differential operators and their eigenvectors.
result Derives a Gel'fand-Kolmogoroff type result for the algebra of symbols of differential operators.
Constructs graded jet bundles for Z-graded manifolds and vector bundles.
problem Generalizing jet manifolds to Z-graded structures for differential equations.
method Directly constructs the sheaf of sections of the k-th order jet bundle of a Z-graded vector bundle.
result Establishes a graded version of Atiyah Lie algebroid.
Embedding methods such as word embedding have become pillars for many applications containing discrete structures. Conventional embedding methods directly associate each symbol with a continuous embedding vector, which is equivalent to applying linear transformation based on "one-hot" encoding of the discrete symbols. …
Neuro-symbolic agent learns systematic generalisation from formal instructions.
problem Achieving zero-shot generalisation of formally specified tasks.
method Combines deep reinforcement learning with temporal logic.
result Systematic learning emerges with convolutional layers and abstract operators.
The space of m-ary differential operators acting on weighted densities is a (m+1)-parameter family of modules over the Lie algebra of vector fields. For almost all the parameters, we construct a canonical isomorphism between this space and the corresponding space of symbols as sl(2)-modules. This yields to the notion o…
We classify nontrivial deformations of the standard embedding of the Lie algebra $\Vect(S^1)$ of smooth vector fields on the circle, into the Lie algebra~$\PD(S^1)$ of pseudodifferential symbols on S1. This approach leads to deformations of the central charge induced on $\Vect(S^1)$ by the canonical central extensio…
Hybrid model learns novel handwritten characters better than neural or symbolic models alone.
problem Generating novel yet structured concepts.
method Neuro-symbolic model combining neural networks and probabilistic programs.
result Hybrid model outperforms alternative models in learning and generalizing novel handwritten characters.
Extends elliptic operator regularity to maximally hypoelliptic operators.
problem Maximally hypoelliptic differential operators and their regularity.
method Define a principal symbol for arbitrary differential operators involving vector fields and their commutators.
result Proves the invertibility of the principal symbol is equivalent to maximally hypoellipticity, answering a conjecture.
Many real-world domains can be expressed as graphs and, more generally, as multi-relational knowledge graphs. Though reasoning and learning with knowledge graphs has traditionally been addressed by symbolic approaches, recent methods in (deep) representation learning has shown promising results for specialized tasks su…
Based on the ideas of Optimal Control, we introduce the new basic characteristic of a bracket generating distribution, the Jacobi symbol. In contrast to the classical Tanaka symbol, the set of Jacobi symbols is discrete and classifiable. We give an explicit and unified algebraic procedure for the construction of the ca…
Characterizes polyhomogeneous symbols and applies to Heisenberg calculus.
problem Understanding polyhomogeneous symbols and their applications.
method Simple characterisation and generalization of A.~Connes' tangent groupoid.
result Heisenberg calculus on contact manifolds coincides with groupoid calculus.
Symbolic grounding in causal dynamics achieves near-infinite temporal consistency.
problem Achieving linear identifiability in non-Gaussian physical systems.
method Physics-Grounded Symbolic Architecture (PGSA)
result PGSA achieves exact linear identifiability for all physical regimes.
We develop theoretical foundations of Resonator Networks, a new type of recurrent neural network introduced in Frady et al. (2020) to solve a high-dimensional vector factorization problem arising in Vector Symbolic Architectures. Given a composite vector formed by the Hadamard product between a discrete set of high-dim…
Deep neural network generates symbolic equations from data.
problem Lack of insight into underlying mappings from traditional deep learning.
method Combines deep learning flexibility with symbolic solutions.
result Accurately generates governing equations for dynamical systems.
CHOOSE enhances shallow Transformers for wireless symbol detection.
problem Improving wireless symbol detection with shallow Transformers.
method Introducing autoregressive latent reasoning steps within hidden space.
result Lightweight Transformers achieve comparable performance to deep models.
We prove the existence and uniqueness of a *projectively equivariant symbol map*, which is an isomorphism between the space of bidifferential operators acting on tensor densities over Rn and that of their symbols, when both are considered as modules over an imbedding of sl(n+1,R) into polynomial vector fields. Th…
Bridging physics and deep learning is a topical challenge. While deep learning frameworks open avenues in physical science, the design of physically-consistent deep neural network architectures is an open issue. In the spirit of physics-informed NNs, PDE-NetGen package provides new means to automatically translate phys…
In this paper, we consider the problem of event classification with multi-variate time series data consisting of heterogeneous (continuous and categorical) variables. The complex temporal dependencies between the variables combined with sparsity of the data makes the event classification problem particularly challengin…
We consider differential operators between sections of arbitrary powers of the determinant line bundle over a contact manifold. We extend the standard notions of the Heisenberg calculus: noncommutative symbolic calculus, the principal symbol, and the contact order to such differential operators. Our first main result i…
Cosmos models scenes using neural encodings and symbolic attributes for compositional generalization.
problem Modeling scenes with high performance on unseen input scenes composed of known visual elements.
method Neurosymbolic grounding with neurosymbolic scene encodings and attention mechanisms.
result Establishes a new state-of-the-art for compositional generalization in world modeling.
Representation learning has become an invaluable approach for learning from symbolic data such as text and graphs. However, while complex symbolic datasets often exhibit a latent hierarchical structure, state-of-the-art methods typically learn embeddings in Euclidean vector spaces, which do not account for this propert…
Unified tensor network formalism for combining neural and symbolic AI.
problem Combining neural and symbolic AI approaches remains a challenge.
method Introduces a tensor network formalism capturing sparsity principles.
result Unified treatment identifies tensor network contractions as a fundamental inference class.
Achieving machine intelligence requires a smooth integration of perception and reasoning, yet models developed to date tend to specialize in one or the other; sophisticated manipulation of symbols acquired from rich perceptual spaces has so far proved elusive. Consider a visual arithmetic task, where the goal is to car…
With a view to bridging the gap between deep learning and symbolic AI, we present a novel end-to-end neural network architecture that learns to form propositional representations with an explicitly relational structure from raw pixel data. In order to evaluate and analyse the architecture, we introduce a family of simp…
A key feature of intelligent behavior is the ability to learn abstract strategies that transfer to unfamiliar problems. Therefore, we present a novel architecture, based on memory-augmented networks, that is inspired by the von Neumann and Harvard architectures of modern computers. This architecture enables the learnin…
This paper develops a novel methodology for using symbolic knowledge in deep learning. From first principles, we derive a semantic loss function that bridges between neural output vectors and logical constraints. This loss function captures how close the neural network is to satisfying the constraints on its output. An…
A new algorithm speeds up sparse regression for discovering equations from data.
problem Learning governing equations from vast data with unsatisfying descriptions.
method SPRINT: a fast algorithm using bisection and analytic bounds to identify optimal rank-1 modifications.
result A calculation that would take millions of years can be done in a day.