Symbolic knowledge in neural models can inadvertently make them more vulnerable to adversarial attacks.
problem Symbolic knowledge in neural models can make models more susceptible to adversarial attacks.
method Investigated deep probabilistic graphical models that incorporate symbolic knowledge and neural nets.
result Symbolic knowledge can propagate the negative effects of adversarial examples, making models more vulnerable.
Hybrid model learns novel handwritten characters better than neural or symbolic models alone.
problem Generating novel yet structured concepts.
method Neuro-symbolic model combining neural networks and probabilistic programs.
result Hybrid model outperforms alternative models in learning and generalizing novel handwritten characters.
Neural-symbolic model improves link prediction in knowledge graphs.
problem Effective relational learning and reasoning for AI systems.
method Neural-symbolic graph neural network that learns over all paths in knowledge graphs.
result Neural-symbolic model outperforms path-based approaches in link prediction.
Many real-world domains can be expressed as graphs and, more generally, as multi-relational knowledge graphs. Though reasoning and learning with knowledge graphs has traditionally been addressed by symbolic approaches, recent methods in (deep) representation learning has shown promising results for specialized tasks su…
Symbolic regression improved by incorporating prior knowledge.
problem Insufficient guidance from training data alone for model accuracy.
method Multi-objective symbolic regression combining training data and prior constraints.
result Models that fit training data well and comply with prior knowledge.
Reinforcement learning and symbolic planning have both been used to build intelligent autonomous agents. Reinforcement learning relies on learning from interactions with real world, which often requires an unfeasibly large amount of experience. Symbolic planning relies on manually crafted symbolic knowledge, which may …
New neural KB representation speeds up reasoning with large symbolic knowledge bases.
problem Efficiently reasoning with large symbolic knowledge bases.
method Sparse-matrix reified knowledge base, enabling fully differentiable, scalable neural modules.
result Competitive performance on KB completion and semantic parsing benchmarks.
New algorithm improves interpretability in sequence classification.
problem Lack of human-independent interpretability metrics in sequence classification.
method Combines linear classifiers with background knowledge embeddings to create a new feature space.
result Preserves predictive power while delivering more interpretable models.
Enhances neural networks with logical knowledge for better performance.
problem Improving neural network performance with logical knowledge.
method Integrating logical knowledge into neural networks through a new final layer with learnable clause weights.
result KENN outperforms other methods in collective classification tasks with relational data.
Shape-constrained symbolic regression improves model extrapolation with prior knowledge.
problem Improving model extrapolation with prior knowledge in symbolic regression.
method Shape-constrained symbolic regression using evolutionary algorithms with interval arithmetic.
result Models with shape constraints have improved extrapolation but lower accuracy on test sets.
This paper develops a novel methodology for using symbolic knowledge in deep learning. From first principles, we derive a semantic loss function that bridges between neural output vectors and logical constraints. This loss function captures how close the neural network is to satisfying the constraints on its output. An…
We developed a caching method to speed up concept learning in complex knowledge bases.
problem Complex concept learning requires many instance retrieval calls, increasing runtime.
method Semantics-aware caching that links concepts to instances via crisp set operations.
result Our cache reduces concept retrieval and learning runtime by an order of magnitude.
We introduce an approach for imposing physically motivated inductive biases on graph networks to learn interpretable representations and improved zero-shot generalization. Our experiments show that our graph network models, which implement this inductive bias, can learn message representations equivalent to the true fo…
Bayesian symbolic regression uncovers missing physics from data with uncertainty quantification.
problem Incomplete knowledge of physical laws from experimental data.
method Bayesian symbolic regression using Reversible Jump Markov Chain Monte Carlo.
result Uncertainty quantification in recovered model structures.
Develops experimental design for discovering missing physics in bioreactors.
problem Discovering missing physics in incomplete model structures of process systems.
method Combines universal differential equations and symbolic regression with sequential experimental design.
result Successfully recovered true model structure of a bioreactor using machine learning techniques.
New CR hypersurfaces in complex space with specific properties.
problem Constructing CR hypersurfaces with arbitrary nilpotent symbols.
method Introduced a class of CR hypersurfaces with methods applicable to all cases with N>5. result Solved equivalence problem for structures with a single Jordan block symbol.
Human ability at solving complex tasks is helped by priors on object and event semantics of their environment. This paper investigates the use of similar prior knowledge for transfer learning in Reinforcement Learning agents. In particular, the paper proposes to use a first-order-logic language grounded in deep neural …
Survey of neurosymbolic AI methods for reasoning over knowledge graphs.
problem Combining symbolic reasoning with deep learning for graph data.
method Logically-informed embedding, embedding with logical constraints, and rule learning approaches.
result A novel taxonomy for classifying neurosymbolic reasoning methods on knowledge graphs.
The paper improves neural network predictions by integrating process knowledge.
problem Improving neural network predictions for process execution data.
method Integrates background process knowledge into neural networks with attention mechanisms.
result Improves prediction accuracy for process execution data.
Study b-6j symbols linking anti-de Sitter tetrahedra to hyperbolic geometry.
problem Analyzing b-6j symbols for quantum invariants. method Examining asymptotics and analytic extensions of 6j-symbols. result Connection between anti-de Sitter tetrahedra and hyperbolic geometry.
FIGARO generates symbolic music with fine-grained control.
problem Minimal control over generated music sequences.
method Description-to-sequence task, learning conditional distribution of sequences given high-level descriptions.
result State-of-the-art controllable symbolic music generation.
This work identifies and mitigates reasoning shortcuts in Neuro-Symbolic models.
problem Neuro-Symbolic models can achieve high accuracy by using unintended concepts.
method Characterized reasoning shortcuts as unintended optima of the learning objective and identified four key conditions.
result Reasoning shortcuts are difficult to mitigate, casting doubt on NeSy solutions' trustworthiness and interpretability.
In a noisy environment, a lossy speech signal can be automatically restored by a listener if he/she knows the language well. That is, with the built-in knowledge of a "language model", a listener may effectively suppress noise interference and retrieve the target speech signals. Accordingly, we argue that familiarity w…
One of the key challenges in applying reinforcement learning to real-life problems is that the amount of train-and-error required to learn a good policy increases drastically as the task becomes complex. One potential solution to this problem is to combine reinforcement learning with automated symbol planning and utili…
Unified approach to structured prediction combining entropy regularization and neuro-symbolic logic.
problem Structured prediction challenges due to large output spaces and insufficient labeled data.
method Neuro-symbolic entropy regularization loss that restricts entropy regularization to valid structures.
result Models predict more accurately and are more likely to be valid.
Generative Neuro-Symbolic model learns from raw data with rich conceptual representations.
problem Learning rich, general-purpose conceptual representations from raw perceptual inputs.
method Generative Neuro-Symbolic (GNS) model combining symbolic and neural network approaches.
result Model learns from raw data and generalizes to 4 unique tasks.
Paper develops a framework to discover bioprocessing regulatory mechanisms using symbolic and statistical learning.
problem Challenges in modeling complex intracellular regulation, stochastic system behavior, and limited experimental data.
method Symbolic and statistical learning framework based on stochastic differential equations and Bayesian learning.
result Improved sample efficiency and robust model selection compared to state-of-the-art approaches.
Deep generative models such as Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs) have recently been applied to style and domain transfer for images, and in the case of VAEs, music. GAN-based models employing several generators and some form of cycle consistency loss have been among the most suc…
Deep Reinforcement Learning (deep RL) has made several breakthroughs in recent years in applications ranging from complex control tasks in unmanned vehicles to game playing. Despite their success, deep RL still lacks several important capacities of human intelligence, such as transfer learning, abstraction and interpre…
We define families of aperiodic words associated to Lorenz knots that arise naturally as syllable permutations of symbolic words corresponding to torus knots. An algorithm to construct symbolic words of satellite Lorenz knots is defined. We prove, subject to the validity of a previous conjecture, that Lorenz knots code…
EvoNUDGE uses graph neural networks to improve genetic programming performance.
problem Efficiency in evolutionary computation for problem solving.
method Graph neural network to elicit additional knowledge from symbolic regression problems.
result EvoNUDGE significantly outperforms conventional and neural genetic programming methods.
Tensor logic aims to unify AI types with scalable and transparent features.
problem Lack of a unified AI programming language with scalability and transparency.
method Introduces tensor logic, a new AI language based on tensor equations.
result Tensor logic enables key AI forms like transformers, formal reasoning, and graphical models.
PDE-NetGen converts physical equations to neural networks for various scientific problems.
problem Bridging physics and deep learning for efficient neural network architectures.
method Combines symbolic calculus and neural network generation to translate PDEs into NN architectures.
result Generates compact, computationally-efficient physics-informed NN architectures.
Bayesian approach models neurodegenerative diseases without clinical labels.
problem Personalized, predictive modeling of neurodegenerative diseases.
method Probabilistic programmed deep kernel learning combining Gaussian processes and neural networks.
result Surpasses deep learning in accuracy and timeliness of predicting neurodegeneration.
Neuro-symbolic system tackles conversational AI's need for natural, broad-ranging dialogue.
problem Understanding unstated presumptions in conversational AI commands.
method Neuro-symbolic theorem prover for multi-hop reasoning.
result Extracts multi-hop reasoning chains from natural language commands.
VaSST uses soft symbolic trees for probabilistic symbolic regression.
problem Efficiently recover symbolic expressions from noisy data.
method Variational inference with soft symbolic trees.
result Superior performance in structural recovery and predictive accuracy.
Semantic Embeddings are a popular way to represent knowledge in the field of zero-shot learning. We observe their interpretability and discuss their potential utility in a safety-critical context. Concretely, we propose to use them to add introspection and error detection capabilities to neural network classifiers. Fir…
We present efficient differentiable implementations of second-order multi-hop reasoning using a large symbolic knowledge base (KB). We introduce a new operation which can be used to compositionally construct second-order multi-hop templates in a neural model, and evaluate a number of alternative implementations, with d…
Automated digital twin discovery from biological data improves drug discovery and personalized medicine.
problem Developing reliable digital twins from noisy, incomplete biological data.
method Symbolic and sparse regression, Bayesian frameworks, deep learning, and large language models.
result Sparse regression generally outperforms symbolic regression, especially with Bayesian frameworks.
Paper investigates differentiable fuzzy implications and their suitability for learning.
problem Analyzing differentiable fuzzy implications and their suitability for learning.
method Investigates the properties of fuzzy implications in a differentiable setting and introduces a new family of fuzzy implications.
result Various fuzzy implications are unsuitable for differentiable learning, and a new family of fuzzy implications is introduced.
We describe dimensionally constrained symbolic regression which has been developed for mass measurement in certain classes of events in high-energy physics (HEP). With symbolic regression, we can derive equations that are well known in HEP. However, in problems with large number of variables, we find that by constraini…
Paper closes neural-symbolic learning loop with grammar model and back-search algorithm.
problem Slow convergence in neural-symbolic learning due to error propagation issues.
method Introduces grammar model as symbolic prior and back-search algorithm for efficient error propagation.
result Significantly outperforms RL methods in performance, converging speed, and data efficiency.
A-NeSI scales approximate inference for probabilistic neurosymbolic learning.
problem Combining neural networks with symbolic reasoning for scalable inference.
method A-NeSI: a new framework for PNL using neural networks for approximate inference.
result A-NeSI achieves scalable approximate inference without semantic changes.
For an arbitrary Riemannian manifold X and Hermitian vector bundles E and F over X we define the notion of the normal symbol of a pseudodifferential operator P from E to F. The normal symbol of P is a certain smooth function from the cotangent bundle T∗X to the homomorphism bundle Hom(E,F) and dep…
Based on the ideas of Optimal Control, we introduce the new basic characteristic of a bracket generating distribution, the Jacobi symbol. In contrast to the classical Tanaka symbol, the set of Jacobi symbols is discrete and classifiable. We give an explicit and unified algebraic procedure for the construction of the ca…
RODE-Net learns ODEs from data with random parameters using neural networks and GANs.
problem Learning ODEs from data with unknown and random parameters.
method RODE-Net combines symbolic networks and GANs to estimate both the ODE and its parameters.
result RODE-Net can accurately estimate the distribution of model parameters and make reliable predictions.
Neural Assistant integrates knowledge reasoning and dialogue generation in a single model.
problem Challenges in task-oriented dialog systems, including multi-turn language understanding and generation, knowledge retrieval and reasoning, and action prediction.
method Develops a single neural network model that jointly predicts text responses and actions from conversation history and external knowledge.
result The model learns to reason on external knowledge with weak supervision, improving factual accuracy and language generation performance.
Enhances Bayesian learning with rule-based evolutionary techniques.
problem Improving Bayesian inference with expert knowledge and data patterns.
method Combines Bayesian inference with rule-based systems and grammatical evolution.
result Automatically derives rules from data, improving point predictions and uncertainty quantification.