New model combines stats and grammar, proving key property.
problem Creating a new model for linguistics and beyond.
method Introducing Markov substitute processes and proving their exponential family property.
result Markov substitute processes with a given support form an exponential family.
Unified recurrent networks reveal differences in complexity levels of grammars.
problem Understanding the complexity and behavior of recurrent networks.
method Connecting recurrent networks with deterministic finite automata and formal grammars.
result Unified recurrent networks improve performance and match grammars from different complexity levels.
New grammar model learns sentence structure with latent variables.
problem Grammar induction for sentences with complex dependencies.
method Compound probabilistic context-free grammar with latent variables, variational inference.
result Effective unsupervised parsing compared to state-of-the-art methods.
Proposes a framework for compositional generalization in language models.
problem Lack of compositional generalization in neural networks compared to humans.
method Introduces Generalized Grammar Rules (GGRs) for transduction tasks, formalizing symmetry-based constraints.
result Framework enables models to generalize compositionally, similar to human learning.
Capsule networks improve at detecting changes in compositionality with routing.
problem Capsule networks struggle with detecting changes in compositionality.
method Introduced a loss function based on routing entropy to improve compositionality.
result Capsule networks with the new loss function better detect changes in compositionality.
Probabilistic grammars improve equation discovery from data.
problem Discovering scientific laws from data using equations.
method Proposed probabilistic context-free grammars to encode soft constraints and a Monte-Carlo algorithm.
result Probabilistic grammars lead to more efficient equation discovery.
Interdisciplinary comparison of sequence modeling methods for next-element prediction.
problem Comparing sequence modeling methods across different fields.
method Experimental evaluation of four real-life sequence datasets using machine learning, process mining, and grammar inference techniques.
result Machine learning techniques outperform interpretability-focused methods in next-element prediction accuracy.
Describes MLC search spaces in MEKA and WEKA software.
problem Understanding MLC algorithms and their transformations into SLC problems.
method Overviewed 26 MLC algorithms and 28 SLC algorithms, proposed a context-free grammar.
result Formal description of MLC search spaces and their transformations.
AG-RL uses action grammars to improve reinforcement learning efficiency.
problem Improving sample efficiency in reinforcement learning.
method Integrates action grammars into reinforcement learning algorithms to enhance performance.
result Significant improvement in performance across multiple Atari games.
Deep learning is currently the subject of intensive study. However, fundamental concepts such as representations are not formally defined -- researchers "know them when they see them" -- and there is no common language for describing and analyzing algorithms. This essay proposes an abstract framework that identifies th…
A grammar-driven tree-to-tree model improves program translation accuracy.
problem Improving program translation accuracy between programming languages.
method A grammar-driven tree-to-tree model that exploits known grammar rules of the target language.
result The grammar-based model outperforms state-of-the-art models in program translation accuracy.
Unsupervised RNNGs perform similarly to supervised ones in language modeling and grammar induction.
problem Training RNNGs requires annotated data, which is costly.
method Amortized variational inference with a neural CRF parser.
result Unsupervised RNNGs achieve comparable performance to supervised ones.
AlphaCFG discovers alpha factors using grammar-guided search.
problem Discovering formulaic alpha factors in finance.
method AlphaCFG uses a grammar-based framework to define and discover alpha factors with syntactic and semantic constraints.
result AlphaCFG outperforms state-of-the-art methods in trading profitability and efficiency.
We propose a new statistical model for computational linguistics. Rather than trying to estimate directly the probability distribution of a random sentence of the language, we define a Markov chain on finite sets of sentences with many finite recurrent communicating classes and define our language model as the invarian…
Codebook for Institutional Grammar 2.0 simplifies policy encoding.
problem Facilitating consistent policy encoding for diverse analytical needs.
method Revised Institutional Grammar with multiple levels of expressiveness.
result Enhanced flexibility and specificity in policy encoding.
Efficient molecular optimization using probabilistic hypergraph grammars and reinforcement learning.
problem Optimizing molecular structures for better performance in various benchmarks.
method Inferred a hypergraph replacement grammar from ChEMBL, used conditional priors for policy model, and applied a modified policy gradient algorithm.
result The approach results in a molecular distribution closer to the training set than using equal or unconditional priors.
Generalized Earley parser predicts future events from sequence data.
problem Predicting future events from unsegmented, unlabeled sequence data.
method Integrates grammar parsing and classification for optimal segmentation and labels.
result Significantly outperforms other approaches in future human activity prediction.
Graphs improve theorem proving in higher-order logic.
problem Challenges in converting higher-order logic formulas into graph-based representations.
method Used graph neural networks (GNNs) to represent and search higher-order logic.
result GNNs outperform state-of-the-art methods in higher-order theorem proving.
Proposes a method to train neural networks directly on compressed text data.
problem Training neural networks on compressed text data without decompression.
method Introduces composer modules to encode symbols from grammar compression rules into vector representations.
result Demonstrates that the proposed method can achieve both memory and computational efficiency while maintaining moderate performance.
Improved AutoML performance with a pipeline grammar and pre-trained model.
problem Automatic machine learning by pipeline synthesis.
method Model-based reinforcement learning and pipeline grammar.
result Improved performance on AutoML benchmark datasets.
Net2Vis automates CNN visualization for publications.
problem Lack of consistent visual representations in deep learning papers.
method Proposes a visual grammar and automated system for generating publication-ready CNN visualizations.
result Reduces time and ambiguity in generating network visualizations.
Automates molecule design with simpler SMILES generation and reinforcement learning.
problem Designing molecules with specific chemical properties.
method Combines context-free grammar for SMILES strings and reinforcement learning with a Transformer model.
result Significantly reduces model steps per atom and beats previous baselines.
MHG-VAE achieves 100% valid molecules with simpler architecture.
problem Generating valid molecules and evaluating properties efficiently.
method MHG-VAE uses molecular hypergraph grammar to guide a single VAE.
result 100% validity achieved with simpler architecture.
Generative model ensures valid discrete data outputs.
problem Challenges in generative modeling of discrete data like arithmetic expressions and molecular structures.
method Grammar Variational Autoencoder (GVAE) that encodes and decodes to/from parse trees, ensuring valid outputs.
result Generates more coherent latent space with valid discrete outputs.
A new probabilistic LGP method improves symbolic regression performance.
problem Traditional LGP's random search limits its effectiveness.
method Integrates SCFG with LGP, updating grammar based on selected individuals.
result Statistically better results on symbolic regression benchmarks.
Ensemble method detects time series anomalies without preselecting parameter values.
problem Anomaly detection requires known anomaly length, limiting practicality.
method Ensemble grammar induction for variable-length anomalies.
result Ensemble approach outperforms existing methods with different parameter selections.
A new CNN-based code generator outperforms RNNs by 5%.
problem Capturing long sequences in code generation using RNNs.
method Grammar-based structural CNN with tree-based and pre-order convolution modules.
result Significantly outperforms previous state-of-the-art methods by 5 percentage points.
Method extracts knowledge from LSTM for sequence validation.
problem Validating sequences generated by unknown automata.
method Clustering hidden states to build a corresponding automaton.
result Automaton accurately predicts sequence validity.
TES-AE uses tree grammars to speed up autoencoding for tree data.
problem Challenges in autoencoding tree data due to its non-vectorial and discrete nature.
method TES-AE combines reservoir computing with tree grammars for faster training.
result TES-AE outperforms D-VAE in speed and accuracy for tree data.
Tree-Transformer improves grammar correction in code and natural language.
problem Grammar correction in tree-structured data.
method Tree-Transformer neural network architecture for tree-structured data.
result Significant improvement in grammar correction accuracy.
Loops in surfaces and chord diagrams are studied with graph factorizations and grammars.
problem Understanding loops in surfaces and their properties.
method Factorization of filoops into spheric and toric sums, and grammars generating chordiagraphs.
result Minimal genus of filoops and stability properties under factorizations.
Spectral regularization simplifies sequence models by focusing on grammatical simplicity.
problem Sequence modeling challenges in learning tasks.
method Introduces spectral regularization based on Hankel matrices and trace norm, addressing bi-infinite matrices with an unbiased estimator.
result Demonstrates spectral regularization's potential benefits on Tomita grammars.
Paper closes neural-symbolic learning loop with grammar model and back-search algorithm.
problem Slow convergence in neural-symbolic learning due to error propagation issues.
method Introduces grammar model as symbolic prior and back-search algorithm for efficient error propagation.
result Significantly outperforms RL methods in performance, converging speed, and data efficiency.
A framework infers causal direction from symbolic sequences using compression measures.
problem Inferring causal direction from two observed discrete symbolic sequences.
method Lossless compressors for inferring context-free grammars (CFGs) and quantifying compression extent.
result Grammar inferred from one sequence better compresses the other sequence, indicating causal direction.
This work verifies recurrent neural networks using DFA and shows some are robust to adversarial samples.
problem Verifying recurrent neural networks for adversarial robustness.
method Metric for string distance, DFA for oracle, Tomita grammars for testing.
result Some recurrent networks are robust to adversarial samples.
Interactive EMA combines multiple explainability methods to improve model understanding.
problem Isolated explanations of machine learning models lead to misunderstandings and wrong reasoning.
method Interactive EMA (IEMA) combines multiple explainability methods sequentially.
result Interactive EMA increases the performance and confidence of human decision making.
The recent proliferation of richly structured probabilistic models raises the question of how to automatically determine an appropriate model for a dataset. We investigate this question for a space of matrix decomposition models which can express a variety of widely used models from unsupervised learning. To enable mod…
A new framework generates large hierarchical search spaces for neural architectures.
problem Discovering neural architectures from simple blocks is hard.
method Context-free grammars for a unified, scalable search space.
result Efficiently searches over complete architectures, outperforming existing methods.
This work decouples language from math problems to enable cross-language learning.
problem Current machine learning representations are language dependent.
method Inspired by linguistics, the work learns language agnostic representations.
result Models trained on one language achieve similar accuracies in other languages.
Generative Adversarial Network repairs software bugs without labeled data.
problem Automated repair of software vulnerabilities.
method Adversarial learning approach mapping between source and target domains.
result Effective at repairing software vulnerabilities, close to seq2seq approaches.
Transformers can be hard to interpret due to complex optima.
problem Interpretability methods fail to capture the full complexity of Transformer models.
method Theoretical analysis and synthetic experiments on Dyck language learning.
result Attention patterns can be nearly randomized while preserving network functionality.
Unsupervised algorithm parses CSG images into CFG without pretraining.
problem Sparse reward problem in unsupervised program synthesis for images.
method Grammar-encoded tree LSTM, entropy regularization, sampling without replacement.
result Recover meaningful programs in large search spaces (up to 3.8imes1028). BOSS optimizes string inputs using string kernels and genetic algorithms.
problem Optimizing string inputs with constraints.
method Bayesian optimization over string kernels and genetic algorithms.
result Significantly improved optimization across various string constraints.
Unified model improves coherence tasks, especially local contexts.
problem Existing neural coherence models struggle with local context tasks.
method Unified neural framework integrating grammar, relations, and patterns.
result Unified model outperforms existing models significantly.
Study shows mutual information can reward structure learning agents without expert systems.
problem Designing rewards for structure learning agents in natural language environments.
method Revisited Information Theory of unsupervised induction of phrase-structure grammars, using random sets of linguistic samples.
result Empirical evidence that simulated semantic structures can be distinguished from random ones by mutual information among their constituents.
Diffusion models learn hierarchical composition rules from data.
problem How many samples do generative models need to learn hierarchical composition rules?
method Theoretical and empirical investigation of diffusion models on probabilistic context-free grammars.
result Diffusion models learn hierarchical composition rules with sample complexity scaling polynomially with context size.
Systematic approach to generating diverse 3D scenes and images for training and benchmarking.
problem Training and benchmarking machine learning algorithms for scene understanding.
method Stochastic grammar-based pipeline for generating and rendering photorealistic 3D scenes with detailed ground truth.
result Improves performance in scene understanding tasks and provides controllable benchmarks for model diagnostics.
Unified theory for neural scaling laws in hierarchically compositional data.
problem Understanding neural scaling laws in hierarchically compositional data.
method Probabilistic context-free grammars and power-law distributed production rules.
result Unified learning curve behavior for classification and next-token prediction tasks.