New grammar model learns sentence structure with latent variables.
problem Grammar induction for sentences with complex dependencies.
method Compound probabilistic context-free grammar with latent variables, variational inference.
result Effective unsupervised parsing compared to state-of-the-art methods.
Probabilistic grammars improve equation discovery from data.
problem Discovering scientific laws from data using equations.
method Proposed probabilistic context-free grammars to encode soft constraints and a Monte-Carlo algorithm.
result Probabilistic grammars lead to more efficient equation discovery.
Automates molecule design with simpler SMILES generation and reinforcement learning.
problem Designing molecules with specific chemical properties.
method Combines context-free grammar for SMILES strings and reinforcement learning with a Transformer model.
result Significantly reduces model steps per atom and beats previous baselines.
A new probabilistic LGP method improves symbolic regression performance.
problem Traditional LGP's random search limits its effectiveness.
method Integrates SCFG with LGP, updating grammar based on selected individuals.
result Statistically better results on symbolic regression benchmarks.
A new framework generates large hierarchical search spaces for neural architectures.
problem Discovering neural architectures from simple blocks is hard.
method Context-free grammars for a unified, scalable search space.
result Efficiently searches over complete architectures, outperforming existing methods.
Generative model ensures valid discrete data outputs.
problem Challenges in generative modeling of discrete data like arithmetic expressions and molecular structures.
method Grammar Variational Autoencoder (GVAE) that encodes and decodes to/from parse trees, ensuring valid outputs.
result Generates more coherent latent space with valid discrete outputs.
We propose a new statistical model for computational linguistics. Rather than trying to estimate directly the probability distribution of a random sentence of the language, we define a Markov chain on finite sets of sentences with many finite recurrent communicating classes and define our language model as the invarian…
BOSS optimizes string inputs using string kernels and genetic algorithms.
problem Optimizing string inputs with constraints.
method Bayesian optimization over string kernels and genetic algorithms.
result Significantly improved optimization across various string constraints.
Describes MLC search spaces in MEKA and WEKA software.
problem Understanding MLC algorithms and their transformations into SLC problems.
method Overviewed 26 MLC algorithms and 28 SLC algorithms, proposed a context-free grammar.
result Formal description of MLC search spaces and their transformations.
Generalized Earley parser predicts future events from sequence data.
problem Predicting future events from unsegmented, unlabeled sequence data.
method Integrates grammar parsing and classification for optimal segmentation and labels.
result Significantly outperforms other approaches in future human activity prediction.
A framework infers causal direction from symbolic sequences using compression measures.
problem Inferring causal direction from two observed discrete symbolic sequences.
method Lossless compressors for inferring context-free grammars (CFGs) and quantifying compression extent.
result Grammar inferred from one sequence better compresses the other sequence, indicating causal direction.
AlphaCFG discovers alpha factors using grammar-guided search.
problem Discovering formulaic alpha factors in finance.
method AlphaCFG uses a grammar-based framework to define and discover alpha factors with syntactic and semantic constraints.
result AlphaCFG outperforms state-of-the-art methods in trading profitability and efficiency.
Unified theory for neural scaling laws in hierarchically compositional data.
problem Understanding neural scaling laws in hierarchically compositional data.
method Probabilistic context-free grammars and power-law distributed production rules.
result Unified learning curve behavior for classification and next-token prediction tasks.
Deep models learn to parse complex language structures from local data patterns.
problem Understanding how deep models parse and represent language structures.
method Introduced tunable probabilistic context-free grammars and a learning algorithm inspired by deep networks.
result Data correlations across scales enable hierarchical language representations.
Diffusion models learn hierarchical composition rules from data.
problem How many samples do generative models need to learn hierarchical composition rules?
method Theoretical and empirical investigation of diffusion models on probabilistic context-free grammars.
result Diffusion models learn hierarchical composition rules with sample complexity scaling polynomially with context size.
Paper details Hilbert-curve for high-performance data mining.
problem Efficiently mapping multi-dimensional data to one dimension.
method Defines Hilbert-curve using finite automaton and context-free grammar.
result Cache-oblivious algorithms for matrix operations and clustering.
The recent proliferation of richly structured probabilistic models raises the question of how to automatically determine an appropriate model for a dataset. We investigate this question for a space of matrix decomposition models which can express a variety of widely used models from unsupervised learning. To enable mod…
einspace expands NAS search space to include diverse neural architectures.
problem NAS results are often limited to existing structures; new designs are rare.
method einspace uses a probabilistic context-free grammar to create a versatile search space.
result einspace discovers novel and improved architectures, including convolutions and attention.
Paper tackles natural language generation using GANs, achieving state-of-the-art results.
problem Discrepancy in progress between image and natural language generation using GANs.
method Introduces a simple baseline for generating natural language from noise without gradient estimators.
result Achieves state-of-the-art results on a Chinese poem generation dataset.
Unsupervised algorithm parses CSG images into CFG without pretraining.
problem Sparse reward problem in unsupervised program synthesis for images.
method Grammar-encoded tree LSTM, entropy regularization, sampling without replacement.
result Recover meaningful programs in large search spaces (up to 3.8imes1028). Early stopping improves generalization in overparameterized diffusion models.
problem Understanding and optimizing generalization in overparameterized diffusion models.
method Revisiting diffusion models, showing generalization occurs before memorization, and developing a phase diagram.
result Generalization time scales with dataset size, supporting early-stopping criteria.
NeSS combines neural and symbolic approaches for better compositional generalization.
problem Lack of compositional generalization in deep learning models.
method NeSS uses a neural network to generate traces, executed by a symbolic stack machine with sequence manipulation.
result Achieves 100% generalization performance across multiple domains.
GFlowNet-EM learns complex latent variable models with discrete structures.
problem Challenges in modeling posteriors over discrete compositional latents with expectation-maximization.
method Uses GFlowNets to learn stochastic policies for sampling from complex posterior distributions.
result GFlowNet-EM enables training expressive LVMs with discrete compositional latents.
Empirical study shows overparameterization benefits unsupervised learning of latent variable models.
problem Improving optimization landscape in unsupervised learning with overparameterization.
method Synthetic and semi-synthetic experiments with various models and training algorithms.
result Overparameterization significantly increases the number of ground truth latent variables recovered.
Proves deep networks can learn hierarchical structures efficiently.
problem Understanding how deep networks learn hierarchical structures in data.
method Random Hierarchy Models, gradient-based methods, layerwise training.
result Proves deep networks can efficiently learn hierarchical structures.
AG-RL uses action grammars to improve reinforcement learning efficiency.
problem Improving sample efficiency in reinforcement learning.
method Integrates action grammars into reinforcement learning algorithms to enhance performance.
result Significant improvement in performance across multiple Atari games.
Transformer models perform slower than convolutional networks in learning hierarchical language structures.
problem Understanding how neural networks learn hierarchical language structures.
method Theoretical scaling laws and empirical validation of neural network performance.
result Convolutional networks outperform transformers in learning hierarchical language structures.
Unified recurrent networks reveal differences in complexity levels of grammars.
problem Understanding the complexity and behavior of recurrent networks.
method Connecting recurrent networks with deterministic finite automata and formal grammars.
result Unified recurrent networks improve performance and match grammars from different complexity levels.
New system learns programs from descriptions, outperforming brute-force methods.
problem Learning to write programs from descriptions.
method Intelligent search system using glass-box loss function.
result Significant improvements in accuracy and time compared to brute-force search.
A grammar-driven tree-to-tree model improves program translation accuracy.
problem Improving program translation accuracy between programming languages.
method A grammar-driven tree-to-tree model that exploits known grammar rules of the target language.
result The grammar-based model outperforms state-of-the-art models in program translation accuracy.
Unsupervised RNNGs perform similarly to supervised ones in language modeling and grammar induction.
problem Training RNNGs requires annotated data, which is costly.
method Amortized variational inference with a neural CRF parser.
result Unsupervised RNNGs achieve comparable performance to supervised ones.
Mahé provides hierarchical explanations for complex interactions in machine learning models.
problem Capturing and explaining complex interactions in machine learning models.
method Model-agnostic hierarchical explanations through local interpretation and context-free generalization.
result Improved local interaction interpretations and successful explanation of context-free interactions.
Codebook for Institutional Grammar 2.0 simplifies policy encoding.
problem Facilitating consistent policy encoding for diverse analytical needs.
method Revised Institutional Grammar with multiple levels of expressiveness.
result Enhanced flexibility and specificity in policy encoding.
Efficient molecular optimization using probabilistic hypergraph grammars and reinforcement learning.
problem Optimizing molecular structures for better performance in various benchmarks.
method Inferred a hypergraph replacement grammar from ChEMBL, used conditional priors for policy model, and applied a modified policy gradient algorithm.
result The approach results in a molecular distribution closer to the training set than using equal or unconditional priors.
Proposes a method to train neural networks directly on compressed text data.
problem Training neural networks on compressed text data without decompression.
method Introduces composer modules to encode symbols from grammar compression rules into vector representations.
result Demonstrates that the proposed method can achieve both memory and computational efficiency while maintaining moderate performance.
Improved AutoML performance with a pipeline grammar and pre-trained model.
problem Automatic machine learning by pipeline synthesis.
method Model-based reinforcement learning and pipeline grammar.
result Improved performance on AutoML benchmark datasets.
Net2Vis automates CNN visualization for publications.
problem Lack of consistent visual representations in deep learning papers.
method Proposes a visual grammar and automated system for generating publication-ready CNN visualizations.
result Reduces time and ambiguity in generating network visualizations.
MHG-VAE achieves 100% valid molecules with simpler architecture.
problem Generating valid molecules and evaluating properties efficiently.
method MHG-VAE uses molecular hypergraph grammar to guide a single VAE.
result 100% validity achieved with simpler architecture.
Ensemble method detects time series anomalies without preselecting parameter values.
problem Anomaly detection requires known anomaly length, limiting practicality.
method Ensemble grammar induction for variable-length anomalies.
result Ensemble approach outperforms existing methods with different parameter selections.
A new CNN-based code generator outperforms RNNs by 5%.
problem Capturing long sequences in code generation using RNNs.
method Grammar-based structural CNN with tree-based and pre-order convolution modules.
result Significantly outperforms previous state-of-the-art methods by 5 percentage points.
Method extracts knowledge from LSTM for sequence validation.
problem Validating sequences generated by unknown automata.
method Clustering hidden states to build a corresponding automaton.
result Automaton accurately predicts sequence validity.
TES-AE uses tree grammars to speed up autoencoding for tree data.
problem Challenges in autoencoding tree data due to its non-vectorial and discrete nature.
method TES-AE combines reservoir computing with tree grammars for faster training.
result TES-AE outperforms D-VAE in speed and accuracy for tree data.
Proposes a framework for compositional generalization in language models.
problem Lack of compositional generalization in neural networks compared to humans.
method Introduces Generalized Grammar Rules (GGRs) for transduction tasks, formalizing symmetry-based constraints.
result Framework enables models to generalize compositionally, similar to human learning.
Tree-Transformer improves grammar correction in code and natural language.
problem Grammar correction in tree-structured data.
method Tree-Transformer neural network architecture for tree-structured data.
result Significant improvement in grammar correction accuracy.
Loops in surfaces and chord diagrams are studied with graph factorizations and grammars.
problem Understanding loops in surfaces and their properties.
method Factorization of filoops into spheric and toric sums, and grammars generating chordiagraphs.
result Minimal genus of filoops and stability properties under factorizations.
Spectral regularization simplifies sequence models by focusing on grammatical simplicity.
problem Sequence modeling challenges in learning tasks.
method Introduces spectral regularization based on Hankel matrices and trace norm, addressing bi-infinite matrices with an unbiased estimator.
result Demonstrates spectral regularization's potential benefits on Tomita grammars.
Paper closes neural-symbolic learning loop with grammar model and back-search algorithm.
problem Slow convergence in neural-symbolic learning due to error propagation issues.
method Introduces grammar model as symbolic prior and back-search algorithm for efficient error propagation.
result Significantly outperforms RL methods in performance, converging speed, and data efficiency.
This work verifies recurrent neural networks using DFA and shows some are robust to adversarial samples.
problem Verifying recurrent neural networks for adversarial robustness.
method Metric for string distance, DFA for oracle, Tomita grammars for testing.
result Some recurrent networks are robust to adversarial samples.