Deep learning model separates syntax and semantics for better language generalization.
problem Standard deep learning methods struggle with systematic generalization in natural language.
method Implemented a Syntactic Attention model that separates syntactic and semantic processing.
result The Syntactic Attention model outperforms standard methods on a compositional generalization task.
SlicStan improves Stan's usability and efficiency.
problem Stan's block syntax sacrifices usability for scalability.
method Formalized Stan, introduced SlicStan with compositional syntax and flexible functions.
result SlicStan facilitates better code reuse and abstraction.
Syntax-enhanced models boost machine translation and NLP performance.
problem Limited training data and complex models struggle in NLP tasks.
method Syntax information was explicitly fed into Transformer and BERT models.
result Syntax-infused models achieved significant BLEU improvements.
Syntax designs adaptive trials for subpopulations with potential benefits.
problem Identifying subpopulations with positive treatment effects in diverse patient populations.
method Adaptive patient recruitment and synthetic control estimation.
result Syntax outperforms conventional trial designs in identifying beneficial subpopulations.
Tasks like code generation and semantic parsing require mapping unstructured (or partially structured) inputs to well-formed, executable outputs. We introduce abstract syntax networks, a modeling framework for these problems. The outputs are represented as abstract syntax trees (ASTs) and constructed by a decoder with …
SLM models code syntax as trees to generate any programming language code.
problem Generating any piece of code in a given language without restrictions.
method Structural language modeling (SLM) decomposes code into ASTs and estimates probabilities over nodes.
result SLM model generates arbitrary code in any language, outperforming previous methods.
The ability to generate natural language sequences from source code snippets has a variety of applications such as code summarization, documentation, and retrieval. Sequence-to-sequence (seq2seq) models, adopted from neural machine translation (NMT), have achieved state-of-the-art performance on these tasks by treating…
Neural network identifies undeclared variables and infers their types.
problem Undeclared variable errors in programs.
method Trained on structural semantic details of AST, identifies and infers types of undeclared variables.
result Correctly identified and inferred types for 80% of programs with undeclared variable errors.
Optimizes tensor program execution time using graph neural networks.
problem Finding optimal configurations for tensor programs is infeasible due to large configuration space.
method Trains a graph convolutional network on an abstract syntax tree to predict execution time.
result Graph-based surrogate model outperforms heuristic-based methods.
Analyzes attention structure in GPT-2 model, revealing specific patterns.
problem Understanding attention mechanisms in Transformer models.
method Visualized and analyzed attention for GPT-2 model, examining interactions over a corpus.
result Attention targets different parts of speech at varying depths, aligning with dependency relations in middle layers.
Model learns code representations from comments for data analysis tasks.
problem Lack of descriptive labels for analyzing large code corpora.
method Weakly supervised transformer architecture for joint code and comment representation.
result Model achieves 38% accuracy increase over expert-supplied heuristics.
Unsupervised RNNGs perform similarly to supervised ones in language modeling and grammar induction.
problem Training RNNGs requires annotated data, which is costly.
method Amortized variational inference with a neural CRF parser.
result Unsupervised RNNGs achieve comparable performance to supervised ones.
SIVAE integrates sentences and their syntactic trees for improved text generation.
problem Improving the grammar of generated text.
method SIVAE uses two separate latent spaces for sentences and syntactic trees, optimizing a joint distribution with two encoders and two decoders.
result SIVAE generates sentences with better grammar compared to existing models.
The detection of software vulnerabilities (or vulnerabilities for short) is an important problem that has yet to be tackled, as manifested by the many vulnerabilities reported on a daily basis. This calls for machine learning methods for vulnerability detection. Deep learning is attractive for this purpose because it a…
The objective of this paper is to define an effective strategy for building an ensemble of Genetic Programming (GP) models. Ensemble methods are widely used in machine learning due to their features: they average out biases, they reduce the variance and they usually generalize better than single models. Despite these a…
LLMs encode latent topic distributions, suggesting Bayesian inference.
problem Capturing topic structure from large language models.
method Connecting LLM optimization to implicit Bayesian inference and de Finetti's theorem.
result LLMs recover latent topic distributions, matching LDA-generated topics.
Mining frequent sequential patterns consists in extracting recurrent behaviors, modeled as patterns, in a big sequence dataset. Such patterns inform about which events are frequently observed in sequences, i.e. what does really happen. Sometimes, knowing that some specific event does not happen is more informative than…
Blang simplifies Bayesian analysis for non-standard data types.
problem Bayesian inference for non-standard data structures.
method Bayesian declarative language, distribution continua, sequential Monte Carlo, non-reversible MCMC.
result Bayesian analysis on arbitrary data types is feasible and efficient.
Metric learning has the aim to improve classification accuracy by learning a distance measure which brings data points from the same class closer together and pushes data points from different classes further apart. Recent research has demonstrated that metric learning approaches can also be applied to trees, such as m…
Metric learning has the aim to improve classification accuracy by learning a distance measure which brings data points from the same class closer together and pushes data points from different classes further apart. Recent research has demonstrated that metric learning approaches can also be applied to trees, such as m…
System uses machine learning and automated reasoning to speed up PBE synthesis.
problem Slow synthesis in PBE due to domain-specific knowledge and large training datasets.
method Preprocess SyGuS PBE problems with a neural network to reduce search space, then use automated reasoning for faster solution.
result System outperforms all competing tools in the 2019 SyGuS Competition for the PBE Strings track by 47.65%.
BayesPy is an open-source Python software package for performing variational Bayesian inference. It is based on the variational message passing framework and supports conjugate exponential family models. By removing the tedious task of implementing the variational Bayesian update equations, the user can construct model…
Improved source code summarization using extended Tree-LSTM.
problem Challenges in applying LSTM to structured source code.
method Extended Tree-LSTM for abstract syntax trees (ASTs).
result Multi-way Tree-LSTM achieves better results than state-of-the-art techniques.
Intelligence emerges from stabilizing invariant cycles in memory.
problem Understanding the nature of intelligence and its emergence.
method Structural-dynamical account rooted in a topological closure law: \(\partial^2=0\).
result Memory-amortized inference (MAI) mechanism that implements SbS \(
ightarrow\) CCUP.
ChemBoost predicts protein-ligand binding affinity using SMILES syntax.
problem Predicting high affinity drug-target interactions from sequence similarity alone.
method ChemBoost uses SMILES syntax to represent ligands as documents and proteins as sequences or ligand-centric features. It learns chemical word embeddings and predicts affinities using eXtreme Gradient Boosting.
result ChemBoost outperforms state-of-the-art systems in predicting protein-ligand affinities.
This paper describes Convex, a convex optimization modeling framework in Julia. Convex translates problems from a user-friendly functional language into an abstract syntax tree describing the problem. This concise representation of the global structure of the problem allows Convex to infer whether the problem complies …
This work is an analytical and numerical study of the composition of several fractals into one and of the relation between the composite dimension and the dimensions of the component fractals. In the case of composition of standard IFS with segments of equal size, the composite dimension can be expressed as a function …
Text documents are structured on multiple levels of detail: individual words are related by syntax, but larger units of text are related by discourse structure. Existing language models generally fail to account for discourse structure, but it is crucial if we are to have language models that reward coherence and gener…
New geometric approach for analyzing compositional data like gut microbiomes.
problem Analyzing non-negative compositional data with relative values only.
method Reinterpret compositional data as quotient topology of a sphere, using spherical harmonics and reflection group actions.
result Construction of Reproducing Kernel Hilbert Space (RKHS) for compositional data.
Study on deep neural networks using branching processes and Mehler's formula.
problem Understanding the mathematical role of activation functions in compositional neural networks.
method Connection between compositional kernels and branching processes via Mehler's formula; new random features algorithm.
result Explicit formulas for eigenvalues of compositional kernels quantify complexity.
Uses art composition attributes to guide CycleGAN image translation.
problem Improving image-to-image translation quality.
method Trained ACAN on art composition attributes to influence CycleGAN.
result CycleGAN translations improved with ACAN constraints.
We establish conditions for compositional generalization in machine learning.
problem Achieving compositional generalization in machine learning models.
method We reformulate compositionality as a property of the data-generating process and derive mild conditions on the training distribution and model architecture.
result Our theoretical framework enables compositional generalization under mild conditions.
This paper proves hyperbolicity of virtual knot compositions.
problem Proving hyperbolicity of virtual knot compositions.
method Exploring the composition of hyperbolic virtual knots.
result Strong lower bounds on the volume of compositions.
Develops methods for causal inference in compositional data using instrumental variables.
problem Interpreting summary statistics like diversity indices as causal effects in compositional data.
method Statistical data transformations and regression techniques tailored for compositional data.
result Advantages and limitations of the proposed methods demonstrated on synthetic and real microbiome data.
In classical field theory, the composite fibred manifolds Y -> Z -> X provides the adequate mathematical formulation of gauge models with broken symmetries, e.g., the gauge gravitation theory. This work is devoted to connections on composite fibred manifolds. In particular, we get the horizontal splitting of the vertic…
The p-index improves investment performance for NYSE stocks but not for SSE stocks.
problem Improving investment performance for stocks using the p-index.
method Comparing different p-ratio strategies and empirical efficient frontiers for SSE and NYSE stocks.
result The p-index enhances investment performance for NYSE stocks but not for SSE stocks.
grangersearch tests causal relationships in time series data.
problem Testing causal relationships between multiple time series.
method Exhaustive pairwise search, automatic lag order optimization, tidyverse integration.
result Automated Granger causality testing simplifies causal analysis.
To identify and classify toxic online commentary, the modern tools of data science transform raw text into key features from which either thresholding or learning algorithms can make predictions for monitoring offensive conversations. We systematically evaluate 62 classifiers representing 19 major algorithmic families …
Model predicts composite structures assembly quality with input uncertainty.
problem Accurate prediction of dimensional deviations and residual stress in composite structures assembly.
method Neural Network Gaussian Process considering input uncertainty.
result NNGPIU model outperforms other methods for nonsmooth, nonlinear responses.
New algorithm reduces complexity for optimizing complex machine learning tasks.
problem Optimizing complex machine learning objectives like reinforcement learning and portfolio management.
method Developed SARAH-Compositional algorithm using Stochastic Recursive Gradient Descent.
result Achieved optimal IFO complexity bounds for stochastic compositional optimization.
This paper introduces compositional data analysis for financial ratios, improving industry-level analysis.
problem Statistical issues with standard financial ratios at industry level.
method Compositional data analysis techniques for financial ratios.
result Improved analysis of financial ratios using compositional data methods.
With the celebrated success of deep learning, some attempts to develop effective methods for detecting malicious PowerShell programs employ neural nets in a traditional natural language processing setup while others employ convolutional neural nets to detect obfuscated malicious commands at a character level. While the…
Study challenges neural models in compositional learning tasks.
problem Challenges in neural models for compositional and relational learning.
method Introduced ConceptWorld environment for generating images from compositional concepts, tested various neural architectures.
result Neural models struggle with longer compositional chains and substitutivity tests.
SCL discovers compositional structures in analogical reasoning tasks.
problem Discovering compositional structures in analogical reasoning tasks like Raven's Progressive Matrices.
method Proposes Scattering Compositional Learner (SCL) that composes neural networks in sequence.
result Achieves state-of-the-art performance on RPM datasets with significant improvements.
Composite neural network improves PM2.5 prediction.
problem Improving PM2.5 prediction accuracy.
method Composite neural network framework with pre-trained models.
result Composite neural network outperforms individual models and new components added.
New filters match advanced composition for adaptive privacy, with practical constants.
problem Limitations of existing adaptive composition methods.
method Constructed new filters and odometers that match advanced composition rates, including constants.
result Achieved fully adaptive privacy with practical filters and odometers.
Paper develops momentum schemes with variance reduction for non-convex composition optimization.
problem Lack of convergence guarantee and efficient momentum design in existing algorithms.
method Develops various momentum schemes with SPIDER-based variance reduction.
result Achieves near-optimal sample complexity and linear convergence rate.
Extends knockoff filter for composite null hypotheses in variable selection.
problem Handling composite null hypotheses in variable selection.
method Developed two methods for composite inference with knockoffs: S-OLS and FRPP.
result Proposed heuristic variants of S-OLS outperforming BH procedure for composite nulls.