Abstract Syntax Networks generate code and parse text with high accuracy.
problem Mapping unstructured inputs to well-formed outputs for code generation and semantic parsing.
method Dynamic AST construction using a decoder with a modular structure.
result 79.2 BLEU and 22.7% exact match accuracy on Hearthstone dataset.
Optimizes tensor program execution time using graph neural networks.
problem Finding optimal configurations for tensor programs is infeasible due to large configuration space.
method Trains a graph convolutional network on an abstract syntax tree to predict execution time.
result Graph-based surrogate model outperforms heuristic-based methods.
Neural network identifies undeclared variables and infers their types.
problem Undeclared variable errors in programs.
method Trained on structural semantic details of AST, identifies and infers types of undeclared variables.
result Correctly identified and inferred types for 80% of programs with undeclared variable errors.
SLM models code syntax as trees to generate any programming language code.
problem Generating any piece of code in a given language without restrictions.
method Structural language modeling (SLM) decomposes code into ASTs and estimates probabilities over nodes.
result SLM model generates arbitrary code in any language, outperforming previous methods.
Syntax-enhanced models boost machine translation and NLP performance.
problem Limited training data and complex models struggle in NLP tasks.
method Syntax information was explicitly fed into Transformer and BERT models.
result Syntax-infused models achieved significant BLEU improvements.
SlicStan improves Stan's usability and efficiency.
problem Stan's block syntax sacrifices usability for scalability.
method Formalized Stan, introduced SlicStan with compositional syntax and flexible functions.
result SlicStan facilitates better code reuse and abstraction.
CODE2SEQ generates natural language sequences from code snippets.
problem Generating natural language descriptions from code.
method CODE2SEQ represents code as AST paths and uses attention to select relevant paths.
result CODE2SEQ outperforms previous models for code-to-text tasks.
Graph2Diff neural network predicts precise code changes for build errors.
problem Fixing build errors in software development.
method Represented code and errors as graphs, used Graph Neural Network to predict precise diffs.
result Graph2Diff achieves over double the accuracy of DeepDelta in predicting precise code changes.
Model ranks abstract anaphors based on their relation to antecedents.
problem Resolving abstract anaphora in text understanding.
method LSTM-Siamese Net learns mention-ranking through artificial data.
result Model outperforms state-of-the-art on shell noun resolution.
Framework for neural network inference on encrypted data.
problem Privacy-preserving machine learning with homomorphic encryption.
method Extensible framework using TensorFlow and SEAL, exposing algebra and deep learning primitives.
result Framework allows training and inference of neural networks on encrypted data.
Model converts code snippets into vectors for predicting method names.
problem Representing code as vectors for semantic analysis.
method Decomposes code into abstract syntax tree paths, learns atomic representations simultaneously with aggregation.
result Code vectors trained on 14M methods can predict method names from unseen files.
A new layer learns abstract relations from graph structure using finite-state automata.
problem Learning abstract relations from graph structure for program analysis.
method Relaxing the problem into learning finite-state automata policies on a graph-based POMDP and training these policies using implicit differentiation.
result GFSA layer finds shortcuts in grid-world graphs and reproduces simple static analyses on Python programs.
Improved source code summarization using extended Tree-LSTM.
problem Challenges in applying LSTM to structured source code.
method Extended Tree-LSTM for abstract syntax trees (ASTs).
result Multi-way Tree-LSTM achieves better results than state-of-the-art techniques.
This paper describes Convex, a convex optimization modeling framework in Julia. Convex translates problems from a user-friendly functional language into an abstract syntax tree describing the problem. This concise representation of the global structure of the problem allows Convex to infer whether the problem complies …
SAPS synthesizes code from natural language specifications.
problem Efficiently translating complex NL specifications into executable code.
method Structure-aware neural network using abstract syntax trees and LSTM.
result SAPS produces correct programs in over 92% of cases.
Unsupervised RNNGs perform similarly to supervised ones in language modeling and grammar induction.
problem Training RNNGs requires annotated data, which is costly.
method Amortized variational inference with a neural CRF parser.
result Unsupervised RNNGs achieve comparable performance to supervised ones.
Metric learning improves tree edit distance for better classification.
problem Improving tree edit distance for better classification accuracy.
method Adaptive symbol embeddings to learn tree edit distance indirectly.
result Embedding edit distance learning (BEDL) improves upon state-of-the-art on multiple tree data sets.
Hybrid approach combines ASTs and deep learning for PowerShell malware detection.
problem Detecting malicious PowerShell scripts effectively.
method Combining static program analysis (ASTs) with deep learning.
result Preliminary results show promising classification of PowerShell scripts by family type.
Syntax designs adaptive trials for subpopulations with potential benefits.
problem Identifying subpopulations with positive treatment effects in diverse patient populations.
method Adaptive patient recruitment and synthetic control estimation.
result Syntax outperforms conventional trial designs in identifying beneficial subpopulations.
A new method embeds tree nodes to vectors for better tree edit distance learning.
problem Learning tree edit distances directly often violates metric axioms and is hard to interpret.
method Adaptive symbol embeddings to learn tree edit distances indirectly.
result Improves tree edit distance learning on multiple datasets.
Intelligence emerges from stabilizing invariant cycles in memory.
problem Understanding the nature of intelligence and its emergence.
method Structural-dynamical account rooted in a topological closure law: \(\partial^2=0\).
result Memory-amortized inference (MAI) mechanism that implements SbS \(
ightarrow\) CCUP.
EgoCoder synthesizes programs from text using neural networks.
problem Automatically generating programs to meet software developer demand.
method Hierarchical sequential neural network model to parse and synthesize programs.
result EgoCoder effectively captures hierarchical and sequential program patterns.
Analyzes attention structure in GPT-2 model, revealing specific patterns.
problem Understanding attention mechanisms in Transformer models.
method Visualized and analyzed attention for GPT-2 model, examining interactions over a corpus.
result Attention targets different parts of speech at varying depths, aligning with dependency relations in middle layers.
SIVAE integrates sentences and their syntactic trees for improved text generation.
problem Improving the grammar of generated text.
method SIVAE uses two separate latent spaces for sentences and syntactic trees, optimizing a joint distribution with two encoders and two decoders.
result SIVAE generates sentences with better grammar compared to existing models.
Deep learning model separates syntax and semantics for better language generalization.
problem Standard deep learning methods struggle with systematic generalization in natural language.
method Implemented a Syntactic Attention model that separates syntactic and semantic processing.
result The Syntactic Attention model outperforms standard methods on a compositional generalization task.
Paper uses weakly-supervised clustering to automatically create network protocol abstractions.
problem Manual definition of abstraction by domain experts is time-consuming.
method Weakly supervised clustering algorithm for automatic abstraction.
result The method successfully matches the reference abstraction with minimal labeled examples.
Machine learning automates proof generation in Coq proof assistant.
problem Automating proof construction in proof assistants.
method Developed ASTactic, a deep learning model generating tactics as ASTs.
result ASTactic can generate effective tactics and prove new theorems.
New system defends deep neural networks like ResNet-34.
problem Training deep neural networks like ResNet-34 is challenging.
method Differentiable abstract interpretation and a DSL for training objectives.
result Can defend significantly larger networks than before.
New approach to abstract neural network representations using renormalization group.
problem Developing truly abstract representations in neural networks.
method Renormalization group approach to expand representations to encompass broader data sets.
result Representations in neural networks become more abstract as data breadth increases and depth increases.
Abstract Neural Networks (ANNs) improve DNN verification efficiency.
problem Efficiently verify safety-critical DNNs without slowing exponentially.
method Introduces ANNs that use abstract domains and activation functions to overapproximate DNNs.
result ANNs can soundly overapproximate DNNs with fewer nodes, improving verification efficiency.
BPE improves text-to-SQL generation by reducing training time and improving accuracy.
problem Improving text-to-SQL generation accuracy with neural models.
method Adapted Byte-Pair Encoding (BPE) for SQL generation, introduced a novel stopping criterion, and used AST BPE.
result Improved accuracy on 5 out of 6 English text-to-SQL tasks, reduced training time by 50%.
Math theory explains how neural networks learn abstract representations.
problem Understanding how neural networks learn abstract representations.
method Mathematical theory reformulating network optimization into mean field optimization over neural preactivations.
result Abstract representations of latent variables are guaranteed to appear in neural networks trained on tasks that depend on these variables.
Text documents are structured on multiple levels of detail: individual words are related by syntax, but larger units of text are related by discourse structure. Existing language models generally fail to account for discourse structure, but it is crucial if we are to have language models that reward coherence and gener…
ARNe model excels in abstract visual reasoning tasks.
problem Abstract visual reasoning using attention mechanisms.
method Hybrid network architecture combining self-attention and relational reasoning.
result ARNe model surpasses WReN model by 11.28 ppt on PGM datasets.
Scalable verifier for recurrent neural networks using polyhedral abstractions.
problem Certifying the correctness of recurrent neural networks.
method Combining sampling, optimization, and Fermat's theorem for polyhedral abstractions; gradient descent for refinement.
result Successfully verified challenging recurrent models in various domains.
Improves neural program synthesis by addressing aliasing and syntax issues.
problem Ignoring program aliasing and syntax in neural program synthesis.
method Reinforcement learning and direct syntax maximization training.
result Improved accuracy, especially with limited training data.
This paper extends ABCD to discover time series structure using probabilistic program synthesis.
problem Discovering structure in time series data.
method Formulating ABCD in probabilistic program synthesis, using abstract syntax trees and probabilistic programs.
result Improved accuracy in time series clustering and interpolation/extrapolation.
Blang simplifies Bayesian analysis for non-standard data types.
problem Bayesian inference for non-standard data structures.
method Bayesian declarative language, distribution continua, sequential Monte Carlo, non-reversible MCMC.
result Bayesian analysis on arbitrary data types is feasible and efficient.
Neural network model improves sentence classification in medical abstracts.
problem Individual sentence classification misses contextual information.
method Combines ANN effectiveness with structured prediction for joint sentence classification.
result Achieves state-of-the-art results on medical abstract datasets.
NHC learns scalable algorithmic solutions from diverse tasks.
problem Neural networks struggle to learn algorithmic strategies.
method Memory-augmented network with abstraction mechanism and evolutionary training.
result Reliable learning of robust and scalable algorithmic solutions.
Abstract graph networks combine modular meta-learning for flexible combinatorial generalization.
problem Modeling flexible generalization to unseen tasks without supervision.
method Abstract graph networks and modular meta-learning.
result Flexible combinatorial generalization achieved.
Proposes deep multimodal fusion for biometric identification.
problem Improving biometric identification accuracy with multiple modalities.
method Joint optimization of multiple modality-specific CNNs at different feature abstraction levels.
result Significant improvement in multimodal person identification performance.
System uses machine learning and automated reasoning to speed up PBE synthesis.
problem Slow synthesis in PBE due to domain-specific knowledge and large training datasets.
method Preprocess SyGuS PBE problems with a neural network to reduce search space, then use automated reasoning for faster solution.
result System outperforms all competing tools in the 2019 SyGuS Competition for the PBE Strings track by 47.65%.
New dataset tests neural networks' abstract reasoning, revealing strengths and weaknesses.
problem Measuring neural networks' ability to perform abstract reasoning.
method Proposed a dataset and challenge inspired by IQ tests, varying training and test data differences.
result Popular models like ResNets perform poorly on subtle differences between training and test data.
Model learns code representations from comments for data analysis tasks.
problem Lack of descriptive labels for analyzing large code corpora.
method Weakly supervised transformer architecture for joint code and comment representation.
result Model achieves 38% accuracy increase over expert-supplied heuristics.
EncGAN learns multi-manifold structure and abstract features using an encoder.
problem Learning multi-manifold structure and abstract features in data.
method Uses an encoder to model manifold structure and invert it for generation, with a single latent space for shared abstract features.
result Successfully learns multi-manifold structure and abstract features on MNIST, 3D-chair, and UT-Zap50k datasets.
Neural networks struggle with abstract patterns, new RBP structures improve performance.
problem Neural networks fail to learn abstract patterns based on identity rules.
method Proposed Relation Based Pattern (RBP) extensions to neural network structures.
result Neural networks with RBP structures achieve perfect performance on synthetic and real-world sequence prediction tasks.
Automated method simplifies stochastic chemical reaction network analysis.
problem Analyzing complex stochastic chemical reaction networks is computationally expensive.
method Uses deep learning to create a discrete-time process from a CTMC, optimizing neural network architecture.
result Automated method improves computational efficiency and accuracy for various CRNs.