Neural program analyzers are brittle and sensitive to input changes.
problem Reliability of neural program analyzers in software engineering.
method Developing testing techniques for neural program analyzers.
result Simple input perturbations can cause neural program analyzers to make mistakes.
Neural program embedding can be helpful in analyzing large software, a task that is challenging for traditional logic-based program analyses due to their limited scalability. A key focus of recent machine-learning advances in this area is on modeling program semantics instead of just syntax. Unfortunately evaluating su…
Supporting evidence for adaptive feature program across diverse models.
problem Analyzing feature learning in neural networks.
method Over-parameterized sequence models and feature error measure (FEM).
result FEM is decreasing during training of adaptive feature models.
BUSTLE synthesizes programs by learning from intermediate values.
problem Challenges in synthesizing complex programs due to large search space.
method Bottom-up search guided by a neural network trained on input-output examples.
result Bottom-up search with execution of intermediate programs provides valuable semantic information.
This paper develops algorithms for high-dimensional stochastic control problems based on deep learning and dynamic programming. Unlike classical approximate dynamic programming approaches, we first approximate the optimal policy by means of neural networks in the spirit of deep reinforcement learning, and then the valu…
Program synthesis from natural language (NL) is practical for humans and, once technically feasible, would significantly facilitate software development and revolutionize end-user programming. We present SAPS, an end-to-end neural network capable of mapping relatively complex, multi-sentence NL specifications to snippe…
SED integrates synthesis, execution, and debugging for neural program synthesis.
problem Challenges in synthesizing complex programs that match specifications.
method SED combines synthesis, execution, and debugging to improve neural program generation.
result SED reduces error rates and outperforms standard decoding methods.
Paper explores understanding of neural source code embeddings.
problem Lack of understanding of contents and characteristics of code2vec embeddings.
method Small case study using code2vec embeddings to create binary SVM classifiers and compare performance with handcrafted features.
result Code2vec embeddings perform similarly to handcrafted features and have more evenly distributed information gains.
Paper analyzes infinite-width attention layers using Tensor Programs.
problem Capturing the infinite-width limit of attention layers.
method Tensor Programs framework to rigorously identify the limit distribution.
result Derives exact form of infinite-width limit distribution without Gaussian approximations.
Inspector Gadget uses crowdsourcing and data programming to label industrial images efficiently.
problem Securing enough labeled data for machine learning in industrial settings.
method Combines crowdsourcing, data augmentation, and data programming.
result Obtains better performance than other weak-labeling techniques.
Synthesizing programs using example input/outputs is a classic problem in artificial intelligence. We present a method for solving Programming By Example (PBE) problems by using a neural model to guide the search of a constraint logic programming system called miniKanren. Crucially, the neural model uses miniKanren's i…
PLANS synthesizes programs from noisy inputs using neural specs and filtering.
problem Synthesizing robust programs from noisy, raw inputs.
method Hybrid model combining neural extraction and rule-based synthesis with noise filtering.
result State-of-the-art performance in diverse environments with no ground-truth training.
A neural program synthesis method with iterative fix operations.
problem Creating correct programs from input-output examples.
method Combines encoder-decoder synthesis with a differentiable fixer.
result Improves synthesis accuracy by reducing discrepancies between outputs and desired outputs.
funcGNN uses graph neural networks to estimate program similarity efficiently.
problem Estimating accurate program similarity for software engineering tasks.
method funcGNN trains on labeled CFG pairs to predict GED between unseen programs using effective embedding vectors.
result funcGNN achieves lower error rate (0.00194) and is 23 times faster than traditional methods.
Adapting neural networks to guide program optimization for better classifiers.
problem Learning differentiable programs with complex architectures.
method Formulating program optimization as a graph search problem, using neural networks as heuristic relaxations.
result Trained neural networks can guide combinatorial search for programmatic classifiers, improving accuracy and interpretability.
CrossBeam learns to search more efficiently in program synthesis.
problem Efficiently searching through vast program spaces.
method Trains a neural model to guide program synthesis, combining previously explored programs.
result CrossBeam explores much smaller portions of the program space compared to state-of-the-art methods.
Unified platform SOCRATES for neural network analysis.
problem Analyzing neural networks for bugs and fairness.
method Standardized format, assertion language, and multiple analysis algorithms.
result Unified platform for neural network analysis.
Neural network identifies undeclared variables and infers their types.
problem Undeclared variable errors in programs.
method Trained on structural semantic details of AST, identifies and infers types of undeclared variables.
result Correctly identified and inferred types for 80% of programs with undeclared variable errors.
System uses neural networks to prove program equivalence via rewrite rules.
problem Proving equivalence between two dataflow graphs.
method Developed a graph-to-sequence neural network trained on example generation to find semantics-preserving rewrite rules.
result System correctly outputs a rewrite sequence for 96% of program pairs, proving equivalence.
We introduce a method for using deep neural networks to amortize the cost of inference in models from the family induced by universal probabilistic programming languages, establishing a framework that combines the strengths of probabilistic programming and deep learning methods. We call what we do "compilation of infer…
Framework uses neural networks to learn fitness functions for machine programming.
problem Automatic software generation and crafting effective fitness functions.
method Genetic algorithms augmented with neural networks and a search heuristic.
result Framework discovers more correct programs with fewer candidate generations.
PG-IM uses neural-symbolic programs to manipulate images.
problem Creating holistic image representations and manipulations.
method PG-IM detects patterns, induces symbolic programs, and manipulates images using a neural network.
result PG-IM learns from a single image and achieves superior performance.
New approach generates better synthetic data for neural program synthesis.
problem Current approaches to neural program synthesis generalize poorly to real data.
method Adversarial approach to control synthetic data distributions.
result Proposed method outperforms current approaches.
A new algebra for probabilistic programming improves tail behavior accuracy.
problem Inaccurate tail behavior in probabilistic models based on neural networks.
method Developed a three-parameter tail asymptotics algebra based on the generalized Gamma distribution.
result Inference algorithms using the heavy-tailed algebra achieve superior performance.
We present a neurosymbolic framework for the lifelong learning of algorithmic tasks that mix perception and procedural reasoning. Reusing high-level concepts across domains and learning complex procedures are key challenges in lifelong learning. We show that a program synthesis approach that combines gradient descent w…
Real-world problems of operations research are typically high-dimensional and combinatorial. Linear programs are generally used to formulate and efficiently solve these large decision problems. However, in multi-period decision problems, we must often compute expected downstream values corresponding to current decision…
Program synthesis is the task of automatically generating a program consistent with a specification. Recent years have seen proposal of a number of neural approaches for program synthesis, many of which adopt a sequence generation paradigm similar to neural machine translation, in which sequence-to-sequence models are …
Neural networks powered with external memory simulate computer behaviors. These models, which use the memory to store data for a neural controller, can learn algorithms and other complex tasks. In this paper, we introduce a new memory to store weights for the controller, analogous to the stored-program memory in modern…
Improves probabilistic programming by analyzing program structure.
problem Inefficiency and limitations of single inference algorithms in probabilistic programming.
method Three novel techniques: static and dynamic analyses to adapt programs for more efficient inference.
result Improves probabilistic programming by making inference more efficient.
New method for efficient probabilistic inference using masked language modeling.
problem Efficient posterior inference in probabilistic programs with many hyper-parameters.
method Formulate inference as masked language modeling, train a neural network to unmask random values.
result Foundation posterior for zero-shot inference and fine-tuning across a range of programs.
We introduce and analyze a new technique for model reduction for deep neural networks. While large networks are theoretically capable of learning arbitrarily complex models, overfitting and model redundancy negatively affects the prediction accuracy and model variance. Our Net-Trim algorithm prunes (sparsifies) a train…
New approach uses neural networks to learn program structure and parameters.
problem Learning programs and their structure efficiently.
method Free category prior over programs, end-to-end learning of structure and parameters.
result Neural networks can serve as primitives in probabilistic programs.
With the recent success of embeddings in natural language processing, research has been conducted into applying similar methods to code analysis. Most works attempt to process the code directly or use a syntactic tree representation, treating it like sentences written in a natural language. However, none of the existin…
Synthesizes static analysis for probabilistic programs.
problem Optimize learning process, verify models, improve programming interface.
method Organize and analyze static analysis techniques for probabilistic programming.
result Future directions for improvement in statistical machine learning.
In this paper, we analyze dynamic programming as a novel approach to solve the problem of maximizing the profits of a bank. The mathematical model of the problem and the description of a bank's work is described in this paper. The problem is then approached using the method of dynamic programming. Dynamic programming m…
Ansor generates high-performance tensor programs for deep learning.
problem Generating high-performance tensor programs for deep learning on various hardware platforms is challenging.
method Ansor uses a hierarchical representation of the search space, sampling programs, and evolutionary search with a learned cost model to find high-performance programs.
result Ansor improves deep neural network execution performance up to 3.8x on Intel CPU, 2.6x on ARM CPU, and 1.7x on NVIDIA GPU.
Study Gaussian-process limits of neural networks using tensor programs.
problem Understanding the behavior of neural networks as they approach infinite width.
method Quantitative analysis through tensor programs and Wasserstein distance.
result Explicit finite-width error bounds, showing convergence to Gaussian-process limits.
The sequence-to-sequence (seq2seq) model for neural machine translation has significantly improved the accuracy of language translation. There have been new efforts to use this seq2seq model for program language translation or program comparisons. In this work, we present the detailed steps of using a seq2seq model to …
New methodology controls synthetic data bias for neural program synthesis.
problem Deep networks generalize poorly to certain data distributions when trained on synthetic examples.
method Proposes a new methodology to control and evaluate the bias of synthetic data distributions over programs and specifications.
result Training deep networks on controlled synthetic data distributions leads to improved cross-distribution generalization performance.
EvoNUDGE uses graph neural networks to improve genetic programming performance.
problem Efficiency in evolutionary computation for problem solving.
method Graph neural network to elicit additional knowledge from symbolic regression problems.
result EvoNUDGE significantly outperforms conventional and neural genetic programming methods.
Neural model learns efficient, general algorithms for various problems.
problem Learning efficient algorithms that generalize well.
method Carefully designed neural model with imitation and reinforcement learning.
result Neural model outperforms custom solutions on sorting and knapsack problems.
Neural networks solve copositive programs, revealing insights into training problems.
problem Training two-layer vector-output ReLU neural networks.
method Convex analysis and copositive programming.
result Neural networks solve copositive programs, providing insights into training problems.
Graph-based approach repairs programs from diagnostic feedback.
problem Learning to repair programs from limited labeled data and compiler error messages.
method Introduces program-feedback graph and graph neural network for reasoning, and self-supervised learning with unlabeled programs.
result DrRepair significantly outperforms prior work, achieving high repair rates.
Although we have tons of machine learning tools to analyze data, most of them require users have some programming backgrounds. Here we introduce a SaaS application which allows users analyze their data without any coding and even without any knowledge of machine learning. Users can upload, train, predict and download t…
IReEn reveals functionality of black-box agents via iterative neural synthesis.
problem Revealing the functionality of a black-box agent without privileged information.
method Iterative refinement of candidate programs using neural program synthesis.
result The approach finds a functional equivalent program in 78% of cases, outperforming state-of-the-art.
Paper provides unbiased spectral moment estimates from finite data.
problem Challenges in estimating spectral moments from limited data.
method Dynamic programming approach to estimate spectral moments of kernel integral operator.
result Demonstrates consistency with theoretical spectra and practical utility in neural networks.
UM trains a neural network to approximate marginal distributions in probabilistic programs.
problem High computational cost and lack of theoretical guarantees in inference methods for probabilistic programs.
method Combining samples from a probabilistic program prior with an augmentation method to train a neural network for any conditional marginal distribution.
result UM trains a single neural network to approximate any conditional marginal distribution, amortizing inference costs.
Paper presents an ADMM-based approach to efficiently integrate quadratic programming layers into neural networks.
problem Integrating quadratic programs into neural networks for optimization.
method An ADMM-based network layer architecture for solving quadratic programs efficiently.
result The ADMM layer is approximately an order of magnitude faster than existing methods for medium scaled problems.