This paper automates tagging programming challenge descriptions.
problem Tagging programming challenge descriptions is tedious for creators.
method Used machine and deep learning methods for automation.
result Deep learning methods outperform traditional IR approaches.
SED integrates synthesis, execution, and debugging for neural program synthesis.
problem Challenges in synthesizing complex programs that match specifications.
method SED combines synthesis, execution, and debugging to improve neural program generation.
result SED reduces error rates and outperforms standard decoding methods.
Graph-based approach repairs programs from diagnostic feedback.
problem Learning to repair programs from limited labeled data and compiler error messages.
method Introduces program-feedback graph and graph neural network for reasoning, and self-supervised learning with unlabeled programs.
result DrRepair significantly outperforms prior work, achieving high repair rates.
Neural program embedding can be helpful in analyzing large software, a task that is challenging for traditional logic-based program analyses due to their limited scalability. A key focus of recent machine-learning advances in this area is on modeling program semantics instead of just syntax. Unfortunately evaluating su…
Framework learns linear programs from optimal decisions.
problem Learning linear programs from optimal decisions is challenging.
method Gradient-based framework for learning linear programs from optimal decisions.
result Successfully learns linear programs and multi-commodity flow instances.
We organized a competition on Autonomous Lifelong Machine Learning with Drift that was part of the competition program of NeurIPS 2018. This data driven competition asked participants to develop computer programs capable of solving supervised learning problems where the i.i.d. assumption did not hold. Large data sets w…
We present a neurosymbolic framework for the lifelong learning of algorithmic tasks that mix perception and procedural reasoning. Reusing high-level concepts across domains and learning complex procedures are key challenges in lifelong learning. We show that a program synthesis approach that combines gradient descent w…
Deep neural networks have been increasingly used in software engineering and program analysis tasks. They usually take a program and make some predictions about it, e.g., bug prediction. We call these models neural program analyzers. The reliability of neural programs can impact the reliability of the encompassing anal…
The task of translating between programming languages differs from the challenge of translating natural languages in that programming languages are designed with a far more rigid set of structural and grammatical rules. Previous work has used a tree-to-tree encoder/decoder model to take advantage of the inherent tree s…
Seq2Seq models perform well in generating If-Then programs from natural language.
problem Creating If-Then programs for business process automation without technical expertise.
method Modeling If-Then programs as a sequence learning task using Seq2Seq approaches.
result Seq2Seq models can effectively generate If-Then programs from natural language.
Challenge evaluates semantic code search using annotated corpus.
problem Evaluating relevant code from natural language queries.
method Release of CodeSearchNet Corpus and expert annotations.
result 99 queries with 4k relevance annotations for evaluation.
The problem of automatic software generation is known as Machine Programming. In this work, we propose a framework based on genetic algorithms to solve this problem. Although genetic algorithms have been used successfully for many problems, one criticism is that hand-crafting its fitness function, the test that aims to…
A new method for efficient inference in probabilistic programs with mixed support.
problem Challenges in inference for programs with both continuous and discrete latent variables.
method Stochastic gradient Markov Chain Monte Carlo algorithms.
result Outperforms existing composing inference baselines and works almost as well as inference in marginalized versions.
Improved inference in probabilistic programs using attention mechanisms.
problem Inference failure in existing IC network architectures due to long-range dependency issues.
method Inference compilation with attention mechanism to model latent variables.
result Attention mechanism enhances proposal distributions to better match true posterior.
Supporting evidence for adaptive feature program across diverse models.
problem Analyzing feature learning in neural networks.
method Over-parameterized sequence models and feature error measure (FEM).
result FEM is decreasing during training of adaptive feature models.
Ansor generates high-performance tensor programs for deep learning.
problem Generating high-performance tensor programs for deep learning on various hardware platforms is challenging.
method Ansor uses a hierarchical representation of the search space, sampling programs, and evolutionary search with a learned cost model to find high-performance programs.
result Ansor improves deep neural network execution performance up to 3.8x on Intel CPU, 2.6x on ARM CPU, and 1.7x on NVIDIA GPU.
NP-HMC extends HMC for nonparametric models in probabilistic programming.
problem Inference for nonparametric models in probabilistic programming.
method Introduces NP-HMC, a generalization of HMC for nonparametric models using tree representable functions.
result Empirically shows significant performance improvements over existing approaches.
BUSTLE synthesizes programs by learning from intermediate values.
problem Challenges in synthesizing complex programs due to large search space.
method Bottom-up search guided by a neural network trained on input-output examples.
result Bottom-up search with execution of intermediate programs provides valuable semantic information.
PLANS synthesizes programs from noisy inputs using neural specs and filtering.
problem Synthesizing robust programs from noisy, raw inputs.
method Hybrid model combining neural extraction and rule-based synthesis with noise filtering.
result State-of-the-art performance in diverse environments with no ground-truth training.
Program synthesis struggles with complex spatial relationships in image classification.
problem Challenges in solving Synthetic Visual Reasoning Test problems.
method Quantitative reanalysis of human and machine performance, improved program synthesis classifier, categorization of SVRT problems.
result Program synthesis is constrained by spatial relationships in images, not just shape specification.
Program synthesis of general-purpose source code from natural language specifications is challenging due to the need to reason about high-level patterns in the target program and low-level implementation details at the same time. In this work, we present PATOIS, a system that allows a neural program synthesizer to expl…
Real-world problems of operations research are typically high-dimensional and combinatorial. Linear programs are generally used to formulate and efficiently solve these large decision problems. However, in multi-period decision problems, we must often compute expected downstream values corresponding to current decision…
Paper introduces a novel method to generate diverse inputs for neural programming by example.
problem Synthesizing programs from input/output pairs using machine learning.
method Uses an SMT solver to generate diverse input-output pairs.
result Generated inputs improve model performance and generalization.
New method uses dynamic programming for meta continual learning.
problem Challenges of generalization and catastrophic forgetting in sequential learning.
method Developed a theoretical framework using dynamic programming for meta continual learning.
result Theoretical and practical method achieves better accuracy than existing methods.
Proposes a method for model-based RL in complex environments without perfect simulators.
problem Lack of cheap and perfect simulators in real-world tasks.
method Induces a world program by learning dynamics and actions in graph-based environments.
result World program enables complex planning tasks in environments without perfect simulators.
A new method streamlines digital payment programming using smart contracts.
problem High costs and security challenges in programming smart contracts for digital payments.
method Transforming digital currencies into token streams and using configurable templates to generate specialized smart contracts.
result Reduces payment programming costs and enhances security, self-enforcement, adaptability, and controllability.
Determining the programming language of a source code file has been considered in the research community; it has been shown that Machine Learning (ML) and Natural Language Processing (NLP) algorithms can be effective in identifying the programming language of source code files. However, determining the programming lang…
New method solves nonseparable stochastic control problems.
problem Nonseparable and non-monotonic stochastic control problems.
method Scenario-decomposition solution framework using progressive hedging algorithm.
result Extends reach of stochastic optimal control.
Functional tensors unify probabilistic programming with automatic differentiation.
problem Designing probabilistic programming systems that can handle diverse inference strategies.
method Introducing functional tensors that capture benefits of tensors and continuous probability distributions.
result Functional tensors enable parallel exact inference for various modeling motifs.
NeurIPS 2019 program improves reproducibility in machine learning.
problem Ensuring machine learning research results are reproducible and reliable.
method Code submission policy, reproducibility challenge, and checklist integration.
result Improved reproducibility standards across the machine learning community.
IReEn reveals functionality of black-box agents via iterative neural synthesis.
problem Revealing the functionality of a black-box agent without privileged information.
method Iterative refinement of candidate programs using neural program synthesis.
result The approach finds a functional equivalent program in 78% of cases, outperforming state-of-the-art.
CLN2INV learns precise loop invariants from program traces.
problem Automated verification of real-world programs with complex loops.
method Continuous Logic Network (CLN) for learning precise loop invariants from program execution traces.
result CLN2INV significantly outperforms existing approaches on the Code2Inv dataset.
Datasets with hundreds of variables and many missing values are commonplace. In this setting, it is both statistically and computationally challenging to detect true predictive relationships between variables and also to suppress false positives. This paper proposes an approach that combines probabilistic programming, …
This paper reviews recent advances in the field of optimization under uncertainty via a modern data lens, highlights key research challenges and promise of data-driven optimization that organically integrates machine learning and mathematical programming for decision-making under uncertainty, and identifies potential r…
What is a systematic way to efficiently apply a wide spectrum of advanced ML programs to industrial scale problems, using Big Models (up to 100s of billions of parameters) on Big Data (up to terabytes or petabytes)? Modern parallelization strategies employ fine-grained operations and scheduling beyond the classic bulk-…
Paper presents an ADMM-based approach to efficiently integrate quadratic programming layers into neural networks.
problem Integrating quadratic programs into neural networks for optimization.
method An ADMM-based network layer architecture for solving quadratic programs efficiently.
result The ADMM layer is approximately an order of magnitude faster than existing methods for medium scaled problems.
Optimizes risk assessment tools using mixed-integer programming.
problem Challenges in healthcare risk assessment due to label scarcity and asymmetric misclassification costs.
method Jointly optimizes scoring weights and category thresholds via mixed-integer programming (MIP).
result Prevents label-scarce category collapse and achieves more accurate risk categorization.
This paper evaluates how well neural models can solve complex tasks by breaking them into simpler ones.
problem Measuring neural models' ability to solve complex tasks by breaking them into simpler subtasks.
method Characterized axes of compositional generalization, introduced a benchmark suite of tasks, and improved Transformer models' attention mechanisms.
result Modified Transformer models generally perform better than natural baselines in solving complex tasks, but challenges remain.
We outline a new approach for solving optimization problems which enforce triangle inequalities on output variables. We refer to this as metric-constrained optimization, and give several examples where problems of this form arise in machine learning applications and theoretical approximation algorithms for graph cluste…
Transformers improve solving mixed-integer programs, especially CLSP.
problem Solving Capacitated Lot Sizing Problem (CLSP) with mixed-integer programming.
method Employing transformer models to predict binary variables in CLSP.
result Transformer model outperforms CPLEX and LSTM in solving CLSP.
New algorithm solves complex stopping problems with robust optimization.
problem Solving complex stochastic optimal stopping problems.
method Simulation-based robust optimization with exact reformulation as a zero-one bilinear program.
result Developed polynomial-time heuristics and algorithms for practical solution.
We consider a class of linear-programming based estimators in reconstructing a sparse signal from linear measurements. Specific formulations of the reconstruction problem considered here include Dantzig selector, basis pursuit (for the case in which the measurements contain no errors), and the fused Dantzig selector (f…
Proposes PKM for soft K-means clustering.
problem Soft K-means (m=1) unsolved since 1981.
method Probabilistic K-Means (PKM) via nonlinear programming.
result Proposed methods solve PKM efficiently.
A new method uses GANs for robust optimization under uncertain data.
problem Optimizing supply chains under demand uncertainty with ambiguous distributions.
method Generative adversarial networks (GANs) for data-driven distributionally robust chance constrained programming.
result The approach effectively handles uncertain data distributions and improves supply chain optimization.
Study uses Perelman and Ricci flow methods to analyze economic inequality.
problem Impact of socio-economic challenges and technological progress on economic inequality.
method Perelman model and Ricci flow methods.
result Technological innovations and social protection programs reduce inequality.
Novel approach trains LLMs for inductive reasoning using probabilistic programs.
problem Training LLMs for inductive reasoning with sparse, ambiguous data.
method Program-based Posterior Training (PPT) using probabilistic inference.
result Significant improvement in estimation accuracy and alignment with human judgments.
Large labeled training sets are the critical building blocks of supervised learning methods and are key enablers of deep learning techniques. For some applications, creating labeled training sets is the most time-consuming and expensive part of applying machine learning. We therefore propose a paradigm for the programm…
Deep learning solves dynamic programming with recursive utility.
problem Challenges in solving high-dimensional discrete-time dynamic programming problems with recursive utility.
method Certainty Equivalent Learning (CEL) algorithm that learns certainty-equivalent value directly with neural networks.
result Accurate value and policy approximations in high-dimensional problems, comparable to VFI in some cases.