A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
A neural program synthesis method with iterative fix operations.
Defines a calculus for integrating Moreau envelopes in differentiable programming.
Adapting neural networks to guide program optimization for better classifiers.
We present DiffTaichi, a new differentiable programming language tailored for building high-performance differentiable physical simulators. Based on an imperative programming language, DiffTaichi generates gradients of simulation steps using source code transformations that preserve arithmetic intensity and parallelism…
Gradient estimation techniques applied to programs with randomness in high energy physics.
Recent work has shown how to embed differentiable optimization problems (that is, problems whose solutions can be backpropagated through) as layers within deep learning architectures. This method provides a useful inductive bias for certain problems, but existing software for differentiable optimization layers is rigid…
Paper presents an ADMM-based approach to efficiently integrate quadratic programming layers into neural networks.
Differentiable programming aids in solving differential equations and their sensitivities.
TomOpt optimizes muon detector designs using differentiable programming.
Extends gradient-based optimization to spline functions.
d3p package enables efficient Bayesian inference with differential privacy.
Differentiable methods fail due to spectral issues in Jacobians.
Hamiltonian Monte Carlo (HMC) is arguably the dominant statistical inference algorithm used in most popular "first-order differentiable" Probabilistic Programming Languages (PPLs). However, the fact that HMC uses derivative information causes complications when the target distribution is non-differentiable with respect…
Simplifies efficient estimation via automatic differentiation and probabilistic programming.
This book is a graduate-level introduction to probabilistic programming. It not only provides a thorough background for anyone wishing to use a probabilistic programming system, but also introduces the techniques needed to design and build these systems. It is aimed at people who have an undergraduate-level understandi…
We review the current state of automatic differentiation (AD) for array programming in machine learning (ML), including the different approaches such as operator overloading (OO) and source transformation (ST) used for AD, graph-based intermediate representations for programs, and source languages. Based on these insig…
Bayesian framework for robust model discovery from noisy data.
Paper introduces DP methods for high-dimensional variable selection.
When samples have internal structure, we often see a mismatch between the objective optimized during training and the model's goal during inference. For example, in sequence-to-sequence modeling we are interested in high-quality translated sentences, but training typically uses maximum likelihood at the word level. The…
Physarum-inspired solver speeds up LP solving in neural networks.
Dynamic programming (DP) solves a variety of structured combinatorial problems by iteratively breaking them down into smaller subproblems. In spite of their versatility, DP algorithms are usually non-differentiable, which hampers their use as a layer in neural networks trained by backpropagation. To address this issue,…
We introduce a novel numerical approach for a class of stochastic dynamic programs which arise as discretizations of backward stochastic differential equations or semi-linear partial differential equations. Solving such dynamic programs numerically requires the approximation of nested conditional expectations, i.e., it…
Data-driven optimization improves mean-variance portfolios by penalizing norms.
Fixed-parameter tractability of private synthetic data generation
A new layer learns abstract relations from graph structure using finite-state automata.
The paper solves optimal control problems for stochastic delay equations.
DILP improves fraud detection explainability without significant performance boost.
Method learns SDEs from data snapshots.
These notes are based on my lectures at IMA summer program "Symmetries and Overdetermined Systems of Partial Differential Equations." Here I try to explain basic ideas of the ambient metric construction by studying the Szego kernel of the sphere.
Differential privacy enables organizations to collect accurate aggregates over sensitive data with strong, rigorous guarantees on individuals' privacy. Previous work has found that under differential privacy, computing multiple correlated aggregates as a batch, using an appropriate \emph{strategy}, may yield higher acc…
Framework learns linear programs from optimal decisions.
Paper proposes a novel RRL framework that learns from images and incorporates expert knowledge.
Probabilistic programs with mixed support (both continuous and discrete latent random variables) commonly appear in many probabilistic programming systems (PPSs). However, the existence of the discrete random variables prohibits many basic gradient-based inference engines, which makes the inference procedure on such mo…
We introduce the C++ library Wedge, based on GiNaC, for symbolic computations in differential geometry. We show how Wedge makes it possible to use the language C++ to perform such computations, and illustrate some advantages of this approach with explicit examples. In particular, we describe a short program to determin…
We present a system for the automatic differentiation of a higher-order functional array-processing language. The core functional language underlying this system simultaneously supports both source-to-source automatic differentiation and global optimizations such as loop transformations. Thanks to this feature, we demo…
Deep learning has seen tremendous success over the past decade in computer vision, machine translation, and gameplay. This success rests in crucial ways on gradient-descent optimization and the ability to learn parameters of a neural network by backpropagating observed errors. However, neural network architectures are …
Neural networks powered with external memory simulate computer behaviors. These models, which use the memory to store data for a neural controller, can learn algorithms and other complex tasks. In this paper, we introduce a new memory to store weights for the controller, analogous to the stored-program memory in modern…
Derivatives, mostly in the form of gradients and Hessians, are ubiquitous in machine learning. Automatic differentiation (AD), also called algorithmic differentiation or simply "autodiff", is a family of techniques similar to but more general than backpropagation for efficiently and accurately evaluating derivatives of…
This paper considers a non-Markov control problem arising in a financial market where asset returns depend on hidden factors. The problem is non-Markov because nonlinear filtering is required to make inference on these factors, and hence the associated dynamic program effectively takes the filtering distribution as one…
New method learns binary decision trees efficiently.
Exact Bayesian inference for discrete models using probability generating functions.
We present a neurosymbolic framework for the lifelong learning of algorithmic tasks that mix perception and procedural reasoning. Reusing high-level concepts across domains and learning complex procedures are key challenges in lifelong learning. We show that a program synthesis approach that combines gradient descent w…
We develop a new Low-level, First-order Probabilistic Programming Language (LF-PPL) suited for models containing a mix of continuous, discrete, and/or piecewise-continuous variables. The key success of this language and its compilation scheme is in its ability to automatically distinguish parameters the density functio…
Unified framework for learning flexible probabilistic programs using DPP and PAC-Bayes bounds.
Develops a mathematical model for automatic differentiation in machine learning.
BPQP improves efficiency of differentiable optimization layers for deep learning.
Automatic differentiation---the mechanical transformation of numeric computer programs to calculate derivatives efficiently and accurately---dates to the origin of the computer age. Reverse mode automatic differentiation both antedates and generalizes the method of backwards propagation of errors used in machine learni…
NumPyro is a lightweight library that provides an alternate NumPy backend to the Pyro probabilistic programming language with the same modeling interface, language primitives and effect handling abstractions. Effect handlers allow Pyro's modeling API to be extended to NumPyro despite its being built atop a fundamentall…