A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Batch Active Learning uses derivative information for Gaussian Process regression.
New method scales Gaussian processes with derivatives using variational inference.
Bayesian optimization has been successful at global optimization of expensive-to-evaluate multimodal objective functions. However, unlike most optimization methods, Bayesian optimization typically does not use derivative information. In this paper we show how Bayesian optimization can exploit derivative information to …
Bayesian optimization sped up with scalable Gaussian processes.
Bayesian optimization improved for nanophotonic device design.
New method speeds up Bayesian inverse problem solving with neural operators.
Derivative-informed models improve financial surrogates for accurate hedging and risk management.
An exciting branch of machine learning research focuses on methods for learning, optimizing, and integrating unknown functions that are difficult or costly to evaluate. A popular Bayesian approach to this problem uses a Gaussian process (GP) to construct a posterior distribution over the function of interest given a se…
Local PBO methods improve preferential BO in high-dimensional problems.
New bounds estimate learning algorithm performance using prediction information.
State-of-the-art methods in convex and non-convex optimization employ higher-order derivative information, either implicitly or explicitly. We explore the limitations of higher-order optimization and prove that even for convex optimization, a polynomial dependence on the approximation guarantee and higher-order smoothn…
Proposes a method to improve surrogate models by incorporating sensitivity information.
Study provides bounds for estimating intrinsic dimension using Gaussian kernels.
Choosing appropriate step sizes is critical for reducing the computational cost of training large-scale neural network models. Mini-batch sub-sampling (MBSS) is often employed for computational tractability. However, MBSS introduces a sampling error, that can manifest as a bias or variance in a line search. This is bec…
This paper studies Bayesian ranking and selection (R&S) problems with correlated prior beliefs and continuous domains, i.e. Bayesian optimization (BO). Knowledge gradient methods [Frazier et al., 2008, 2009] have been widely studied for discrete R&S problems, which sample the one-step Bayes-optimal point. When used ove…
We establish that first-order methods avoid saddle points for almost all initializations. Our results apply to a wide variety of first-order methods, including gradient descent, block coordinate descent, mirror descent and variants thereof. The connecting thread is that such algorithms can be studied from a dynamical s…
New algorithms for interpreting complex multivariate functions.
Unified framework for information-theoretic bounds on learning algorithms.
Paper explores Elliptical Wishart distributions in signal processing and machine learning.
The quandle homology theory is generalized to the case when the coefficient groups admit the structure of Alexander quandles, by including an action of the infinite cyclic group in the boundary operator. Theories of Alexander extensions of quandles in relation to low dimensional cocycles are developed in parallel to gr…
LazyDINO efficiently solves high-dimensional Bayesian inverse problems with fast and scalable solutions.
Derivative-free method solves stochastic optimization problems with noisy objectives and constraints.
Technical report on f-divergences and f-GAN training properties.
The thesis optimizes quantum state exploration using bandit algorithms.
Optimization on manifolds is a class of methods for optimization of an objective function, subject to constraints which are smooth, in the sense that the set of points which satisfy the constraints admits the structure of a differentiable manifold. While many optimization problems are of the described form, technicalit…
We use the Ozsvath-Szabo theory of Floer homology to define an invariant of knot complements in three-manifolds. This invariant takes the form of a filtered chain complex, which we call CF_r. It carries information about the Floer homology of large integral surgeries on the knot. Using the exact triangle, we derive inf…
Lower bounds on Bayes risk for realizable models derived using information theory.
A companion paper to "On knot Floer homology in branched double covers" applied to braided branched loci. We reprove the main result of that paper concerning alternating branched loci when projected to an annulus, without using Khovanov homology. This provides two advantages: 1) the results hold for integer coefficient…
We construct and analyze symmetrized delay correlation matrices for empirical data sets for atmopheric and financial data to derive information about correlation between different entities of the time series over time. The information about correlations is obtained by comparing the results for the eigenvalue distributi…
In this paper, we consider derivative free optimization problems, where the objective function is smooth but is computed with some amount of noise, the function evaluations are expensive and no derivative information is available. We are motivated by policy optimization problems in reinforcement learning that have rece…
Hamiltonian Monte Carlo (HMC) is arguably the dominant statistical inference algorithm used in most popular "first-order differentiable" Probabilistic Programming Languages (PPLs). However, the fact that HMC uses derivative information causes complications when the target distribution is non-differentiable with respect…
We develop a framework for warm-starting Bayesian optimization, that reduces the solution time required to solve an optimization problem that is one in a sequence of related problems. This is useful when optimizing the output of a stochastic simulator that fails to provide derivative information, for which Bayesian opt…
New samplers improve MCMC efficiency in high dimensions.
Bayesian optimization is an approach to optimizing objective functions that take a long time (minutes or hours) to evaluate. It is best-suited for optimization over continuous domains of less than 20 dimensions, and tolerates stochastic noise in function evaluations. It builds a surrogate for the objective and quantifi…
Extends DML for parametric problems, improving accuracy and efficiency in pricing and calibration.
We study the problem of robust subspace recovery (RSR) in the presence of adversarial outliers. That is, we seek a subspace that contains a large portion of a dataset when some fraction of the data points are arbitrarily corrupted. We first examine a theoretical estimator that is intractable to calculate and use it to …
Information-theoretic quantities, such as conditional entropy and mutual information, are critical data summaries for quantifying uncertainty. Current widely used approaches for computing such quantities rely on nearest neighbor methods and exhibit both strong performance and theoretical guarantees in certain simple sc…
The paper addresses bias in survival analysis due to informative censoring.
Overlapping clusters are common in models of many practical data-segmentation applications. Suppose we are given elements to be clustered into possibly overlapping clusters, and an oracle that can interactively answer queries of the form "do elements and belong to the same cluster?" The goal is to recov…
We propose a novel, theoretically-grounded, acquisition function for Batch Bayesian optimization informed by insights from distributionally ambiguous optimization. Our acquisition function is a lower bound on the well-known Expected Improvement function, which requires evaluation of a Gaussian Expectation over a multiv…
Proposes a robust IV estimator using optimal transport for corrupted or adversarial data.
GOALS improves learning rate selection for dynamic MBSS in deep learning.
ISALT uses inference to simulate SDEs with large time-steps, improving efficiency.
We present the first public release of our generic neural network training algorithm, called SkyNet. This efficient and robust machine learning tool is able to train large and deep feed-forward neural networks, including autoencoders, for use in a wide range of supervised and unsupervised learning applications, such as…
A new method clusters intersecting lines using hypergraphs.
Reinforcement Learning (RL) algorithms allow artificial agents to improve their action selections so as to increase rewarding experiences in their environments. Deep Reinforcement Learning algorithms require solving a nonconvex and nonlinear unconstrained optimization problem. Methods for solving the optimization probl…
Study on statistical estimation over Gaussian MAC, comparing analog and digital schemes.
Learning parameters from voluminous data can be prohibitive in terms of memory and computational requirements. We propose a "compressive learning" framework where we estimate model parameters from a sketch of the training data. This sketch is a collection of generalized moments of the underlying probability distributio…