A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Local PBO methods improve preferential BO in high-dimensional problems.
Choosing appropriate step sizes is critical for reducing the computational cost of training large-scale neural network models. Mini-batch sub-sampling (MBSS) is often employed for computational tractability. However, MBSS introduces a sampling error, that can manifest as a bias or variance in a line search. This is bec…
Batch Active Learning uses derivative information for Gaussian Process regression.
New method scales Gaussian processes with derivatives using variational inference.
Bayesian optimization sped up with scalable Gaussian processes.
Bayesian optimization improved for nanophotonic device design.
GOALS improves learning rate selection for dynamic MBSS in deep learning.
Bayesian optimization has been successful at global optimization of expensive-to-evaluate multimodal objective functions. However, unlike most optimization methods, Bayesian optimization typically does not use derivative information. In this paper we show how Bayesian optimization can exploit derivative information to …
New method speeds up Bayesian inverse problem solving with neural operators.
Global optimization of expensive functions has important applications in physical and computer experiments. It is a challenging problem to develop efficient optimization scheme, because each function evaluation can be costly and the derivative information of the function is often not available. We propose a novel globa…
Derivative-informed models improve financial surrogates for accurate hedging and risk management.
Bayesian optimization is an approach to optimizing objective functions that take a long time (minutes or hours) to evaluate. It is best-suited for optimization over continuous domains of less than 20 dimensions, and tolerates stochastic noise in function evaluations. It builds a surrogate for the objective and quantifi…
An exciting branch of machine learning research focuses on methods for learning, optimizing, and integrating unknown functions that are difficult or costly to evaluate. A popular Bayesian approach to this problem uses a Gaussian process (GP) to construct a posterior distribution over the function of interest given a se…
In this paper, we consider derivative free optimization problems, where the objective function is smooth but is computed with some amount of noise, the function evaluations are expensive and no derivative information is available. We are motivated by policy optimization problems in reinforcement learning that have rece…
State-of-the-art methods in convex and non-convex optimization employ higher-order derivative information, either implicitly or explicitly. We explore the limitations of higher-order optimization and prove that even for convex optimization, a polynomial dependence on the approximation guarantee and higher-order smoothn…
Study provides bounds for estimating intrinsic dimension using Gaussian kernels.
This paper studies Bayesian ranking and selection (R&S) problems with correlated prior beliefs and continuous domains, i.e. Bayesian optimization (BO). Knowledge gradient methods [Frazier et al., 2008, 2009] have been widely studied for discrete R&S problems, which sample the one-step Bayes-optimal point. When used ove…
We establish that first-order methods avoid saddle points for almost all initializations. Our results apply to a wide variety of first-order methods, including gradient descent, block coordinate descent, mirror descent and variants thereof. The connecting thread is that such algorithms can be studied from a dynamical s…
Proposes a method to improve surrogate models by incorporating sensitivity information.
New bounds estimate learning algorithm performance using prediction information.
Reinforcement Learning (RL) algorithms allow artificial agents to improve their action selections so as to increase rewarding experiences in their environments. Deep Reinforcement Learning algorithms require solving a nonconvex and nonlinear unconstrained optimization problem. Methods for solving the optimization probl…
The quandle homology theory is generalized to the case when the coefficient groups admit the structure of Alexander quandles, by including an action of the infinite cyclic group in the boundary operator. Theories of Alexander extensions of quandles in relation to low dimensional cocycles are developed in parallel to gr…
LazyDINO efficiently solves high-dimensional Bayesian inverse problems with fast and scalable solutions.
Derivative-free method solves stochastic optimization problems with noisy objectives and constraints.
Technical report on f-divergences and f-GAN training properties.
Optimization on manifolds is a class of methods for optimization of an objective function, subject to constraints which are smooth, in the sense that the set of points which satisfy the constraints admits the structure of a differentiable manifold. While many optimization problems are of the described form, technicalit…
We use the Ozsvath-Szabo theory of Floer homology to define an invariant of knot complements in three-manifolds. This invariant takes the form of a filtered chain complex, which we call CF_r. It carries information about the Floer homology of large integral surgeries on the knot. Using the exact triangle, we derive inf…
A companion paper to "On knot Floer homology in branched double covers" applied to braided branched loci. We reprove the main result of that paper concerning alternating branched loci when projected to an annulus, without using Khovanov homology. This provides two advantages: 1) the results hold for integer coefficient…
New algorithms for interpreting complex multivariate functions.
We construct and analyze symmetrized delay correlation matrices for empirical data sets for atmopheric and financial data to derive information about correlation between different entities of the time series over time. The information about correlations is obtained by comparing the results for the eigenvalue distributi…
Hamiltonian Monte Carlo (HMC) is arguably the dominant statistical inference algorithm used in most popular "first-order differentiable" Probabilistic Programming Languages (PPLs). However, the fact that HMC uses derivative information causes complications when the target distribution is non-differentiable with respect…
We develop a framework for warm-starting Bayesian optimization, that reduces the solution time required to solve an optimization problem that is one in a sequence of related problems. This is useful when optimizing the output of a stochastic simulator that fails to provide derivative information, for which Bayesian opt…
Unified framework for information-theoretic bounds on learning algorithms.
New samplers improve MCMC efficiency in high dimensions.
We study the problem of robust subspace recovery (RSR) in the presence of adversarial outliers. That is, we seek a subspace that contains a large portion of a dataset when some fraction of the data points are arbitrarily corrupted. We first examine a theoretical estimator that is intractable to calculate and use it to …
Information-theoretic quantities, such as conditional entropy and mutual information, are critical data summaries for quantifying uncertainty. Current widely used approaches for computing such quantities rely on nearest neighbor methods and exhibit both strong performance and theoretical guarantees in certain simple sc…
Overlapping clusters are common in models of many practical data-segmentation applications. Suppose we are given elements to be clustered into possibly overlapping clusters, and an oracle that can interactively answer queries of the form "do elements and belong to the same cluster?" The goal is to recov…
We propose a novel, theoretically-grounded, acquisition function for Batch Bayesian optimization informed by insights from distributionally ambiguous optimization. Our acquisition function is a lower bound on the well-known Expected Improvement function, which requires evaluation of a Gaussian Expectation over a multiv…
Paper explores Elliptical Wishart distributions in signal processing and machine learning.
Proposes a robust IV estimator using optimal transport for corrupted or adversarial data.
Efficient search methods can outperform random search on challenging tasks.
A new framework generates large hierarchical search spaces for neural architectures.
MICO uses mutual information co-training to improve selective search efficiency.
Neural architecture search methods are able to find high performance deep learning architectures with minimal effort from an expert. However, current systems focus on specific use-cases (e.g. convolutional image classifiers and recurrent language models), making them unsuitable for general use-cases that an expert migh…
A method to reduce memory usage in NAS by pruning the search space.
Paper uses CMAB to improve NAS efficiency and accuracy.
In recent years, \emph{search story}, a combined display with other organic channels, has become a major source of user traffic on platforms such as e-commerce search platforms, news feed platforms and web and image search platforms. The recommended search story guides a user to identify her own preference and personal…
Approaches to learning Bayesian networks from data typically combine a scoring function with a heuristic search procedure. Given a Bayesian network structure, many of the scoring functions derived in the literature return a score for the entire equivalence class to which the structure belongs. When using such a scoring…