arXiv research

Study noise-free kernel bandits, finding upper bounds on regret.

Random exploration optimizes Bayesian optimization with optimal error rates and computational efficiency.

Algorithm optimizes cascaded functions with known structure.

A corrected EI acquisition function handles noisy observations in Bayesian optimization.

MSLG generates soft labels to improve DNN performance on noisy datasets.

FEDMD-NFDP improves federated learning privacy without sacrificing performance.

In this paper, we study a classification problem in which sample labels are randomly corrupted. In this scenario, there is an unobservable sample with noise-free labels. However, before being observed, the true labels are independently flipped with a probability $ρ\in[0,0.5)$, and the random label noise can be class-co…

Study learns linear utility functions from comparisons, showing learnability gaps between passive and active learning.

We introduce Bayesian optimization, a technique developed for optimizing time-consuming engineering simulations and for fitting machine learning models on large datasets. Bayesian optimization guides the choice of experiments during materials design and discovery to find good material designs in as few experiments as p…

We consider the robust phase retrieval problem of recovering the unknown signal from the magnitude-only measurements, where the measurements can be contaminated by both sparse arbitrary corruption and bounded random noise. We propose a new nonconvex algorithm for robust phase retrieval, namely Robust Wirtinger Flow to …

In this work, we investigate black-box optimization from the perspective of frequentist kernel methods. We propose a novel batch optimization algorithm, which jointly maximizes the acquisition function and select points from a whole batch in a holistic way. Theoretically, we derive regret bounds for both the noise-free…

This article considers algorithmic and statistical aspects of linear regression when the correspondence between the covariates and the responses is unknown. First, a fully polynomial-time approximation scheme is given for the natural least squares optimization problem in any constant dimension. Next, in an average-case…

Improves state space models' resistance to noise.

Paper analyzes convergence rate of noisy Bayesian Optimization with Expected Improvement.

Pre-conditioning is a well-known concept that can significantly improve the convergence of optimization algorithms. For noise-free problems, where good pre-conditioners are not known a priori, iterative linear algebra methods offer one way to efficiently construct them. For the stochastic optimization problems that dom…

Enhanced Gaussian process models accelerate optimization and posterior approximation.

We develop model free PAC performance guarantees for multiple concurrent MDPs, extending recent works where a single learner interacts with multiple non-interacting agents in a noise free environment. Our framework allows noisy and resource limited communication between agents, and develops novel PAC guarantees in this…

Bayesian optimization is an approach to optimizing objective functions that take a long time (minutes or hours) to evaluate. It is best-suited for optimization over continuous domains of less than 20 dimensions, and tolerates stochastic noise in function evaluations. It builds a surrogate for the objective and quantifi…

New algorithm tackles high-dimensional simulation optimization, converging efficiently.

This paper develops nudging algorithms using learned surrogates for state estimation in dynamical systems.

New method uses SLL to create masks for PX in noisy optimization problems.

Proposes a graph-based method for active learning in regression.

Noise-free sampling method using Wasserstein proximal for faster convergence.

Proves SQ lower bounds for learning two-hidden-layer neural networks.

Noise-resilient method improves Hurst exponent estimation accuracy in noisy data.

qEUBO optimizes decision-making with noisy feedback.

Bayesian optimization (BO) methods often rely on the assumption that the objective function is well-behaved, but in practice, this is seldom true for real-world objectives even if noise-free observations can be collected. Common approaches, which try to model the objective as precisely as possible, often fail to make p…

Paper establishes no-regret property for practical EGO optimization.

Accurate approximations to density functionals have recently been obtained via machine learning (ML). By applying ML to a simple function of one variable without any random sampling, we extract the qualitative dependence of errors on hyperparameters. We find universal features of the behavior in extreme limits, includi…

We study clustering algorithms based on neighborhood graphs on a random sample of data points. The question we ask is how such a graph should be constructed in order to obtain optimal clustering results. Which type of neighborhood graph should one choose, mutual k-nearest neighbor or symmetric k-nearest neighbor? What …

We consider the problem of learning from distributed data in the agnostic setting, i.e., in the presence of arbitrary forms of noise. Our main contribution is a general distributed boosting-based procedure for learning an arbitrary concept space, that is simultaneously noise tolerant, communication efficient, and compu…

Measures time-delay embedding for noisy, sparse data.

We investigate the problem of classification in the presence of unknown class-conditional label noise in which the labels observed by the learner have been corrupted with some unknown class dependent probability. In order to obtain finite sample rates, previous approaches to classification with unknown class-conditiona…

New framework for online control in evolving populations.

Develops a generalized version of Chung's Lemma for stochastic optimization methods.

Quantum machine learns to clean up blurry images.

This paper proposes an out-of-sample extension framework for a global manifold learning algorithm (Isomap) that uses temporal information in out-of-sample points in order to make the embedding more robust to noise and artifacts. Given a set of noise-free training data and its embedding, the proposed framework extends t…

Common nonlinear activation functions used in neural networks can cause training difficulties due to the saturation behavior of the activation function, which may hide dependencies that are not visible to vanilla-SGD (using first order gradients only). Gating mechanisms that use softly saturating activation functions t…

We consider classification in the presence of class-dependent asymmetric label noise with unknown noise probabilities. In this setting, identifiability conditions are known, but additional assumptions were shown to be required for finite sample rates, and so far only the parametric rate has been obtained. Assuming thes…

We study the theoretical advantages of active learning over passive learning. Specifically, we prove that, in noise-free classifier learning for VC classes, any passive learning algorithm can be transformed into an active learning algorithm with asymptotically strictly superior label complexity for all nontrivial targe…

RHPSVM improves SVM performance with robust loss function.

Efficient exploration is a long-standing problem in sensorimotor learning. Major advances have been demonstrated in noise-free, non-stochastic domains such as video games and simulation. However, most of these formulations either get stuck in environments with stochastic dynamics or are too inefficient to be scalable t…

We investigate the "compass rose" (Crack, T.F. and Ledoit, O. (1996), Journal of Finance, 51(2), pg. 751-762) patterns revealed in phase portraits (delay plots) of stock returns. The structures observed in these diagrams have been attributed mainly to price clustering and discreteness. Using wavelet based denoising, we…

We consider the online version of the isotonic regression problem. Given a set of linearly ordered points (e.g., on the real line), the learner must predict labels sequentially at adversarially chosen positions and is evaluated by her total squared loss compared against the best isotonic (non-decreasing) function in hi…

A new method reduces the time needed for Bayesian optimization by a factor of 10-100.

MEDIDA discovers model errors in chaotic systems using sparse regression and data assimilation.

In this paper we study the matrix completion problem: Suppose $X \in {\mathbb R}^{n_r \times n_c}$ is unknown except for a known upper bound $r$ on its rank. By measuring a small number $m \ll n_r n_c$ of elements of $X$, is it possible to recover $X$ exactly with noise-free measurements, or to construct a good approxi…

The paper analyzes Random Search and introduces BLiN-MOS for bandit learning.