arXiv research

A new method to measure neural network expressiveness using tighter upper bounds.

This paper improves Bayesian optimization methods with tighter regret bounds and practical solutions.

New bounds for SGD show improved performance in various settings.

Improved bounds for Black-Scholes volatility lead to faster root-finding.

Stochastic variational inference (SVI) plays a key role in Bayesian deep learning. Recently various divergences have been proposed to design the surrogate loss for variational inference. We present a simple upper bound of the evidence as the surrogate loss. This evidence upper bound (EUBO) equals to the log marginal li…

Method bounds tail probabilities of continuous RVs.

We give bounds on the number of non-simple closed curves on a negatively curved surface, given upper bounds on both length and self-intersection number. In particular, it was previously known that the number of all closed curves of length at most $L$ grows exponentially in $L$. We get exponentially tighter bounds given…

Efficient local Lipschitz bounds improve neural network robustness.

The study tightens risk bounds for mixtures of experts using local differential privacy.

Debona improves neural network verification by faster and tighter bounds.

PopArt efficiently solves sparse linear bandits with tighter recovery guarantees.

An information-theoretic upper bound on the generalization error of supervised learning algorithms is derived. The bound is constructed in terms of the mutual information between each individual training sample and the output of the learning algorithm. The bound is derived under more general conditions on the loss func…

The standard approach to supervised classification involves the minimization of a log-loss as an upper bound to the classification error. While this is a tight bound early on in the optimization, it overemphasizes the influence of incorrectly classified examples far from the decision boundary. Updating the upper bound …

New algorithms reduce reinforcement learning regret in factored MDPs.

Price of anarchy, the performance ratio, which could characterize the loss of efficiency of the distributed supply chain management compared with the integrated supply chain management is discussed by utilizing newsvendor problem in single period which is well-known. In particular, some of remarkable distributed polici…

Gen-CUDE is a neural network for denoising noisy channels.

New method improves understanding of machine learning model performance.

Algorithm identifies best arm with prior info in structured bandits.

Automating statistical modelling is a challenging problem in artificial intelligence. The Automatic Statistician takes a first step in this direction, by employing a kernel search algorithm with Gaussian Processes (GP) to provide interpretable statistical models for regression problems. However this does not scale due …

In this paper we study the differentially private Empirical Risk Minimization (ERM) problem in different settings. For smooth (strongly) convex loss function with or without (non)-smooth regularization, we give algorithms that achieve either optimal or near optimal utility bounds with less gradient complexity compared …

Improved robustness for deep neural networks with tighter bounds and attacks.

Paper provides an upper bound for bias of Nadaraya-Watson kernel regression.

Strong worst-case performance bounds for episodic reinforcement learning exist but fortunately in practice RL algorithms perform much better than such bounds would predict. Algorithms and theory that provide strong problem-dependent bounds could help illuminate the key features of what makes a RL problem hard and reduc…

An important class of distance metrics proposed for training generative adversarial networks (GANs) is the integral probability metric (IPM), in which the neural net distance captures the practical GAN training via two neural networks. This paper investigates the minimax estimation problem of the neural net distance ba…

In this work, we present a novel upper bound of target error to address the problem for unsupervised domain adaptation. Recent studies reveal that a deep neural network can learn transferable features which generalize well to novel tasks. Furthermore, a theory proposed by Ben-David et al. (2010) provides a upper bound …

We provide bounds for kernel matrices and new approximations for high-dimensional data.

We derive an upper bound on the local Rademacher complexity of $\ell_p$-norm multiple kernel learning, which yields a tighter excess risk bound than global approaches. Previous local approaches aimed at analyzed the case $p=1$ only while our analysis covers all cases $1\leq p\leq\infty$, assuming the different feature …

New method improves robustness of smoothed classifiers against adversarial attacks.

We investigate the complexity of deep neural networks (DNN) that represent piecewise linear (PWL) functions. In particular, we study the number of linear regions, i.e. pieces, that a PWL function represented by a DNN can attain, both theoretically and empirically. We present (i) tighter upper and lower bounds for the m…

Prediction intervals are a valuable way of quantifying uncertainty in regression problems. Good prediction intervals should be both correct, containing the actual value between the lower and upper bound at least a target percentage of the time; and tight, having a small mean width of the bounds. Many prior techniques f…

Estimation of individual treatment effects is commonly used as the basis for contextual decision making in fields such as healthcare, education, and economics. However, it is often sufficient for the decision maker to have estimates of upper and lower bounds on the potential outcomes of decision alternatives to assess …

Investigates Lipschitz continuity in neural networks across various settings.

New bound on machine learning model performance using Jensen-Shannon information.

Improved regret bounds for DP-KLUCB and DP-IMED in Bernoulli bandits.

LSCI provides locally adaptive prediction sets for operator models with tighter coverage.

We consider the problem of online planning in a Markov Decision Process when given only access to a generative model, restricted to open-loop policies - i.e. sequences of actions - and under budget constraint. In this setting, the Open-Loop Optimistic Planning (OLOP) algorithm enjoys good theoretical guarantees but is …

The paper calculates upper bounds on ReLU network Lipschitz constants.

In this paper, we reformulate the forest representation learning approach as an additive model which boosts the augmented feature instead of the prediction. We substantially improve the upper bound of generalization gap from $\mathcal{O}(\sqrt\frac{\ln m}{m})$ to $\mathcal{O}(\frac{\ln m}{m})$, while $λ$ - the margin r…

Study bounds on kernel function entropy for finite measures.

The paper introduces a method to learn and apply value envelopes for faster online reinforcement learning.

Proposes efficient bounds for causal effect estimation under weak confounding.

Variational Inference is a powerful tool in the Bayesian modeling toolkit, however, its effectiveness is determined by the expressivity of the utilized variational distributions in terms of their ability to match the true posterior distribution. In turn, the expressivity of the variational family is largely limited by …

New algorithm optimizes Hölder smooth functions in RKHS with tighter regret bounds.

New tighter confidence bounds for sequential kernel regression.

Motivated by the pressing need for efficient optimization in online recommender systems, we revisit the cascading bandit model proposed by Kveton et al. (2015). While Thompson sampling (TS) algorithms have been shown to be empirically superior to Upper Confidence Bound (UCB) algorithms for cascading bandits, theoretica…

Sharp bounds derived for test error of finite-rank kernel ridge regression.

New tighter bounds for learning algorithms from Steinke & Zakynthinou's supersample setting.

Loopy and generalized belief propagation are popular algorithms for approximate inference in Markov random fields and Bayesian networks. Fixed points of these algorithms correspond to extrema of the Bethe and Kikuchi free energy. However, belief propagation does not always converge, which explains the need for approach…