arXiv research

New PAC-Bayes bounds for unbounded loss functions.

New aggregation strategy handles unbounded losses with regret bounds.

Two new algorithms improve performance in adversarial bandits with unbounded losses.

New margin-based learning guarantees improve generalization bounds.

Study uses spectral risk for learning with heavy-tailed data.

Paper tackles robust deep learning from weakly dependent data with unbounded loss and input.

New PAC-Bayes training method improves model generalization for unbounded loss.

New algorithms for online learning without boundedness or Lipschitz loss assumptions.

Solves open problem on universally consistent online learning with unbounded losses.

The paper proves deep learning can be robust with certain loss functions.

Uniform deviation bounds limit the difference between a model's expected loss and its loss on an empirical sample uniformly for all models in a learning problem. As such, they are a critical component to empirical risk minimization. In this paper, we provide a novel framework to obtain uniform deviation bounds for loss…

This paper shows that every sublevel set of the loss function of a class of deep over-parameterized neural nets with piecewise linear activation functions is connected and unbounded. This implies that the loss has no bad local valleys and all of its global minima are connected within a unique and potentially very large…

We present new excess risk bounds for general unbounded loss functions including log loss and squared loss, where the distribution of the losses may be heavy-tailed. The bounds hold for general estimators, but they are optimized when applied to $η$-generalized Bayesian, MDL, and empirical risk minimization estimators. …

Paper develops estimators for unbounded density ratios with applications in error control.

New PAC-Bayes bounds for unbounded losses using Cramér-Chernoff techniques.

Consider a financial market in which an agent trades with utility-induced restrictions on wealth. For a utility function which satisfies the condition of reasonable asymptotic elasticity at $-\infty$ we prove that the utility-based super-replication price of an unbounded (but sufficiently integrable) contingent claim i…

Regularized empirical risk minimization including support vector machines plays an important role in machine learning theory. In this paper regularized pairwise learning (RPL) methods based on kernels will be investigated. One example is regularized minimization of the error entropy loss which has recently attracted qu…

This work proposes the Bregman-Tweedie classification model and analyzes the domain structure of the extended exponential function, an extension of the classic generalized exponential function with additional scaling parameter, and related high-level mathematical structures, such as the Bregman-Tweedie loss function an…

We exhibit a strong link between frequentist PAC-Bayesian risk bounds and the Bayesian marginal likelihood. That is, for the negative log-likelihood loss function, we show that the minimization of PAC-Bayesian generalization risk bounds maximizes the Bayesian marginal likelihood. This provides an alternative explanatio…

We give the proof of a tight lower bound on the probability that a binomial random variable exceeds its expected value. The inequality plays an important role in a variety of contexts, including the analysis of relative deviation bounds in learning theory and generalization bounds for unbounded loss functions.

This note shows how to transform high-probability to in-expectation guarantees in machine learning.

SGD converges globally to logistic loss minima for two-layer nets.

Improved bounds for unbounded losses using transductive priors.

New $\alpha$-divergence loss function improves neural density ratio estimation.

New PAC-Bayes bounds for heavy-tailed losses using supermartingales.

New framework captures long-term decision dependence in online learning.

We prove semi-empirical concentration inequalities for random variables which are given as possibly nonlinear functions of independent random variables. These inequalities describe concentration of random variable in terms of the data/distribution-dependent Efron-Stein (ES) estimate of its variance and they do not requ…

New inequalities for unbounded functions improve denoising score matching.

This paper extends the standard chaining technique to prove excess risk upper bounds for empirical risk minimization with random design settings even if the magnitude of the noise and the estimates is unbounded. The bound applies to many loss functions besides the squared loss, and scales only with the sub-Gaussian or …

We show that domains, that allow for convex functions with unbounded gradient at their boundary, are convex.

Paper develops proper, lower-bounded losses for weakly supervised classification.

This paper introduces new loss functions for balanced multi-class classification.

Derandomizing PAC-Bayes bounds for smooth loss functions

Derives bounds for deterministic predictors using smooth loss functions.

Study shows convergence rate for empirical minimizer of unbounded functions with fast growth.

We consider the problem of unconstrained online convex optimization (OCO) with sub-exponential noise, a strictly more general problem than the standard OCO. In this setting, the learner receives a subgradient of the loss functions corrupted by sub-exponential noise and strives to achieve optimal regret guarantee, witho…

Reduces bounded loss learning to binary classification.

Develops PAC-Bayesian framework for physics-informed machine learning.

Kolmogorov neural networks can represent various types of functions.

Paper proposes robust risk measures for non-negative risks with partial information.

PMT uses public data moments to make DP feasible for unbounded data.

For a domain $Ω\subset\mathbb R^n$, we introduce the concept of a uniformly $C^m$ defining function. We characterize uniformly $C^m$ defining functions in terms of the signed distance function for the boundary and provide a large class of examples of unbounded domains with uniformly $C^m$ defining functions. Some of ou…

Study improves robust nonparametric regression in heavy-tailed noise.

Novel oracle-type inequality for logistic loss in DNNs achieves sharp convergence rates.

We study ancient solutions of polynomial growth to both continuous-time and discrete-time heat equations on graphs with unbounded Laplacians. We generalize Colding and Minicozzi's theorem [CM19] on manifolds, and the result [Hua19] on graphs with normalized Laplacians to the setting of graphs with unbounded Laplacians:…

Paper tackles online control of linear systems with unbounded noise.

Global convergence of SGD proven for two-layer neural nets with regularization.

We develop the setting of sequential prediction based on shifting experts and on a "smooth" version of the method of specialized experts. To aggregate experts predictions, we use the AdaHedge algorithm, which is a version of the Hedge algorithm with adaptive learning rate, and extend it by the meta-algorithm Fixed Shar…