arXiv research

Two new algorithms improve performance in adversarial bandits with unbounded losses.

New aggregation strategy handles unbounded losses with regret bounds.

New PAC-Bayes bounds for unbounded loss functions.

New algorithms for online learning without boundedness or Lipschitz loss assumptions.

Solves open problem on universally consistent online learning with unbounded losses.

New PAC-Bayes bounds for unbounded losses using Cramér-Chernoff techniques.

New PAC-Bayes training method improves model generalization for unbounded loss.

Study uses spectral risk for learning with heavy-tailed data.

Improved bounds for unbounded losses using transductive priors.

New margin-based learning guarantees improve generalization bounds.

Paper tackles robust deep learning from weakly dependent data with unbounded loss and input.

Uniform deviation bounds limit the difference between a model's expected loss and its loss on an empirical sample uniformly for all models in a learning problem. As such, they are a critical component to empirical risk minimization. In this paper, we provide a novel framework to obtain uniform deviation bounds for loss…

We present new excess risk bounds for general unbounded loss functions including log loss and squared loss, where the distribution of the losses may be heavy-tailed. The bounds hold for general estimators, but they are optimized when applied to $η$-generalized Bayesian, MDL, and empirical risk minimization estimators. …

Paper develops estimators for unbounded density ratios with applications in error control.

Consider a financial market in which an agent trades with utility-induced restrictions on wealth. For a utility function which satisfies the condition of reasonable asymptotic elasticity at $-\infty$ we prove that the utility-based super-replication price of an unbounded (but sufficiently integrable) contingent claim i…

This paper shows that every sublevel set of the loss function of a class of deep over-parameterized neural nets with piecewise linear activation functions is connected and unbounded. This implies that the loss has no bad local valleys and all of its global minima are connected within a unique and potentially very large…

The paper proves deep learning can be robust with certain loss functions.

Develops PAC-Bayesian framework for physics-informed machine learning.

New framework captures long-term decision dependence in online learning.

We develop the setting of sequential prediction based on shifting experts and on a "smooth" version of the method of specialized experts. To aggregate experts predictions, we use the AdaHedge algorithm, which is a version of the Hedge algorithm with adaptive learning rate, and extend it by the meta-algorithm Fixed Shar…

Efficiently estimates quantiles and maximum in unbounded datasets with differential privacy.

Stable GFlowNets prevent loss spikes and mode collapse in training.

Regularized empirical risk minimization including support vector machines plays an important role in machine learning theory. In this paper regularized pairwise learning (RPL) methods based on kernels will be investigated. One example is regularized minimization of the error entropy loss which has recently attracted qu…

New PAC-Bayes bounds for heavy-tailed losses using supermartingales.

We give the proof of a tight lower bound on the probability that a binomial random variable exceeds its expected value. The inequality plays an important role in a variety of contexts, including the analysis of relative deviation bounds in learning theory and generalization bounds for unbounded loss functions.

This note shows how to transform high-probability to in-expectation guarantees in machine learning.

We exhibit a strong link between frequentist PAC-Bayesian risk bounds and the Bayesian marginal likelihood. That is, for the negative log-likelihood loss function, we show that the minimization of PAC-Bayesian generalization risk bounds maximizes the Bayesian marginal likelihood. This provides an alternative explanatio…

In several real-world applications involving decision making under uncertainty, the traditional expected value objective may not be suitable, as it may be necessary to control losses in the case of a rare but extreme event. Conditional Value-at-Risk (CVaR) is a popular risk measure for modeling the aforementioned objec…

We consider prediction with expert advice under the log-loss with the goal of deriving efficient and robust algorithms. We argue that existing algorithms such as exponentiated gradient, online gradient descent and online Newton step do not adequately satisfy both requirements. Our main contribution is an analysis of th…

The paper improves PAC-Bayes bounds for losses with finite moments.

Paper develops proper, lower-bounded losses for weakly supervised classification.

This paper extends the standard chaining technique to prove excess risk upper bounds for empirical risk minimization with random design settings even if the magnitude of the noise and the estimates is unbounded. The bound applies to many loss functions besides the squared loss, and scales only with the sub-Gaussian or …

SGD converges globally to logistic loss minima for two-layer nets.

We consider the problem of unconstrained online convex optimization (OCO) with sub-exponential noise, a strictly more general problem than the standard OCO. In this setting, the learner receives a subgradient of the loss functions corrupted by sub-exponential noise and strives to achieve optimal regret guarantee, witho…

This work proposes the Bregman-Tweedie classification model and analyzes the domain structure of the extended exponential function, an extension of the classic generalized exponential function with additional scaling parameter, and related high-level mathematical structures, such as the Bregman-Tweedie loss function an…

PMT uses public data moments to make DP feasible for unbounded data.

Unbounded convex domains have zero mean curvature on disconnected boundaries.

We prove semi-empirical concentration inequalities for random variables which are given as possibly nonlinear functions of independent random variables. These inequalities describe concentration of random variable in terms of the data/distribution-dependent Efron-Stein (ES) estimate of its variance and they do not requ…

New $\alpha$-divergence loss function improves neural density ratio estimation.

The paper proves homotopy equivalences for spaces of unbounded Fredholm operators.

Golden L surface has unbounded bunching of saddle connections

We construct an unbounded representative for the shriek class associated to the embeddings of spheres into Euclidean space. We equip this unbounded Kasparov cycle with a connection and compute the unbounded Kasparov product with the Dirac operator on $\mathbb R^{n+1}$. We find that the resulting spectral triple for the…

This memoir presents a systematic study of the utility maximization problem of an investor in a constrained and unbounded financial market. Building upon the work of Hu et al. (2005) [Ann. Appl. Probab., 15, 1691--1712] in a bounded framework, we extend our analysis to the more challenging unbounded case. Our methodolo…

It is shown that the compactly supported identity component of the diffeomorphism group of the 2-dimensional punctured torus $\mathbb T^2_p$ is an unbounded group. It follows that the fragmentation norm of $\mathbb T^2_p$ is unbounded.

New inequalities for unbounded functions improve denoising score matching.

Study focal surfaces of wave fronts with unbounded curvatures.

This paper introduces new loss functions for balanced multi-class classification.

Study improves robust nonparametric regression in heavy-tailed noise.