New aggregation strategy handles unbounded losses with regret bounds.
problem Online optimization with unbounded loss functions.
method Follow The Regularized Leader (FTRL) with φ-divergence.
result Worst regret bound for unbounded losses with alternative divergences.
New PAC-Bayes bounds for unbounded loss functions.
problem Generalization bounds for learning problems with unbounded loss functions.
method Introducing HYPE, a new notion for loss range, and deriving a novel PAC-Bayesian generalization bound.
result PAC-Bayes framework extended to unbounded loss functions.
New bounds for general unbounded loss functions, optimizing for various estimators.
problem Excess risk bounds for general unbounded loss functions, including log loss and squared loss.
method Optimized bounds for η η η -generalized Bayesian, MDL, and empirical risk minimization estimators, using v v v -GRIP and witness conditions. result Achieves i l d e O ( 1 / n ) ilde{O}(1/n) i l d e O ( 1/ n ) rates for certain loss functions under favorable v v v and small model complexity. Two new algorithms improve performance in adversarial bandits with unbounded losses.
problem Adversarial Multi-Armed Bandits with unbounded losses.
method Developed UMAB-NN and UMAB-G for non-negative and general unbounded losses respectively.
result UMAB-NN achieves the first adaptive and scale-free regret bound for non-negative unbounded losses.
Paper provides uniform deviation bounds for unbounded loss functions, improving k-Means clustering bounds.
problem Uniform deviation bounds for unbounded loss functions, specifically k-Means clustering.
method Novel framework to obtain uniform deviation bounds for unbounded loss functions.
result Improved bounds for k-Means clustering under weak assumptions, achieving $\mathcal{O}\left(m^{-\frac12}
ight)$ rate.
New margin-based learning guarantees improve generalization bounds.
problem Improving generalization bounds for machine learning models.
method Relative deviation margin bounds using empirical margin loss and Rademacher complexity.
result Distribution-dependent generalization bounds for unbounded loss functions.
New PAC-Bayes training method improves model generalization for unbounded loss.
problem Improving generalization of complex models under unbounded loss.
method Established new PAC-Bayes bound for unbounded loss, jointly training prior and posterior.
result Outperforms existing PAC-Bayes training algorithms and matches ERM accuracy.
The paper proves deep learning can be robust with certain loss functions.
problem The robustness of deep learning models under flawed data.
method Empirical-risk minimization with unbounded, Lipschitz-continuous loss functions.
result These loss functions provide efficient prediction under minimal data assumptions.
Solves open problem on universally consistent online learning with unbounded losses.
problem Open problem on universally consistent online learning with unbounded losses.
method Constructs random measurable partitions of the instance space.
result Simple memorization rule is optimistically universal for any unbounded loss.
New PAC-Bayes bounds for unbounded losses using Cramér-Chernoff techniques.
problem Developing bounds for unbounded losses in PAC-Bayesian settings.
method Introducing a new PAC-Bayes oracle bound using Cramér-Chernoff bounds and controlling random variable tails.
result Our bounds generalize and improve upon previous results, providing more informative and potentially tighter bounds.
Paper develops estimators for unbounded density ratios with applications in error control.
problem Estimating density ratios with unbounded domains and ranges.
method Least squares and logistic regression loss functions for density ratio estimation.
result Established upper bounds on estimation errors with optimal rates for unbounded density ratios.
Study uses spectral risk for learning with heavy-tailed data.
problem Learning with heavy-tailed loss distributions.
method Spectral risk with Lipschitz-continuous density, derivative-free learning.
result Excess risk guarantees and improved performance over traditional methods.
Paper tackles robust deep learning from weakly dependent data with unbounded loss and input.
problem Tackles robust deep learning from weakly dependent data with unbounded loss and input.
method Establishes non-asymptotic bounds for expected excess risk under strong mixing and ψ ψ ψ -weak dependence assumptions. result Derives a relationship between bounds and r r r , and shows convergence rate close to i.i.d. results for r = ∞ r=\infty r = ∞ . New algorithms for online learning without boundedness or Lipschitz loss assumptions.
problem Online learning with unbounded domains and non-Lipschitz losses.
method Developed an algorithm with a specific regret bound and used it for saddle-point optimization.
result First algorithm achieving non-trivial dynamic regret in an unbounded domain for non-Lipschitz losses.
Consider a financial market in which an agent trades with utility-induced restrictions on wealth. For a utility function which satisfies the condition of reasonable asymptotic elasticity at − ∞ -\infty − ∞ we prove that the utility-based super-replication price of an unbounded (but sufficiently integrable) contingent claim i…
Deep learning networks have connected sublevel sets, avoiding local minima.
problem Finding local minima in deep learning networks.
method Analyzing sublevel sets of loss functions in over-parameterized neural nets.
result Sublevel sets are connected and unbounded, ensuring all global minima are accessible.
This note shows how to transform high-probability to in-expectation guarantees in machine learning.
problem The challenge of constructing reliable machine learning models due to sampling randomness.
method Transforming high-probability to in-expectation guarantees using a witness condition for unbounded loss functions.
result A technical transformation method for generalization guarantees in machine learning.
Link between PAC-Bayesian bounds and Bayesian marginal likelihood.
problem Understanding the connection between frequentist and Bayesian approaches in risk minimization.
method Exhibit a strong link between PAC-Bayesian risk bounds and Bayesian marginal likelihood, especially for the negative log-likelihood loss function.
result PAC-Bayesian minimization maximizes Bayesian marginal likelihood, providing an alternative to Bayesian Occam's razor.
We give the proof of a tight lower bound on the probability that a binomial random variable exceeds its expected value. The inequality plays an important role in a variety of contexts, including the analysis of relative deviation bounds in learning theory and generalization bounds for unbounded loss functions.
Regularized empirical risk minimization including support vector machines plays an important role in machine learning theory. In this paper regularized pairwise learning (RPL) methods based on kernels will be investigated. One example is regularized minimization of the error entropy loss which has recently attracted qu…
Proposes a new classification model using extended exponential functions.
problem Improving classification accuracy in binary linear classification problems.
method Developed a Bregman-Tweedie classification model based on extended exponential functions.
result The H-Bregman and L-Bregman sub-models outperform traditional methods in ranking and classification accuracy.
New PAC-Bayes bounds for heavy-tailed losses using supermartingales.
problem Extending PAC-Bayes bounds to heavy-tailed losses.
method Using supermartingales and bounded variance assumption.
result PAC-Bayes generalization bounds for heavy-tailed losses.
New framework captures long-term decision dependence in online learning.
problem Long-term dependence on past decisions in online learning.
method Introduces Online Convex Optimization with Unbounded Memory (OCO-UMB) and p p p -effective memory capacity. result Proves O ( H p T ) O(\sqrt{H_p T}) O ( H p T ) upper bound on policy regret and matching lower bound. We prove new concentration inequalities for random variables.
problem Concentration of random variables in nonlinear functions.
method Efron-Stein inequalities and PAC-Bayesian approach.
result User-friendly concentration bounds for various applications.
This paper introduces new loss functions for balanced multi-class classification.
problem Balancing class imbalance in multi-class classification.
method Introduces two new surrogate loss families: GLA and GCA.
result GCA losses offer stronger theoretical guarantees in imbalanced settings.
Study ancient solutions on graphs with unbounded Laplacians, generalizing previous results.
problem Understanding ancient solutions on graphs with unbounded Laplacians.
method Generalizing Colding and Minicozzi's theorem and Hua's result to graphs with unbounded Laplacians.
result The dimension of the space of ancient solutions of polynomial growth is bounded by the dimension of harmonic functions with the same growth.
SGD converges globally to logistic loss minima for two-layer nets.
problem Global convergence of SGD for logistic loss on two-layer neural nets.
method Demonstrates existence of Frobenius norm regularized logistic loss functions as Villani functions, proving convergence and exponential rate.
result SGD converges globally to the global minima of appropriately regularized logistic empirical risk of depth 2 nets.
Derandomizing PAC-Bayes bounds for smooth loss functions
problem Derandomizing PAC-Bayes bounds for smooth loss functions
method Exploiting smoothness properties of both the loss and the predictor class
result Bounds for deterministic predictors that involve flatness quantities
Improved bounds for unbounded losses using transductive priors.
problem Sequential regression and classification with unbounded losses.
method Exponential weights algorithm with transductive priors.
result Statistical bounds independent of design vectors and optimal solution norm.
Derives bounds for deterministic predictors using smooth loss functions.
problem Generalizing probabilistic predictors to deterministic ones.
method Exploits smoothness properties of loss and predictor classes, controlling the Jensen gap class through Rademacher complexity.
result Derives bounds for deterministic predictors involving flatness quantities from Jacobians and Hessians.
New method combines adaptive learning rate and flexible prediction for online loss aggregation.
problem Online aggregation of unbounded losses using shifting experts.
method Adapted AdaHedge algorithm with Fixed Share meta-algorithm for signed unbounded losses.
result Improved shifting regret and validity of regret bounds in adversarial setting.
Paper develops proper, lower-bounded losses for weakly supervised classification.
problem Weakly supervised classification with corrupted labels.
method Representation theorem for proper losses, derived condition for lower-boundedness, generalized logit squeezing.
result Proper and lower-bounded losses for weak-label learning.
New α \alpha α -divergence loss function improves neural density ratio estimation.
problem Optimization challenges in existing DRE methods, especially overfitting and high sample requirements.
method Derived α \alpha α -divergence loss function ( α \alpha α -Div) for neural density ratio estimation. result The α \alpha α -divergence loss function ( α \alpha α -Div) offers stable and effective optimization for DRE. Paper proposes robust risk measures for non-negative risks with partial information.
problem Tackles robustness of distortion risk measures under distributional uncertainty.
method Introduces new uncertainty sets and derives closed-form expressions for risk maximization.
result Derives closed-form expressions for risk maximization over uncertainty sets.
Study shows convergence rate for empirical minimizer of unbounded functions with fast growth.
problem Convergence rate of empirical minimizer for unbounded functions with fast growth.
method Analyzes L 1 L^1 L 1 -distance convergence rate of the empiric minimizer for coercive functions sampled with noise. result Convergence rate is bounded above by a n n − 1 / q a_n n^{-1/q} a n n − 1/ q , where q q q is the dimension and a n = o ( n ε ) a_n = o(n^\varepsilon) a n = o ( n ε ) for every ε > 0 \varepsilon > 0 ε > 0 . New inequalities for unbounded functions improve denoising score matching.
problem Statistical error bounds for denoising score matching with unbounded objective functions.
method Derive new concentration inequalities using McDiarmid's inequality and Rademacher complexity bounds.
result Improved statistical error bounds for denoising score matching.
Study improves robust nonparametric regression in heavy-tailed noise.
problem Robust nonparametric regression with heavy-tailed noise and unbounded functions.
method Huber regression in reproducing kernel Hilbert spaces (RKHS), probabilistic effective hypothesis space, new comparison theorems.
result Explicit finite-sample error bounds and convergence rates for Huber regression in RKHS under heavy-tailed noise.
New method improves generalization of SGD with momentum.
problem Lack of theoretical understanding of generalization error in momentum-based SGD.
method Introduced SGD with early momentum (SGDEM) and analyzed its generalization properties.
result SGDEM can train machine learning models with a guarantee for generalization.
PMT uses public data moments to make DP feasible for unbounded data.
problem Applying differential privacy to unbounded data distributions.
method Public-moment-guided Truncation (PMT) using second-moments from public data.
result PMT improves the accuracy and stability of DP models.
We show that domains, that allow for convex functions with unbounded gradient at their boundary, are convex.
Novel oracle-type inequality for logistic loss in DNNs achieves sharp convergence rates.
problem Generalization analysis for binary classification with DNNs and logistic loss.
method Established an oracle-type inequality to handle the boundedness of the target function.
result Optimal convergence rates for fully connected ReLU DNN classifiers trained with logistic loss.
AES learns feasible domains in unbounded spaces with bounded query budget.
problem Learning feasible domains in unbounded input spaces with limited query budget.
method Active Expansion Sampling (AES) progressively expands knowledge of the input space, switching between learning decision boundaries and searching for new feasible domains.
result AES has a misclassification loss guarantee within the explored region, independent of iterations or labeled samples.
We develop a new bound for estimating CVaR from samples of an unbounded random variable.
problem Estimating CVaR from i.i.d. samples of an unbounded random variable.
method Derive a one-sided concentration bound for a CVaR estimator.
result A novel concentration bound for CVaR estimation.
Stable GFlowNets prevent loss spikes and mode collapse in training.
problem Unstable training of GFlowNets leading to loss spikes and mode collapse.
method Assessed sensitivity of GFlowNet objectives, derived loss-to-TV bounds, and proposed Stable GFlowNets.
result Stable GFlowNets improve training behavior and distributional fidelity.
Parameter-free online convex optimization with sub-exponential noise achieves optimal regret.
problem Online convex optimization with sub-exponential noise, especially when subgradients are unbounded.
method Designing a novel parameter-free algorithm BANCO via a reduction to betting on noisy coins.
result BANCO achieves the optimal regret rate in the problem of unconstrained online convex optimization with sub-exponential noise.
New analysis for black-box learning without gradients, improving generalization bounds.
problem Generalization error analysis for derivative-free optimization.
method Zeroth-order Stochastic Search (ZoSS) algorithm for Lipschitz and smooth losses.
result Generalization bounds independent of model dimension, batch size, and number of perturbed evaluations.
Efficiently estimates quantiles and maximum in unbounded datasets with differential privacy.
problem Efficiently estimating quantiles and maximum in unbounded datasets with differential privacy.
method Simple invocation of a subroutine called AboveThreshold, iteratively called in Sparse Vector Technique.
result Improved estimates on highest quantiles with robustness and accuracy.
The paper introduces neural INGARCH models for time series of counts.
problem Analyzing time series of counts using traditional INGARCH models.
method Combining artificial neural networks with INGARCH models.
result Neural INGARCH models outperform traditional models in information loss.