Paper analyzes time series prediction using empirical risk minimization.
problem Optimizing 1-step-ahead prediction for time series.
method Empirical risk minimization applied to recursive algorithms for time series forecasting.
result Empirical risk minimization achieves optimal predictive performance.
The paper analyzes the performance of empirical risk minimization for p-norm linear regression.
problem Empirical risk minimization on p-norm linear regression. method Analyzes performance under various conditions and moment assumptions.
result High probability excess risk bounds for empirical risk minimizer, matching asymptotic rates.
Improved sample complexity for diffusion models without needing empirical risk minimizers.
problem Theoretical limitations in sample complexity for diffusion models.
method Structured decomposition of score estimation error, eliminating dependence on neural network parameters.
result Achieved sample complexity bound of O(ε^(-4)) without empirical risk minimizer access.
New learning algorithm for real analytic functions without gradient descent.
problem Learning real analytic functions without gradient descent.
method Taylor approximation and sampling data distribution.
result Nonuniform learning result for real analytic functions.
New method estimates Schrödinger bridge potentials via empirical risk minimization.
problem Estimating Schrödinger bridge potentials from samples.
method Rewriting Schrödinger system as a fixed-point equation and estimating the potential via empirical risk minimization.
result Uniform concentration of empirical risk around population counterpart under sub-Gaussian assumptions.
New approach avoids excess empirical risk in domain generalization.
problem Learning models that generalize to unseen distributions from diverse data sets.
method Minimizes penalty under constraint of optimal empirical risk, leveraging rate-distortion theory.
result Significant improvements in domain generalization performance across multiple methods.
Paper tackles heavy-tailed data without finite variance, proposing robust risk minimization.
problem Empirical risk minimization under heavy-tailed data with finite p-th moment. method Minimizes risk values robustly estimated via Catoni's method, using generalized generic chaining.
result Shows better performance of optimizer based on empirical risks via Catoni-style estimation.
New regularization method reduces support of empirical risk minimization solutions.
problem Regularization in empirical risk minimization with relative entropy.
method Introduces Type-II regularization, characterizes solutions, analyzes properties of relative entropy.
result Type-II regularization collapses solution support into reference measure's support.
Corrects sample selection bias in empirical risk minimization using importance sampling.
problem Statistical learning with biased training data.
method Weighted empirical risk minimization using importance sampling.
result Generalization capacity preserved with estimated importance weights.
STORM enables edge computing for empirical risk minimization.
problem Training models on edge devices for streaming data.
method Online sketching for empirical risk minimization.
result STORM can estimate least-squares objective accurately.
ERM with f-divergence regularization yields unique solution.
problem Optimizing empirical risk with f-divergence. method Mild conditions on f lead to unique optimal measure. result Equivalence of ERM-fDR to different f-divergence regularization. Optimizes bilevel empirical risk minimization with improved oracle calls.
problem Optimizing bilevel empirical risk minimization problems.
method Proposes a bilevel extension of the SARAH algorithm.
result Demonstrates improved oracle calls to achieve stationarity.
Improved multi-group learning with group-realizable concepts.
problem Enhancing multi-group learning efficiency.
method Empirical risk minimization over group-realizable concepts.
result Improved sample complexity in group-realizable settings.
Noise-ignorant empirical risk minimization achieves state-of-the-art performance on noisy data.
problem Learning with noisy labels in multi-class classification problems.
method Introducing relative signal strength (RSS) to quantify transferability and applying Noise Ignorant Empirical Risk Minimization (NI-ERM).
result NI-ERM achieves state-of-the-art performance on CIFAR-N data challenge.
Prove non-asymptotic bounds for minimal risk in statistical learning
problem Estimating minimal risk in statistical learning
method Using concentration inequalities
result Non-asymptotic bounds for minimal risk
The paper analyzes local minima in high-dimensional empirical risk minimization.
problem Understanding local minima in high-dimensional data models.
method Using Kac-Rice formula and proportional asymptotics, the paper derives bounds on local minima.
result Sharp asymptotics on estimation and prediction errors are derived.
In this paper, we present a simple analysis of {\bf fast rates} with {\it high probability} of {\bf empirical minimization} for {\it stochastic composite optimization} over a finite-dimensional bounded convex set with exponential concave loss functions and an arbitrary convex regularization. To the best of our knowledg…
Dual optimization connects ERM-fDR to normalization function.
problem Empirical risk minimization with f-divergence regularization.
method Dual formulation, Legendre-Fenchel transform, implicit function theorem, nonlinear ODE.
result Computational method to calculate normalization function efficiently.
Paper presents ERM with f-divergence regularization and its properties.
problem Minimizing empirical risk with f-divergence constraints. method Introduces normalization function and solves ERM-fDR via ODE. result Characterizes difference between empirical risks and provides numerical algorithm.
New algorithms achieve uniform stability for empirical risk minimization.
problem Designing uniformly stable optimization algorithms for empirical risk minimization.
method Black-box conversion of smooth optimization algorithms and development of Mirror Descent for smooth optimization.
result Optimal algorithms with uniform stability and convergence rates for smooth optimization.
Sharp bounds on ERM's minimal error in regression.
problem Understanding ERM's performance in regression tasks.
method Sharp lower bounds for ERM in random and fixed design settings.
result ERM's performance depends on the global or local complexity of the model.
A new DP algorithm for weighted ERM protects sensitive data in predictive models.
problem Protecting sensitive personal information in predictive models trained via ERM.
method Proposes the first differentially private algorithm for weighted ERM with formal privacy guarantees.
result Demonstrates strong DP guarantees while maintaining robust performance in real-world data.
Reweighting improves risk bounds in certain data regions.
problem Improving risk bounds in classification and heteroscedastic regression.
method Weighted empirical risk minimization with a data-dependent weight function.
result A weighted ERM estimator can achieve superior performance in specific sub-regions.
New method uses robust estimators for Newton's method in empirical risk minimization.
problem Improving robustness in empirical risk minimization.
method Robust Newton's method with gradient and Hessian replaced by robust estimators.
result Faster convergence rates in high-dimensional settings.
We study in this paper the consequences of using the Mean Absolute Percentage Error (MAPE) as a measure of quality for regression models. We show that finding the best model under the MAPE is equivalent to doing weighted Mean Absolute Error (MAE) regression. We also show that, under some asumptions, universal consisten…
Mirror descent algorithm recovers low-rank matrices in matrix sensing.
problem Matrix sensing with low-rank matrices under certain conditions.
method Discrete-time mirror descent applied to empirical risk with Bregman divergence analysis.
result Mirror descent converges to a matrix minimizing a specific nuclear norm-related quantity.
AMP regularization improves deep learning models by favoring flat minima.
problem Improving deep learning model generalization and avoiding overfitting.
method AMP regularization uses adversarial model perturbation to minimize a norm-bounded perturbation of the empirical risk.
result AMP regularization leads to state-of-the-art performance across various deep architectures.
Empirical risk minimization is the main tool for prediction problems, but its extension to relational data remains unsolved. We solve this problem using recent ideas from graph sampling theory to (i) define an empirical risk for relational data and (ii) obtain stochastic gradients for this empirical risk that are autom…
Improved privacy and utility in machine learning with adaptive differential privacy.
problem Enhancing privacy in machine learning models while maintaining utility.
method Adaptive differentially private (ADP) learning method that optimally adapts noise to stepsize.
result ADP method significantly improves utility compared to standard differentially private methods.
Differentially private method for estimating individualized treatment rules.
problem Estimating individualized treatment rules while preserving privacy.
method Differentially private two-stage empirical risk minimization (DP-2ERM).
result Improved privacy-utility trade-off demonstrated through simulations and applications.
Paper shows robust estimators converge to true risk minimizers at optimal rates.
problem Understanding asymptotic properties of robust risk minimizers.
method Investigates robust analogues of empirical risk minimization, focusing on median of means estimator.
result Robust minimizers converge to true minimizers at optimal rates and have similar asymptotic variance.
Optimal score function estimation via empirical risk minimization
problem Estimating the score function of a probability measure on the flat torus from a sample
method Constraining the hypothesis space to a Sobolev ball
result Minimax estimation rates are achieved
Online TERM improves robustness and fairness in streaming data.
problem Streaming data's lack of worst-case fairness and robustness in ERM.
method Proposes an online TERM formulation to balance average-case accuracy with worst-case fairness and robustness.
result Negative tilting effectively suppresses outlier influence, positive tilting improves recall with minimal precision loss.
Deep neural network with l_1-regularization achieves nearly optimal risk bounds.
problem Achieving optimal risk bounds in deep learning.
method Empirical risk minimization with l_1-regularization.
result Adaptively nearly-minimax risk bound across various function classes.
In stochastic optimization, the population risk is generally approximated by the empirical risk. However, in the large-scale setting, minimization of the empirical risk may be computationally restrictive. In this paper, we design an efficient algorithm to approximate the population risk minimizer in generalized linear …
We derive optimal statistical and computational complexity bounds for exp-concave stochastic minimization in terms of the effective dimension. For common eigendecay patterns of the population covariance matrix, this quantity is significantly smaller than the ambient dimension. Our results reveal interesting connections…
This paper investigates robust versions of the general empirical risk minimization algorithm, one of the core techniques underlying modern statistical methods. Success of the empirical risk minimization is based on the fact that for a "well-behaved" stochastic process {f(X), f∈F} indexed b…
In this paper we study the differentially private Empirical Risk Minimization (ERM) problem in different settings. For smooth (strongly) convex loss function with or without (non)-smooth regularization, we give algorithms that achieve either optimal or near optimal utility bounds with less gradient complexity compared …
The study uncovers the breakdown of Gaussian universality in high-dimensional empirical risk minimization.
problem Understanding the breakdown of Gaussian universality in high-dimensional empirical risk minimization.
method Extending the Convex Gaussian Min-Max Theorem to non-Gaussian settings, deriving asymptotic min-max characterizations, and proving asymptotic equivalence of regularizers.
result The projection of the ERM estimator onto a test covariate approximately follows a Gaussian convolution under certain conditions.
New method turns optimization algorithms into uniformly stable learning algorithms for non-Euclidean norms.
problem Non-Euclidean norms in binary classification problems.
method Black-box reduction method using uniformly convex regularizers.
result Achieves optimal statistical risk bounds on excess risk for non-Euclidean norms.
Unified framework for N-tuples learning improves weakly supervised tasks.
problem Reducing annotation burden in supervised learning.
method Empirical risk minimization framework integrating pointwise unlabeled data.
result Framework improves generalization across various N-tuples learning tasks.
Entropy asymmetry affects regularization in ERM, leading to biased solutions.
problem Analyzing the impact of relative entropy asymmetry in ERM regularization.
method Examined Type-I and Type-II ERM-RER, comparing their solutions and properties.
result Type-II ERM-RER regularization introduces a strong bias against training data.
Pairwise similarities and dissimilarities between data points might be easier to obtain than fully labeled data in real-world classification problems, e.g., in privacy-aware situations. To handle such pairwise information, an empirical risk minimization approach has been proposed, giving an unbiased estimator of the cl…
Analyzes empirical risk minimization in finance, showing effectiveness and generalization issues.
problem Analyzing empirical risk minimization in finance for optimal hedging and investment decisions.
method Classical statistical machine learning techniques and non-asymptotic estimates based on Rademacher complexity.
result Over-training leads to anticipative decisions, but non-asymptotic estimates show convergence for large training sets.
In this work we investigate to which extent one can recover class probabilities within the empirical risk minimization (ERM) paradigm. The main aim of our paper is to extend existing results and emphasize the tight relations between empirical risk minimization and class probability estimation. Based on existing literat…
Unified framework TERM improves fairness and robustness.
problem Outliers and subgroup fairness in empirical risk minimization.
method Unified framework TERM with a hyperparameter tilt.
result TERM improves fairness and robustness.
ERM performs well in feature learning with minimal feature maps.
problem Empirical risk minimization in feature learning with square loss.
method Asymptotic and non-asymptotic analysis of ERM performance.
result Excess risk quantiles of ERM match those of oracle procedure under certain conditions.
A new method sorts models to find the best one with minimal risk.
problem Finding the best model with minimal risk among nested models.
method Nested Empirical Risk (NER) and Sorted NER (S-NER) methods.
result The S-NER method selects the true model order and the most parsimonious model.