A new DP algorithm for weighted ERM protects sensitive data in predictive models.
problem Protecting sensitive personal information in predictive models trained via ERM.
method Proposes the first differentially private algorithm for weighted ERM with formal privacy guarantees.
result Demonstrates strong DP guarantees while maintaining robust performance in real-world data.
Reweighting improves risk bounds in certain data regions.
problem Improving risk bounds in classification and heteroscedastic regression.
method Weighted empirical risk minimization with a data-dependent weight function.
result A weighted ERM estimator can achieve superior performance in specific sub-regions.
Corrects sample selection bias in empirical risk minimization using importance sampling.
problem Statistical learning with biased training data.
method Weighted empirical risk minimization using importance sampling.
result Generalization capacity preserved with estimated importance weights.
Differentially private method for estimating individualized treatment rules.
problem Estimating individualized treatment rules while preserving privacy.
method Differentially private two-stage empirical risk minimization (DP-2ERM).
result Improved privacy-utility trade-off demonstrated through simulations and applications.
The paper analyzes prediction error in nonstationary settings using weighted risk minimization.
problem Prediction under distribution drift and nonstationary conditions.
method General decomposition of excess risk into learning and drift terms, proving oracle inequalities under mixing conditions.
result Oracle inequalities for the learning error, providing bounds that hold uniformly over arbitrary weight classes.
We study in this paper the consequences of using the Mean Absolute Percentage Error (MAPE) as a measure of quality for regression models. We show that finding the best model under the MAPE is equivalent to doing weighted Mean Absolute Error (MAE) regression. We also show that, under some asumptions, universal consisten…
A new one-step method for covariate shift adaptation.
problem Real-world data often violates the assumption of same distribution for training and test samples.
method Proposes a one-step optimization approach to jointly learn the model and weights.
result The proposed method achieves a generalization error bound and is empirically effective.
Gradient descent recovers planted weights in shallow neural networks with quadratic activations.
problem Learning shallow neural networks with quadratic activations and planted weights.
method Analysis of optimization landscape, gradient descent, semicircle law for Wishart ensemble.
result Gradient descent can recover planted weights if initialized below an energy barrier.
We tackle imbalanced classification by weighting losses and derive robust risks.
problem Imbalanced classification where a label has low marginal probability.
method We examine convergence rates of weighted risks, define robust risks, and derive new robust risk problems.
result We show that particular weightings lead to conditional value at risk (CVaR) and derive new robust risk problems.
Method identifies change points in high-dimensional models using sample weights.
problem Identifying change points in high-dimensional generalized linear models.
method Sample-weighted empirical risk minimization (Weighted ERM).
result Weighted ERM yields precise asymptotic performance characterization for Gaussian designs.
We tackle the issue of classifier combinations when observations have multiple views. Our method jointly learns view-specific weighted majority vote classifiers (i.e. for each view) over a set of base voters, and a second weighted majority vote classifier over the set of these view-specific weighted majority vote class…
New algorithm reduces ERM problem size while maintaining accuracy.
problem Empirical risk minimization problem size reduction.
method Adaptive Deterministic Uniform-Weight Trimming (ADUWT) algorithm.
result Uniform (1±ε) relative-error approximation for ERM objective. DRSS method identifies unnecessary samples and features in DR covariate shift.
problem Identifying unnecessary samples and features in DR covariate shift.
method Combines DR learning and safe screening techniques.
result DRSS method provides reliable identification of unnecessary samples and features under specified distribution uncertainty.
We consider a generic convex optimization problem associated with regularized empirical risk minimization of linear predictors. The problem structure allows us to reformulate it as a convex-concave saddle point problem. We propose a stochastic primal-dual coordinate (SPDC) method, which alternates between maximizing ov…
Study uses spectral risk for learning with heavy-tailed data.
problem Learning with heavy-tailed loss distributions.
method Spectral risk with Lipschitz-continuous density, derivative-free learning.
result Excess risk guarantees and improved performance over traditional methods.
A new meta-learning framework that assigns weights to source tasks based on target samples.
problem Learning initialization for target tasks with limited labeled examples.
method A general framework that assigns weights to the loss of different source tasks, which can depend on the target samples. Provides upper bounds and develops a learning algorithm based on minimizing the error bound with respect to an empirical IPM.
result Empirically, the weighted meta-learning algorithm finds better initializations than uniformly-weighted meta-learning algorithms.
This work analyzes IRM and ERM from sample complexity perspective, revealing different behaviors under various distribution shifts.
problem Choosing between IRM and ERM for OOD generalization.
method Sample complexity analysis comparing IRM and ERM under different data generation mechanisms.
result IRM is preferred over ERM for certain distribution shifts, leading to better OOD generalization.
New method combines experimental and observational data for causal inference.
problem Combining internal validity of experiments and larger sample sizes of observations.
method Empirical risk minimization (ERM) framework with cross-validation.
result Efficacy and reliability demonstrated on real and synthetic data.
New guarantees for ERM with adaptively collected data.
problem Failure of ERM guarantees with adaptively collected data.
method Importance sampling weighted ERM algorithm with maximal inequality.
result First generalization guarantees and fast convergence rates for adaptively collected data.
The paper analyzes the performance of empirical risk minimization for p-norm linear regression.
problem Empirical risk minimization on p-norm linear regression. method Analyzes performance under various conditions and moment assumptions.
result High probability excess risk bounds for empirical risk minimizer, matching asymptotic rates.
Let $\cF$ be a set of M classification procedures with values in [−1,1]. Given a loss function, we want to construct a procedure which mimics at the best possible rate the best procedure in $\cF$. This fastest rate is called optimal rate of aggregation. Considering a continuous scale of loss functions with various …
New bound improves on weighted majority vote risk estimation.
problem Improving risk estimation for weighted majority vote.
method Novel Chebyshev-Cantelli inequality and PAC-Bayes-Bennett inequality.
result New bounds improve on existing methods.
Active inference framework improves U-statistic estimation efficiency.
problem Costly acquisition of labels for U-statistics. method Active inference framework with optimal sampling rule.
result Substantial gains in estimation efficiency over baseline methods.
Prove non-asymptotic bounds for minimal risk in statistical learning
problem Estimating minimal risk in statistical learning
method Using concentration inequalities
result Non-asymptotic bounds for minimal risk
Distributed machine learning is an approach allowing different parties to learn a model over all data sets without disclosing their own data. In this paper, we propose a weighted distributed differential privacy (WD-DP) empirical risk minimization (ERM) method to train a model in distributed setting, considering differ…
New method integrates real and synthetic data to improve machine learning models.
problem Expensive or impractical collection of high-quality data limits machine learning.
method Weighted empirical risk minimization approach for integrating surrogate data.
result Integrating surrogate data can significantly reduce test error on the original distribution.
Paper proposes methods for transfer learning with random coefficient ridge regression.
problem Estimation and prediction in high-dimensional settings with related models.
method Two estimators using weighted sums of ridge estimates from target and source models.
result Explicit expression of estimation and prediction risks derived using random matrix theory.
New algorithm reduces online logistic regression regret without exponential constant.
problem Improper learning in online logistic regression with logarithmic regret.
method Regularized empirical risk minimization with surrogate losses.
result Regret scaling as O(B log(Bn)) with low computational complexity.
We consider the classic supervised learning problem, where a continuous non-negative random label Y (i.e. a random duration) is to be predicted based upon observing a random vector X valued in Rd with d≥1 by means of a regression rule with minimum least square error. In various applications, rangi…
Paper analyzes time series prediction using empirical risk minimization.
problem Optimizing 1-step-ahead prediction for time series.
method Empirical risk minimization applied to recursive algorithms for time series forecasting.
result Empirical risk minimization achieves optimal predictive performance.
We analyze the generalization and robustness of the batched weighted average algorithm for V-geometrically ergodic Markov data. This algorithm is a good alternative to the empirical risk minimization algorithm when the latter suffers from overfitting or when optimizing the empirical risk is hard. For the generalization…
Paper tackles heavy-tailed data without finite variance, proposing robust risk minimization.
problem Empirical risk minimization under heavy-tailed data with finite p-th moment. method Minimizes risk values robustly estimated via Catoni's method, using generalized generic chaining.
result Shows better performance of optimizer based on empirical risks via Catoni-style estimation.
Novel analysis improves weighted majority vote in multiclass classification.
problem Improving the performance of weighted majority vote in multiclass classification.
method Analyzes expected risk of weighted majority vote, considering prediction correlations and provides a bound for efficient minimization.
result Minimization of the new bound typically does not degrade the test error of the ensemble.
Improved sample complexity for diffusion models without needing empirical risk minimizers.
problem Theoretical limitations in sample complexity for diffusion models.
method Structured decomposition of score estimation error, eliminating dependence on neural network parameters.
result Achieved sample complexity bound of O(ε^(-4)) without empirical risk minimizer access.
New bounds for balanced classification improve understanding of imbalanced datasets.
problem Negligible size of the minority class in imbalanced datasets.
method Developed non-asymptotic and consistent bounds for balanced empirical risk minimization and balanced nearest neighbors estimates.
result Improved understanding of class-weighting benefits in real-world imbalanced classification settings.
Paper shows robust estimators converge to true risk minimizers at optimal rates.
problem Understanding asymptotic properties of robust risk minimizers.
method Investigates robust analogues of empirical risk minimization, focusing on median of means estimator.
result Robust minimizers converge to true minimizers at optimal rates and have similar asymptotic variance.
This work tackles robust multi-source domain adaptation under label shift.
problem Label shift and data contamination in multi-source domain adaptation.
method Domain-weighted empirical risk minimization framework with refinement procedure.
result The proposed method achieves superior performance in multi-category classification problems.
This study proposes an equal-weight portfolio strategy to reduce risk compared to traditional ETFs.
problem Risk of passive ETFs not matching optimal portfolio weights.
method Introduced an equal-weight portfolio strategy to reduce idiosyncratic risk.
result Equal-weight portfolio has lower risk than traditional ETFs, especially during idiosyncratic events.
New learning algorithm for real analytic functions without gradient descent.
problem Learning real analytic functions without gradient descent.
method Taylor approximation and sampling data distribution.
result Nonuniform learning result for real analytic functions.
We study in this paper the consequences of using the Mean Absolute Percentage Error (MAPE) as a measure of quality for regression models. We show that finding the best model under the MAPE is equivalent to doing weighted Mean Absolute Error (MAE) regression. We show that universal consistency of Empirical Risk Minimiza…
We study in this paper the consequences of using the Mean Absolute Percentage Error (MAPE) as a measure of quality for regression models. We prove the existence of an optimal MAPE model and we show the universal consistency of Empirical Risk Minimization based on the MAPE. We also show that finding the best model under…
Importance sampling has become an important tool for the computation of tail-based risk measures. Since such quantities are often determined mainly by rare events standard Monte Carlo can be inefficient and importance sampling provides a way to speed up computations. This paper considers moderate deviations for the wei…
Recently theoretical guarantees have been obtained for matrix completion in the non-uniform sampling regime. In particular, if the sampling distribution aligns with the underlying matrix's leverage scores, then with high probability nuclear norm minimization will exactly recover the low rank matrix. In this article, we…
New method estimates Schrödinger bridge potentials via empirical risk minimization.
problem Estimating Schrödinger bridge potentials from samples.
method Rewriting Schrödinger system as a fixed-point equation and estimating the potential via empirical risk minimization.
result Uniform concentration of empirical risk around population counterpart under sub-Gaussian assumptions.
This guide simplifies high-probability regret bounds in empirical risk minimization.
problem High-probability regret bounds in empirical risk minimization.
method Modular presentation, three-step recipe, localized Rademacher complexity, local maximal inequalities, metric-entropy integrals.
result Recover familiar rates for various function classes and derive regret bounds for nuisance components.
Estimates multiple means in high dimensions using convex combinations.
problem Estimating multiple multi-dimensional means from samples.
method Convex combinations of empirical means with data-dependent weights.
result Our methods asymptotically approach oracle (minimax) improvement.
We study the rates of convergence from empirical surrogate risk minimizers to the Bayes optimal classifier. Specifically, we introduce the notion of \emph{consistency intensity} to characterize a surrogate loss function and exploit this notion to obtain the rate of convergence from an empirical surrogate risk minimizer…
New approach avoids excess empirical risk in domain generalization.
problem Learning models that generalize to unseen distributions from diverse data sets.
method Minimizes penalty under constraint of optimal empirical risk, leveraging rate-distortion theory.
result Significant improvements in domain generalization performance across multiple methods.