Study non-asymptotic bounds for robust estimators under misspecified models.
problem Evaluate performance of robust estimators under adversarial conditions.
method Propose a general approach to adversarial risk analysis, including investigations on generalization and approximation errors.
result Establish non-asymptotic upper bounds for adversarial excess risk under Lipschitz loss functions.
The paper analyzes the excess risk of PCA and provides a precise characterization.
problem Understanding the excess risk of principal component analysis (PCA).
method Established a central limit theorem for PCA error and derived the excess risk distribution.
result Obtained a non-asymptotic upper bound on the excess risk of PCA.
Paper analyzes SGHMC for non-convex optimization with discontinuous gradients.
problem Training neural networks with ReLU activation.
method Non-asymptotic convergence analysis of SGHMC with discontinuous gradients.
result Explicit upper bounds for expected excess risk in non-convex optimization.
We consider a high dimensional binary classification problem and construct a classification procedure by minimizing the empirical misclassification risk with a penalty on the number of selected features. We derive non-asymptotic probability bounds on the estimated sparsity as well as on the excess misclassification ris…
Study optimizes prediction error for growing-dimensional PFLM models.
problem Optimizing prediction error for growing-dimensional PFLM models.
method Penalized least-squares approach in RKHS with effective dimension consideration.
result Shows exact upper bound for excess prediction risk in non-asymptotic form.
Improves statistical learning bounds with self-concordant losses.
problem Statistical prediction with nuisance components.
method Orthogonal statistical learning with self-concordant loss.
result Non-asymptotic bounds on excess risk improved by a dimension factor.
New Langevin algorithm works well even for rough distributions.
problem Sampling from non-smooth distributions.
method Simple Langevin algorithm without smoothness assumptions.
result Algorithm performs well even with discontinuous gradients.
In this paper, we are concerned with a non-asymptotic analysis of sampling algorithms used in nonconvex optimization. In particular, we obtain non-asymptotic estimates in Wasserstein-1 and Wasserstein-2 distances for a popular class of algorithms called Stochastic Gradient Langevin Dynamics (SGLD). In addition, the afo…
The paper provides theoretical guarantees for neural network-based anomaly detection.
problem Theoretical guarantees for unsupervised neural network-based anomaly detection.
method Casting anomaly detection as a binary classification problem, establishing non-asymptotic upper bounds and convergence rates.
result The convergence rate on the excess risk matches the minimax optimal rate.
SMP estimator improves density and logistic regression under misspecification.
problem Improper estimator for optimal excess risk in misspecified models.
method SMP minimizes a new excess risk bound for statistical learning.
result SMP achieves optimal excess risk of O((d+B2R2)/n) for logistic regression. We provide non-asymptotic excess risk guarantees for statistical learning in a setting where the population risk with respect to which we evaluate the target parameter depends on an unknown nuisance parameter that must be estimated from data. We analyze a two-stage sample splitting meta-algorithm that takes as input ar…
ERM performs well in feature learning with minimal feature maps.
problem Empirical risk minimization in feature learning with square loss.
method Asymptotic and non-asymptotic analysis of ERM performance.
result Excess risk quantiles of ERM match those of oracle procedure under certain conditions.
Proposes RaT to mitigate bias in student-teacher estimation.
problem Systematic bias in teacher's predictions propagates to student model.
method Uses teacher to estimate residuals in student's predictions.
result RaT method reduces teacher bias effect and achieves optimal rate.
Framework for domain adaptation using pseudo-labels from unlabeled data.
problem Improving prediction accuracy in target domain with covariate shift.
method Kernel GLMs with labeled and pseudo-labeled data, using imputation model for target data.
result Non-asymptotic excess-risk bounds for effective labeled sample size.
Deep neural networks classify unbounded Gaussian mixture data without dimensionality issues.
problem Binary classification of unbounded Gaussian mixture data.
method Deep ReLU neural networks with non-asymptotic upper bounds and convergence rates.
result Deep ReLU networks can classify unbounded Gaussian mixture data without dimensionality constraints.
This paper investigates WDRO for nonparametric regression, achieving robustness against distributional uncertainty.
problem Addressing model misspecification in nonparametric regression under distributional uncertainty.
method Wasserstein distributionally robust optimization (WDRO) with structural distinction based on Wasserstein distance order.
result Achieves a convergence rate of n−2β/(d+2β) up to logarithmic factors, showing minimax optimality. A new tamed stochastic gradient Hamiltonian Monte Carlo algorithm for superlinearly growing stochastic gradients.
problem Sampling and stochastic optimization problems with superlinearly growing stochastic gradients.
method Tamed Stochastic Gradient Hamiltonian Monte Carlo (tSGHMC) algorithm.
result Established a non-asymptotic error bound in Wasserstein-2 distance with a convergence rate of 1/4. Maximum likelihood estimator performance in logistic regression analyzed.
problem Performance of maximum likelihood estimator in logistic regression.
method Sharp non-asymptotic guarantees for existence and excess logistic risk.
result Sharp guarantees for the existence and excess risk of MLE in logistic regression.
Paper tackles robust deep learning from weakly dependent data with unbounded loss and input.
problem Tackles robust deep learning from weakly dependent data with unbounded loss and input.
method Establishes non-asymptotic bounds for expected excess risk under strong mixing and ψ-weak dependence assumptions. result Derives a relationship between bounds and r, and shows convergence rate close to i.i.d. results for r=∞. New method bounds high-dimensional regression without estimating design covariance.
problem High-dimensional linear regression with random design.
method Error-in-operator approach that incorporates design covariance into empirical risk minimization.
result Dimension-free bounds on excess prediction risk derived.
This work characterizes the benefits of averaging schemes widely used in conjunction with stochastic gradient descent (SGD). In particular, this work provides a sharp analysis of: (1) mini-batching, a method of averaging many samples of a stochastic gradient to both reduce the variance of the stochastic gradient estima…
Paper establishes bounds for RNN-TPPs, showing four-layer networks can achieve vanishing errors.
problem Understanding theoretical limits of RNN-TPPs.
method Characterized RNN complexity, constructed neural approximations, applied truncation technique.
result Four-layer RNN-TPPs can achieve vanishing generalization errors.
Deep neural networks enforce non-crossing quantile regression curves.
problem Estimating quantile regression curves without crossing.
method Penalized deep ReQU neural networks with a non-crossing penalty.
result Established non-asymptotic risk and error bounds for the estimated QRP.
New algorithms sample from log concave distributions without gradient Lipschitz continuity.
problem Sampling from log concave distributions without gradient Lipschitz continuity.
method Two algorithms based on monotone polygonal (tamed) Euler schemes.
result Non-asymptotic 2-Wasserstein distance bounds between the process and target measure.
The paper develops a deep neural network estimator for weakly dependent processes with various loss functions.
problem Learning weakly dependent processes with a broad class of loss functions.
method Sparse-penalized deep neural networks with ψ-weak dependence structure and θ∞-coefficients. result Oracle inequalities for the excess risk of the sparse-penalized deep neural networks estimators.
In statistical learning theory, convex surrogates of the 0-1 loss are highly preferred because of the computational and theoretical virtues that convexity brings in. This is of more importance if we consider smooth surrogates as witnessed by the fact that the smoothness is further beneficial both computationally- by at…
New tool detects 'fleeting modes' causing excess risk in financial markets.
problem Detecting portfolios with statistically significant excess risk in financial markets.
method Random Matrix Theory to identify 'fleeting modes' independent of underlying correlation structure.
result Fleeting modes exist in both futures and equity markets, and momentum is a source of excess risk.
The paper explores the information-theoretic nature of excess risk in machine learning.
problem Understanding the excess risk in machine learning models.
method Formulates the minimax excess risk as a zero-sum game and modifies it to allow swapping of the order of play.
result Proves that under certain conditions, the duality gap is zero, allowing for the application of Bayesian results to provide bounds on minimax excess risk.
Paper approximates risk measures using SGD with Langevin dynamics.
problem Approximating arbitrary law invariant risk measures.
method Stochastic Gradient Langevin Dynamics (SGD-Langevin) for general risk measures.
result Non-asymptotic convergence rates of the approximation algorithm.
Study reveals benign overfitting in time series models with over-parameterization.
problem Analyzing over-parameterized linear models with dependent time-series data.
method Developed an estimator using interpolation and derived non-asymptotic risk bounds.
result Risk bound is influenced by the coherence of temporal covariance matrices at different time steps.
New bounds for linear interpolators show how they generalize under covariate shifts.
problem Understanding how linear interpolators generalize under covariate shifts.
method Proved non-asymptotic excess risk bounds for benignly-overfit linear interpolators in transfer learning.
result Identified beneficial and malignant covariate shifts based on overparameterization degree.
We consider high-dimensional binary classification by sparse logistic regression. We propose a model/feature selection procedure based on penalized maximum likelihood with a complexity penalty on the model size and derive the non-asymptotic bounds for the resulting misclassification excess risk. The bounds can be reduc…
Study excess risk in statistical inference with transformations.
problem Excess risk in estimating random variables from feature vectors and transformations.
method Characterize lossless transformations, develop test statistics, and information-theoretic bounds.
result Strongly consistent partitioning test statistic for lossless transformations.
Optimizes reward learning design for complex tasks using nonparametric methods.
problem Challenges in specifying reward functions for complex tasks.
method Models rewards and policies as nonparametric functions in RKHSs, derives risk bounds, and optimizes query design.
result Derives non-asymptotic excess risk bounds and finite sample statistical rates for reward learning.
Optimizes shortfall risk using gradient-based methods.
problem Optimizing utility-based shortfall risk measures.
method Gradient-based stochastic optimization, non-asymptotic bounds derivation.
result Non-asymptotic convergence rate for optimizing UBSR.
This paper establishes non-asymptotic learning bounds for the DR covariate shift adaptation.
problem Distribution shift between training and test domains in machine learning.
method Doubly-robust (DR) estimator combining density ratio estimation and pilot regression model.
result First non-asymptotic learning bounds for DR covariate shift adaptation.
TUSLA algorithm solves non-convex optimization problems with ReLU activations.
problem Non-convex stochastic optimization with super-linearly growing and discontinuous gradients.
method Non-asymptotic analysis of TUSLA algorithm for non-convex learning.
result TUSLA provides non-asymptotic error bounds in Wasserstein distances for non-convex learning.
Paper improves clustering risk bounds for kernel k-means.
problem Improving clustering risk bounds for kernel k-means.
method Analyzes kernel k-means and Nyström approximation.
result Achieves nearly optimal excess clustering risk bound.
The study finds a trade-off between model size, test loss, and training loss for linear predictors.
problem Finding the optimal balance between model size, test loss, and training loss for linear predictors.
method Established an algorithm and distribution-independent trade-off using non-asymptotic analysis.
result Models with low test loss are either classical (close to noise level training loss) or modern (large number of parameters).
Proposes a non-crossing deep neural network quantile regression method.
problem Quantile crossing in nonparametric quantile regression.
method Non-crossing constraints via rectified linear unit penalty function.
result Established non-asymptotic upper bounds for excess risk.
New method estimates model risk without knowing function class.
problem Evaluating model risk for complex, opaque models.
method Wild refitting with Bregman losses and randomized symmetrization.
result Valid upper bound on excess risk for opaque models.
Estimates and optimizes UBSR risk in recursive settings.
problem Estimating and optimizing UBSR risk in a recursive setting with one-at-a-time samples.
method Casts UBSR as a root finding problem, uses stochastic approximation and gradient descent.
result Derives non-asymptotic bounds on estimation and optimization errors.
New schemes improve error estimates for sampling from non-log-concave distributions.
problem Improving sampling from non-log-concave distributions with super-linear drift growth.
method Developed tamed Euler and randomized Euler schemes with error estimates.
result Near-optimal error bounds for sampling and optimization problems.
Local SGD proves efficient in overparameterized linear regression.
problem Efficiently learning overparameterized linear models in distributed settings.
method Distributed SGD (DSGD) with overparameterized models.
result Excess risk of SGD is smaller than ridge regression in the same sample complexity.
Study finds high cyber risk stocks generate significant excess returns.
problem Understanding and quantifying cyber risk's impact on stock returns.
method Machine learning algorithm measuring cyber risk proximity to a corpus.
result High cyber risk stocks generate an excess return of 18.72% p.a.
New method improves training of PINNs for PDEs by adding noisy supervision terms.
problem Slow or failed convergence of PINNs on challenging PDEs.
method Operator preconditioning using Feynman-Kac supervision and non-asymptotic error bounds.
result Non-asymptotic error bounds for FK-PINNs, showing improved performance over standard PINNs.
Study uses spectral risk for learning with heavy-tailed data.
problem Learning with heavy-tailed loss distributions.
method Spectral risk with Lipschitz-continuous density, derivative-free learning.
result Excess risk guarantees and improved performance over traditional methods.
New algorithm tackles optimization problems with discontinuous gradients in finance and insurance.
problem Optimization problems with discontinuous stochastic gradients in finance and insurance.
method Langevin dynamics based algorithm e-THεO POULA. result Non-asymptotic error bounds and expected excess risk estimates for e-THεO POULA.