Study tail behavior of sum of heavy-tailed risks with copulas.
problem Analyzing the tail behavior of sums of heavy-tailed risks with dependence modeled by copulas.
method Modeling dependence with copulas and analyzing tail asymptotics of sums of heavy-tailed risks.
result Obtained asymptotic expansions for Value-at-Risk of aggregate risk.
A simple log-transform fixes heavy-tailed data for generative models.
problem Standard generative models struggle with heavy-tailed data.
method Apply the soft-log transform to data before training and exponentiate samples after generation.
result Log-FM outperforms specialized baselines on multivariate benchmarks.
Study improves ERM for heavy-tailed data with dependent inputs.
problem Empirical Risk Minimization with dependent and heavy-tailed data.
method Extending risk bounds for ERM with heavy-tailed, dependent data.
result Established risk bounds for ERM with dependent and heavy-tailed data.
HTFM improves mode coverage and tail-statistic recovery for heavy-tailed data.
problem Tackles heavy-tailed data in various domains with rare events.
method Proposes a framework using clock-conditioned Gaussian sources and truncated logsignature features.
result Improves mode coverage, sample quality, and tail-statistic recovery over Gaussian flow matching and baselines.
TTF improves performance of normalizing flows for heavy-tailed distributions.
problem Improving performance of normalizing flows for heavy-tailed distributions.
method Uses a Gaussian base distribution and a final transformation layer to produce heavy tails.
result Experimental results show TTF outperforms current methods, especially in high-dimensional or heavy-tailed scenarios.
Self-regulating annealing improves sampling from heavy-tailed datasets.
problem Sampling from heavy-tailed distributions using diffusion models.
method Proposed an SDE-based sampler with a state-dependent diffusion coefficient.
result State dependence induces a self-regulating annealing mechanism.
The paper examines how heavy-tailed risks behave under Gaussian copula models.
problem Understanding tail risk probabilities with heavy-tailed marginal risks and Gaussian dependence.
method Modeling heavy-tailed risks using regular variation and analyzing tail probabilities under Gaussian copula.
result The rate of decay of tail set probabilities varies with the type of tail sets and Gaussian correlation matrix.
Efficiently estimates sparse linear regression with heavy-tailed and outlier-contaminated data.
problem Estimating sparse linear regression coefficients with heavy-tailed and outlier-contaminated data.
method Efficient computation of estimators with sharp error bounds.
result Sharp error bounds for efficient estimators.
Heavy-tailed distributions emerge in SGD's parameter evolution.
problem Understanding heavy-tailed distributions in SGD parameter evolution.
method Continuous diffusion approximation of SGD (homogenized SGD) analysis.
result Explicit upper and lower bounds on tail-index of homogenized SGD.
New diffusion models capture heavy-tailed distributions better.
problem Diffusion models struggle with rare or extreme events in heavy-tailed distributions.
method Repurposed diffusion framework using multivariate Student-t distributions, tailored perturbation kernel, and γ-divergence. result Our models generate rare and extreme events more effectively than standard diffusion models.
Is AdamW effective under heavy-tailed noise?
problem Stochastic gradient noise in LLM pretraining is typically heavy-tailed.
method Formulate as an open problem, prove a positive weighted-metric benchmark, and give a corridor lower-bound mechanism.
result No rigorous convergence theory for AdamW established in heavy-tailed regime.
New PAC-Bayes bounds for heavy-tailed losses using supermartingales.
problem Extending PAC-Bayes bounds to heavy-tailed losses.
method Using supermartingales and bounded variance assumption.
result PAC-Bayes generalization bounds for heavy-tailed losses.
New algorithms handle heavy-tailed rewards in reinforcement learning.
problem Learning from heavy-tailed rewards in reinforcement learning.
method Robust mean estimation techniques for constructing algorithms.
result Near-optimal regret bounds achieved in heavy-tailed reward settings.
New models explain heavy-tailed behavior in neural networks.
problem Heavy-tailed spectral densities in neural networks.
method High-temperature Marchenko-Pastur (HTMP) ensemble models.
result Heavy-tailed behavior arises from three factors: data structure, training temperature, and eigenvector entropy.
New bounds link SGD's generalization to heavy tails without topological assumptions.
problem Linking SGD's generalization error to heavy tails without additional assumptions.
method Developed Wasserstein stability bounds for heavy-tailed SDEs and their discretizations, converting to generalization bounds.
result Generalization bounds for a broader class of objective functions, including non-convex functions, without topological assumptions.
PH-VAE models heavy-tailed data with flexible Phase-Type distributions.
problem Standard VAEs fail to capture heavy-tailed behavior in real-world data.
method PH-VAE uses Phase-Type distributions defined by continuous-time Markov chains to adaptively model tail behavior.
result PH-VAE significantly outperforms existing heavy-tail-aware VAEs in approximating diverse heavy-tailed distributions.
Studied how heavy-tailed behavior affects SGD's generalization in quadratic optimization.
problem Link between heavy-tailed behavior and generalization in SGD.
method Used heavy-tailed stochastic differential equation and proved stability bounds.
result Stability of SGD depends on the loss function's tail behavior.
New concentration inequalities for tensors with heavy-tailed coefficients.
problem Developing bounds for Euclidean functions of tensors with sub-Weibull distributions.
method Extending concentration inequalities to sub-Weibull random tensors, using new inequalities for heavy-tailed random variables and martingale analysis.
result Established a phase transition between sub-gaussian and heavy-tailed regimes for Euclidean functions of tensors.
Privacy-preserving SGD with heavy-tailed noise achieves differential privacy guarantees.
problem Privacy preservation in noisy SGD with heavy-tailed noise.
method Differential privacy guarantees for SGD with heavy-tailed noise.
result SGD with heavy-tailed perturbations achieves (0,O(1/n))-DP. New bounds for heavy-tailed SDEs without info-theory terms.
problem Understanding generalization of heavy-tailed stochastic optimization.
method Fractional Fokker-Planck equation to estimate entropy flows.
result High-probability bounds with better dimension dependence.
Study on error probability for classification of heavy-tailed renewal processes.
problem Error probability in classification of heavy-tailed renewal processes.
method Asymptotic expressions for Bhattacharyya bound on misclassification error probabilities.
result Obtained asymptotic expressions for misclassification error probabilities.
We propose a new heavy-tailed distribution --- Gaussian-Chain (GC) distribution, which is inspirited by the hierarchical structures prevailing in social organizations. We determine the mean, variance and kurtosis of the Gaussian-Chain distribution to show its heavy-tailed property, and compute the tail distribution tab…
Proposes a new distribution for robust time series modeling with heavy tails.
problem Robust modeling of time series with heavy-tailed noise.
method Spliced Binned-Pareto distribution for non-stationary time series.
result Accurately models extreme events and captures time dependencies in higher moments.
Markov chain decoders improve generative models' ability to produce heavy-tailed data.
problem Generative models struggle with heavy-tailed distributions.
method Replaced Gaussian decoder with Markov chain-based Phase-Type distributions.
result Significantly reduced tail Kolmogorov-Smirnov distance and extreme quantile error.
Paper tackles robust batched bandits for heavy-tailed rewards.
problem Clinical trials and other applications with heavy-tailed rewards.
method Proposes robust batched bandit algorithms for heavy-tailed rewards in finite-arm and Lipschitz-continuous settings.
result Heavier-tailed rewards require fewer batches for near-optimal regret in the instance-independent regime and Lipschitz setting.
The paper improves machine learning for heavy-tailed panel data.
problem Improving estimates for financial and economic data with fat tails.
method Sparse-group LASSO regularization and Fuk-Nagaev concentration inequality.
result Oracle inequalities for panel data estimators.
New framework controls generalization for heavy-tailed data in RLHF and SGLD.
problem Heavy-tailed data in modern learning pipelines.
method Tail-dependent information-theoretic framework for sub-Weibull data.
result Sharp generalization bounds for heavy-tailed data.
New RDP guarantees for heavy-tailed SDEs and SGD.
problem Characterizing differential privacy for heavy-tailed noise in learning algorithms.
method Rényi flow computations and fractional Poincaré inequalities.
result First RDP guarantees for heavy-tailed SDEs with weaker dependence on dimension.
New sampling method for heavy-tailed distributions using Langevin Algorithm.
problem Sampling from heavy-tailed distributions efficiently.
method Transformed Unadjusted Langevin Algorithm on specific transformations.
result Polynomial-order oracle complexities for certain heavy-tailed densities.
New algorithm tackles multi-agent bandits with heavy-tailed data.
problem Maximizing system performance in multi-agent settings with heavy-tailed data.
method Algorithm exploits hub-like structures and synchronization among clients.
result Regret bound of O(M1−α1logT) for homogeneous settings, O(MlogT) for heterogeneous. Study heavy-tailed weights' impact on neural network's spectral distribution.
problem Analyzing spectral distribution of conjugate kernel matrices with heavy-tailed weights.
method Computed limiting eigenvalue distribution through moments, considering heavy-tailed distributions and nonlinear activation functions.
result Heavy-tailed weights induce strong correlations, leading to fundamentally different spectral behavior.
Paper tackles robust offline RL with heavy-tailed rewards.
problem Real-world applications often encounter heavy-tailed rewards, challenging offline RL.
method Proposes ROAM and ROOM algorithms using median-of-means method for robust off-policy evaluation and OPO.
result Demonstrates superior performance on heavy-tailed reward datasets compared to existing methods.
Econometric framework integrates heavy-tailed distributions with behavioral probability weighting for better asset pricing.
problem Underestimation of Value-at-Risk by traditional models in asset pricing.
method Developed an econometric framework combining heavy-tailed Student's t distributions with behavioral probability weighting. result Student's t specifications outperform Gaussian models in 88.4% of cases, reducing underestimation of Value-at-Risk by 16.5 percentage points. Work on SGDm under heavy-tailed noise, revealing its generalization properties.
problem Understanding generalization of SGDm under heavy-tailed noise.
method Analysis of continuous-time limit (SDE) and discrete-time SGDm, establishing generalization bounds.
result SGDm can have worse generalization in the presence of heavy-tailed noise for quadratic loss functions.
Optimized method tackles convex optimization with heavy-tailed noise.
problem Convex optimization problems with noisy gradients.
method Vanilla stochastic proximal subgradient method without gradient clipping or normalization.
result Achieves optimal complexity for various convex optimization types under heavy-tailed noise.
Efficiently estimates sparse linear regression with heavy-tailed data and outliers.
problem Sparse estimation of linear regression coefficients with heavy-tailed covariates and noises, including outliers.
method Efficient computation of robust estimator with nearly optimal error bound.
result Nearly optimal error bound for robust sparse estimation.
The paper investigates heavy-tailed behavior in offline SGD, showing it approximates power-law tails.
problem Understanding heavy-tailed behavior in offline (multi-pass) SGD with finite data.
method Proves nonasymptotic Wasserstein convergence bounds for offline SGD to online SGD.
result Offline SGD exhibits approximate power-law tails as the number of data points increases.
This work compresses heavy-tailed weight matrices for tighter generalization bounds.
problem Empirical evidence linking heavy-tailed weight matrices to test set accuracy but lack of formal relationship with generalization bounds.
method Utilized the compression framework to show that heavy-tailed matrices can be compressed, resulting in sparse weight matrices.
result Demonstrated a non-vacuous generalization bound for compressed networks with heavy-tailed weight matrices.
This work extends diffusion models to handle heavy-tailed targets, improving score estimation and sampling guarantees.
problem Score estimation and sampling guarantees for heavy-tailed targets in diffusion models.
method Kernel density estimation and minimax rates analysis for score estimation and sampling guarantees.
result Sharp minimax rates for score estimation and sampling guarantees for heavy-tailed targets, revealing qualitative differences between exponential and polynomial tails.
New class of heavy-tailed distributions shows weighted averages dominate individual variables.
problem Understanding and comparing risks in heavy-tailed distributions.
method Introducing a new class of heavy-tailed distributions and proving stochastic dominance relations.
result Weighted averages of random variables in this class are stochastically larger than individual variables.
Diversification improves profits for heavy-tailed investments.
problem Investment portfolios of Pareto-distributed returns.
method Stochastic dominance and majorization order.
result Diversification increases first-order stochastic dominance for heavy-tailed returns.
Heavy-tailed distributions are frequently used to enhance the robustness of regression and classification methods to outliers in output space. Often, however, we are confronted with "outliers" in input space, which are isolated observations in sparsely populated regions. We show that heavy-tailed stochastic processes (…
A hybrid model combines BPH and HE distributions for better heavy-tailed distribution approximation.
problem Accurate modeling of heavy-tailed distributions in various applications.
method A hybrid model of Bernstein phase-type and hyperexponential distributions with optimized parameters.
result Significant improvement in capturing both body and tail of heavy-tailed distributions.
COMET Flows model multivariate extremes with heavy tails and asymmetric dependence.
problem Normalizing flows struggle with multivariate extremes and asymmetric tail dependence.
method COMET Flows decomposes modeling into marginal and copula parts; uses tail belief and kernel density for marginals, and low-dimensional manifold for tail dependence.
result COMET Flows outperform other models in capturing heavy-tailed marginals and asymmetric tail dependence.
Heavy-tailed regularization improves deep neural network performance.
problem Improving generalization of deep neural networks.
method Introducing Heavy-Tailed Regularization, using differentiable penalty terms and Bayesian statistics.
result Heavy-tailed regularization outperforms conventional regularization techniques.
Truncated SGD with heavy-tailed noise eliminates sharp local minima.
problem Avoiding sharp local minima in deep learning models.
method Truncated SGD with heavy-tailed gradient noise.
result Truncated SGD can eliminate sharp local minima entirely from its training trajectory.
Study on U-statistics with heavy-tailed samples, providing tail bounds and LDP.
problem Deviation of U-statistics with heavy-tailed samples.
method Exponential tail bounds and Large Deviation Principle (LDP) for U-statistics.
result Obtained an exponential upper bound for U-statistics tail decay, showing two regions of decay.
New algorithm reduces worst-case regret for heavy-tailed bandits.
problem Stochastic Multi-Armed Bandit problem with heavy-tailed rewards.
method Modified minimax policy MOSS with saturated empirical mean.
result Worst-case regret matching lower bound for heavy-tailed distributions.