New model captures time-varying volatility with stochastic exponential tails.
problem Capturing time-varying volatility and stochastic skewness in financial markets.
method Normal Tempered Stable distribution with time-varying parameter.
result Model better explains market option prices with stochastic exponential tails.
Paper provides tail bounds for stochastic mirror descent in heavy-tailed noise.
problem Optimizing convex and Lipschitz functions with heavy-tailed noise.
method Develops tail bounds for optimization error of Stochastic Mirror Descent.
result Tail bounds extend to heavier-tailed noise regimes without diameter constraints.
New algorithm improves heavy-tailed statistical estimation in streaming data.
problem Heavy-tailed statistical estimation in streaming data.
method Clipped stochastic gradient descent algorithm with improved analysis.
result Guarantees exponential concentration with O(1) batch size for mean estimation and linear regression. We propose a stochastic process driven by the memory effect with novel distributions which include both exponential and leptokurtic heavy-tailed distributions. A class of the distributions is analytically derived from the continuum limit of the discrete binary process with the renormalized auto-correlation. The moment …
We present an empirical study of the subordination hypothesis for a stochastic time series of a stock price. The fluctuating rate of trading is identified with the stochastic variance of the stock price, as in the continuous-time random walk (CTRW) framework. The probability distribution of the stock price changes (log…
We propose a stochastic process driven by memory effect with novel distributions including both exponential and leptokurtic heavy-tailed distributions. A class of distribution is analytically derived from the continuum limit of the discrete binary process with the renormalized auto-correlation and the closed form momen…
Stochastic approximation algorithms show exponential progress bounds.
problem Analyzing the convergence of stochastic approximation algorithms.
method Developed geometric ergodicity proofs to establish exponential concentration bounds.
result Proved faster convergence rates for specific algorithms.
We compare systematically several classes of stochastic volatility models of stock market fluctuations. We show that the long-time return distribution is either Gaussian or develops a power-law tail, while the short-time return distribution has generically a stretched-exponential form, but can assume also an algebraic …
Optimizes regret distribution in stochastic bandits for risk balance.
problem Balancing regret expectation and tail risk in stochastic bandits.
method Characterizes optimal regret tail probability for any threshold, proposes new policies.
result Discovers an intrinsic gap in optimal tail rate based on time horizon uncertainty.
We consider strictly stationary heavy tailed time series whose finite-dimensional exponent measures are concentrated on axes, and hence their extremal properties cannot be tackled using classical multivariate regular variation that is suitable for time series with extremal dependence. We recover relevant information ab…
Study on U-statistics with heavy-tailed samples, providing tail bounds and LDP.
problem Deviation of U-statistics with heavy-tailed samples.
method Exponential tail bounds and Large Deviation Principle (LDP) for U-statistics.
result Obtained an exponential upper bound for U-statistics tail decay, showing two regions of decay.
We study the probability distribution of stock returns at mesoscopic time lags (return horizons) ranging from about an hour to about a month. While at shorter microscopic time lags the distribution has power-law tails, for mesoscopic times the bulk of the distribution (more than 99% of the probability) follows an expon…
New framework controls generalization for heavy-tailed data in RLHF and SGLD.
problem Heavy-tailed data in modern learning pipelines.
method Tail-dependent information-theoretic framework for sub-Weibull data.
result Sharp generalization bounds for heavy-tailed data.
Paper develops a TR-SSQP method for noisy optimization with heavy-tailed noise.
problem Optimization problems with stochastic objectives and heavy-tailed noise.
method Trust-Region Stochastic Sequential Quadratic Programming (TR-SSQP) method.
result Achieves high-probability first-order and second-order stationarity bounds for heavy-tailed noise.
We study tilting subweibull distributions and their tail behavior.
problem Understanding tail behavior of subweibull distributions.
method Alternative characterizations and conditions for tail behavior preservation.
result Conditions for tail behavior preservation after exponential tilting.
Paper develops robust estimators and strategies for stochastic MABs with heavy-tailed rewards.
problem Stochastic multi-armed bandits with heavy-tailed rewards.
method Proposes a novel robust estimator and perturbation-based exploration strategy.
result Develops upper and lower regret bounds for various perturbations.
Optimal algorithm for minimizing regret in heavy-tailed bandits.
problem Minimizing regret in stochastic multi-armed bandits with heavy-tailed distributions.
method Proposes an optimal algorithm under the assumption of uniformly bounded moments of order (1+ε).
result Matches the lower bound exactly in the first-order term and provides a finite-time bound on its regret.
We improve bounds for stochastic processes, especially those with heavy tails.
problem Bounding the concentration of sub-ψ processes with heavy tails. method Variational approach to concentration, focusing on sub-Gaussian and other tail conditions.
result First dimension-free self-normalized empirical Bernstein inequality.
We consider uncorrelated Stein-Stein, Heston, and Hull-White models and their perturbations by compound Poisson processes with jump amplitudes distributed according to a double exponential law. Similar perturbations of the Black-Scholes model were studied by S. Kou. For perturbed stochastic volatility models, we obtain…
We introduce a stochastic model to explain a double power-law distribution which exhibits two different Paretian behaviors in the upper and the lower tail and widely exists in social and economic systems. The model incorporates fitness consideration and noise fluctuation. We find that if the number of variables (e.g. t…
This work achieves exponential concentration in heavy-tailed data over CAT(κ) spaces using the Fréchet median.
problem Achieving robust estimation in heavy-tailed data distributions.
method Developing a concentration bound for the Fréchet median in CAT(κ) spaces.
result Exponential concentration of the Fréchet median in CAT(κ) spaces over heavy-tailed data.
Exponential Lasso improves Lasso's robustness to outliers and heavy-tailed noise.
problem Lasso's sensitivity to outliers and heavy-tailed noise in high-dimensional statistics.
method Integrates an exponential-type loss function into the Lasso framework.
result Achieves strong statistical convergence rates robust to heavy-tailed contamination.
This work extends diffusion models to handle heavy-tailed targets, improving score estimation and sampling guarantees.
problem Score estimation and sampling guarantees for heavy-tailed targets in diffusion models.
method Kernel density estimation and minimax rates analysis for score estimation and sampling guarantees.
result Sharp minimax rates for score estimation and sampling guarantees for heavy-tailed targets, revealing qualitative differences between exponential and polynomial tails.
New insights into natural exponential families improve regret bounds for bandit problems.
problem Improving regret bounds for bandit problems with subexponential tails.
method Proving self-concordance for natural exponential families and applying to bandits.
result Optimistic algorithms for generalized linear bandits have second-order regret bounds that are free of an exponential dependence on problem parameters.
New algorithms achieve high-probability parameter-free regret in online convex optimization with heavy-tailed data.
problem Achieving high-probability parameter-free regret in online convex optimization with heavy-tailed data.
method Developed new regularization techniques to handle exponentially large iterates and heavy-tailed subgradients.
result Achieved regret bound of O(∥u∥T1/plog(1/δ)) with high probability for subgradients with bounded pth moments. Study differentially private linear regression with heavy-tailed data.
problem Differentially private ℓ1-norm linear regression with heavy-tailed data. method Exponential mechanism for ℓ2-norm bounded second moment; relaxation to ℓ2-norm bounded θ-th moment; coordinate-wise bounded moments. result Achieved upper bounds for privacy-preserving linear regression under various moment conditions.
Density expansions for hypoelliptic diffusions (X1,...,Xd) are revisited. In particular, we are interested in density expansions of the projection (XT1,...,XTl), at time T>0, with l≤d. Global conditions are found which replace the well-known "not-in-cutlocus" condition known from heat-kernel asymptot…
Gradient descent implicitly follows regularization for general losses.
problem The implicit bias of gradient descent methods in machine learning.
method Empirical risk minimization over linear predictors with arbitrary convex, strictly decreasing losses.
result Gradient descent and regularization paths converge to the same direction for non-attained risks.
A simple log-transform fixes heavy-tailed data for generative models.
problem Standard generative models struggle with heavy-tailed data.
method Apply the soft-log transform to data before training and exponentiate samples after generation.
result Log-FM outperforms specialized baselines on multivariate benchmarks.
New algorithms improve stopping time for best arm identification.
problem Efficiently identifying the best alternative in experiments.
method Proposed algorithms with exponential-tailed stopping time.
result Proved that some algorithms never stop, leading to new methods.
Modified lognormal distribution with flexible tails for skewed data.
problem Skewed and fat-tailed data in natural and engineering datasets.
method Developed a family of three-parameter non-Gaussian probability density functions based on generalized kappa-exponential and kappa-logarithm functions.
result Closed-form analytic expressions for statistical functions and maximum-likelihood estimation.
Study improves ERM for heavy-tailed data with dependent inputs.
problem Empirical Risk Minimization with dependent and heavy-tailed data.
method Extending risk bounds for ERM with heavy-tailed, dependent data.
result Established risk bounds for ERM with dependent and heavy-tailed data.
Paper establishes convergence rates and concentration bounds for stochastic approximation and reinforcement learning with Markovian noise.
problem Analyzing convergence rates and concentration bounds for stochastic approximation and reinforcement learning with Markovian noise.
method Novel discretization of the mean ODE of stochastic approximation algorithms using intervals with diminishing length.
result First almost sure convergence rate and maximal concentration bound with exponential tails for contractive stochastic approximation algorithms with Markovian noise.
New ensemble method improves model stability exponentially.
problem Improving model stability for discontinuous base learners.
method Selecting the most frequently generated model from subsamples.
result Exponentially decaying tails for excess risk.
Heavy-tailed distributions emerge in SGD's parameter evolution.
problem Understanding heavy-tailed distributions in SGD parameter evolution.
method Continuous diffusion approximation of SGD (homogenized SGD) analysis.
result Explicit upper and lower bounds on tail-index of homogenized SGD.
New method for Bayesian inference of Lévy-driven SDEs with jumps.
problem Bayesian inference for Lévy-driven SDEs is challenging due to discontinuities and heavy tails.
method Neural exponential tilting framework for variational inference.
result Accurately captures jump dynamics and reliable posterior inference in heavy-tailed regimes.
Stochastic gradient methods can converge in expectation under heavy-tailed noise.
problem Convergence of stochastic gradient methods under heavy-tailed noise.
method Comprehensive study of stochastic optimization under heavy-tailed noise for extsfSGD, extsfSMD, extsfASMD, extsfSGDM in convex and nonconvex optimization. result Established in-expectation convergence results for various stochastic gradient methods.
New tail dependence measures for stock indices.
problem Measuring tail dependence between financial variables.
method Introducing a new stochastic order and studying monotone tail dependence measures.
result Advantage of new tail dependence measures over classical ones.
A fast method for training linear classifiers maximizes margins.
problem Training linear classifiers with maximum margins.
method Momentum-based gradient method derived from convex dual with Nesterov acceleration.
result Exponentially faster convergence rate compared to standard methods.
Optimized method tackles convex optimization with heavy-tailed noise.
problem Convex optimization problems with noisy gradients.
method Vanilla stochastic proximal subgradient method without gradient clipping or normalization.
result Achieves optimal complexity for various convex optimization types under heavy-tailed noise.
We introduce a new statistical tool (the TP-statistic and TE-statistic) designed specifically to compare the behavior of the sample tail of distributions with power-law and exponential tails as a function of the lower threshold u. One important property of these statistics is that they converge to zero for power laws o…
New class of heavy-tailed distributions shows weighted averages dominate individual variables.
problem Understanding and comparing risks in heavy-tailed distributions.
method Introducing a new class of heavy-tailed distributions and proving stochastic dominance relations.
result Weighted averages of random variables in this class are stochastically larger than individual variables.
We consider a priori generalization bounds developed in terms of cross-validation estimates and the stability of learners. In particular, we first derive an exponential Efron-Stein type tail inequality for the concentration of a general function of n independent random variables. Next, under some reasonable notion of s…
We analyze systems of agents sharing light-tailed risky claims issued by different financial objects. Assuming exponentially distributed claims, we obtain that both agents' and system's losses follow generalized exponential mixture distributions. We show that this leads to qualitatively different results on individual …
Paper examines the structure of stochastic gradients in deep learning.
problem Exploring the structure and heavy tails of stochastic gradients in deep learning.
method Conducted formal statistical tests on stochastic gradients and gradient noise.
result Stochastic gradients and gradient noise do not exhibit power-law heavy tails, but their covariance spectra do.
Stochastic optimization's success linked to heavy-tailed noise.
problem Understanding stochastic optimization's success mechanisms.
method Modeling stochastic optimization as random recurrence relations, analyzing multiplicative noise and heavy-tailed behavior.
result Multiplicative noise leads to heavy-tailed stationary behavior in optimization parameters.
Researchers study heavy-tail properties of SGD using stochastic recurrence equations.
problem Analyzing heavy-tail properties of Stochastic Gradient Descent (SGD).
method Modeling SGD iterations as multivariate affine stochastic recursions and applying the theory of irreducible-proximal (i-p) matrices.
result Extended results of Gürbüzbalaban et al. (2020) by using the theory of i-p matrices.
New method bounds stochastic subgradient methods with heavy-tailed noise.
problem Bounding stochastic subgradient methods under heavy-tailed noise.
method Clipped version of projected stochastic subgradient method.
result Near optimal any-time and finite horizon bounds for averaging schemes.