Improved privacy for heavy-hitters detection.
problem Private detection of heavy-hitters in dynamic data.
method New algorithm for privately identifying heavy-hitters.
result Improved error guarantees over existing techniques.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “heavy hitters”
In many applications that involve processing high-dimensional data, it is important to identify a small set of entities that account for a significant fraction of detections. Rather than formalize this as a clustering problem, in which all detections must be grouped into hard or soft categories, we formalize it as an i…
Estimates heavy hitters in data streams with queries, balancing accuracy and efficiency.
problem Identifying elements with high probability in i.i.d. samples.
method Sequential estimation algorithms for two query models: index and pair queries.
result Upper and lower bounds on query complexity for different distributions and noise models.
A very simple event frequency approximation algorithm that is sensitive to event timeliness is suggested. The algorithm iteratively updates categorical click-distribution, producing (path of) a random walk on a standard -dimensional simplex. Under certain conditions, this random walk is self-similar and corresponds …
New sketches for weighted sampling without replacement improve accuracy and efficiency.
problem Efficiently sampling weighted data with high accuracy and minimal redundancy.
method Design of novel composable sketches for WOR sampling, based on CountSketch.
result First to provide WOR sampling for and signed updates.
Deconvoluting Kernel Density Estimation and Regression for Locally Differentially Private Datamath.ST
Paper tackles privacy-preserving data density issues using deconvolution.
problem Privacy-preserving noise affects data density, leading to under/over-estimation.
method Develops deconvoluting kernel density estimators and regression models.
result Demonstrates improved accuracy in estimating heavy-hitters with locally differential data.
Extracting information from electronic health records (EHR) is a challenging task since it requires prior knowledge of the reports and some natural language processing algorithm (NLP). With the growing number of EHR implementations, such knowledge is increasingly challenging to obtain in an efficient manner. We address…
Adaptive Threshold Samplingstat.ML
Sampling is a fundamental problem in computer science and statistics. However, for a given task and stream, it is often not possible to choose good sampling probabilities in advance. We derive a general framework for adaptively changing the sampling probabilities via a collection of thresholds.In general, adaptive samp…
Two novel search strategies reduce complexity for target localization with size-dependent noise.
problem Target localization with varying measurement noise based on query region size.
method Proposes and strategies with low complexity and connected query geometry.
result Unified analysis shows asymptotically optimal in search time, near-optimal in rate.
New algorithms handle heavy-tailed rewards in reinforcement learning.
problem Learning from heavy-tailed rewards in reinforcement learning.
method Robust mean estimation techniques for constructing algorithms.
result Near-optimal regret bounds achieved in heavy-tailed reward settings.
Study improves ERM for heavy-tailed data with dependent inputs.
problem Empirical Risk Minimization with dependent and heavy-tailed data.
method Extending risk bounds for ERM with heavy-tailed, dependent data.
result Established risk bounds for ERM with dependent and heavy-tailed data.
Heavy Lasso improves robustness in high-dimensional linear regression with heavy-tailed errors.
problem Challenges of classical Lasso in handling heavy-tailed noise and outliers.
method Data-augmented soft-thresholding with Student's t-distribution loss.
result Heavy Lasso achieves comparable rates to Huber loss under theoretical bounds.
Sparse Linear Regression when Noises and Covariates are Heavy-Tailed and Contaminated by Outliersstat.ML
Efficiently estimates sparse linear regression with heavy-tailed and outlier-contaminated data.
problem Estimating sparse linear regression coefficients with heavy-tailed and outlier-contaminated data.
method Efficient computation of estimators with sharp error bounds.
result Sharp error bounds for efficient estimators.
Is AdamW effective under heavy-tailed noise?
problem Stochastic gradient noise in LLM pretraining is typically heavy-tailed.
method Formulate as an open problem, prove a positive weighted-metric benchmark, and give a corridor lower-bound mechanism.
result No rigorous convergence theory for AdamW established in heavy-tailed regime.
Self-regulating annealing improves sampling from heavy-tailed datasets.
problem Sampling from heavy-tailed distributions using diffusion models.
method Proposed an SDE-based sampler with a state-dependent diffusion coefficient.
result State dependence induces a self-regulating annealing mechanism.
New diffusion models capture heavy-tailed distributions better.
problem Diffusion models struggle with rare or extreme events in heavy-tailed distributions.
method Repurposed diffusion framework using multivariate Student-t distributions, tailored perturbation kernel, and -divergence.
result Our models generate rare and extreme events more effectively than standard diffusion models.
New bounds link SGD's generalization to heavy tails without topological assumptions.
problem Linking SGD's generalization error to heavy tails without additional assumptions.
method Developed Wasserstein stability bounds for heavy-tailed SDEs and their discretizations, converting to generalization bounds.
result Generalization bounds for a broader class of objective functions, including non-convex functions, without topological assumptions.
New concentration inequalities for tensors with heavy-tailed coefficients.
problem Developing bounds for Euclidean functions of tensors with sub-Weibull distributions.
method Extending concentration inequalities to sub-Weibull random tensors, using new inequalities for heavy-tailed random variables and martingale analysis.
result Established a phase transition between sub-gaussian and heavy-tailed regimes for Euclidean functions of tensors.
New models explain heavy-tailed behavior in neural networks.
problem Heavy-tailed spectral densities in neural networks.
method High-temperature Marchenko-Pastur (HTMP) ensemble models.
result Heavy-tailed behavior arises from three factors: data structure, training temperature, and eigenvector entropy.
New PAC-Bayes bounds for heavy-tailed losses using supermartingales.
problem Extending PAC-Bayes bounds to heavy-tailed losses.
method Using supermartingales and bounded variance assumption.
result PAC-Bayes generalization bounds for heavy-tailed losses.
New bounds for heavy-tailed SDEs without info-theory terms.
problem Understanding generalization of heavy-tailed stochastic optimization.
method Fractional Fokker-Planck equation to estimate entropy flows.
result High-probability bounds with better dimension dependence.
PPO's gradients are heavy-tailed, affecting learning; a robust estimator improves performance.
problem Heavy-tailedness of PPO gradients causing learning issues.
method Characterized heavy-tailed gradients, identified likelihood ratios and advantages as sources, proposed GMOM as a robust estimator.
result GMOM improves PPO performance without clipping tricks.
Nonconvex optimization algorithms with random initialization have attracted increasing attention recently. It has been showed that many first-order methods always avoid saddle points with random starting points. In this paper, we answer a question: can the nonconvex heavy-ball algorithms with random initialization avoi…
Study on error probability for classification of heavy-tailed renewal processes.
problem Error probability in classification of heavy-tailed renewal processes.
method Asymptotic expressions for Bhattacharyya bound on misclassification error probabilities.
result Obtained asymptotic expressions for misclassification error probabilities.
We propose a new heavy-tailed distribution --- Gaussian-Chain (GC) distribution, which is inspirited by the hierarchical structures prevailing in social organizations. We determine the mean, variance and kurtosis of the Gaussian-Chain distribution to show its heavy-tailed property, and compute the tail distribution tab…
TTF improves performance of normalizing flows for heavy-tailed distributions.
problem Improving performance of normalizing flows for heavy-tailed distributions.
method Uses a Gaussian base distribution and a final transformation layer to produce heavy tails.
result Experimental results show TTF outperforms current methods, especially in high-dimensional or heavy-tailed scenarios.
HTFM improves mode coverage and tail-statistic recovery for heavy-tailed data.
problem Tackles heavy-tailed data in various domains with rare events.
method Proposes a framework using clock-conditioned Gaussian sources and truncated logsignature features.
result Improves mode coverage, sample quality, and tail-statistic recovery over Gaussian flow matching and baselines.
Heavy Ball method speeds up finding global optima in non-convex problems.
problem Finding global optima in non-convex optimization problems.
method Heavy Ball momentum in non-convex optimization.
result Heavy Ball helps iterates enter a benign region faster, containing a global optimal point.
Studied how heavy-tailed behavior affects SGD's generalization in quadratic optimization.
problem Link between heavy-tailed behavior and generalization in SGD.
method Used heavy-tailed stochastic differential equation and proved stability bounds.
result Stability of SGD depends on the loss function's tail behavior.
Study tail behavior of sum of heavy-tailed risks with copulas.
problem Analyzing the tail behavior of sums of heavy-tailed risks with dependence modeled by copulas.
method Modeling dependence with copulas and analyzing tail asymptotics of sums of heavy-tailed risks.
result Obtained asymptotic expansions for Value-at-Risk of aggregate risk.
New sampling method for heavy-tailed distributions using Langevin Algorithm.
problem Sampling from heavy-tailed distributions efficiently.
method Transformed Unadjusted Langevin Algorithm on specific transformations.
result Polynomial-order oracle complexities for certain heavy-tailed densities.
Heavy-tailed distributions emerge in SGD's parameter evolution.
problem Understanding heavy-tailed distributions in SGD parameter evolution.
method Continuous diffusion approximation of SGD (homogenized SGD) analysis.
result Explicit upper and lower bounds on tail-index of homogenized SGD.
Paper tackles robust offline RL with heavy-tailed rewards.
problem Real-world applications often encounter heavy-tailed rewards, challenging offline RL.
method Proposes ROAM and ROOM algorithms using median-of-means method for robust off-policy evaluation and OPO.
result Demonstrates superior performance on heavy-tailed reward datasets compared to existing methods.
Privacy-preserving SGD with heavy-tailed noise achieves differential privacy guarantees.
problem Privacy preservation in noisy SGD with heavy-tailed noise.
method Differential privacy guarantees for SGD with heavy-tailed noise.
result SGD with heavy-tailed perturbations achieves -DP.
Work on SGDm under heavy-tailed noise, revealing its generalization properties.
problem Understanding generalization of SGDm under heavy-tailed noise.
method Analysis of continuous-time limit (SDE) and discrete-time SGDm, establishing generalization bounds.
result SGDm can have worse generalization in the presence of heavy-tailed noise for quadratic loss functions.
Optimized method tackles convex optimization with heavy-tailed noise.
problem Convex optimization problems with noisy gradients.
method Vanilla stochastic proximal subgradient method without gradient clipping or normalization.
result Achieves optimal complexity for various convex optimization types under heavy-tailed noise.
Efficiently estimates sparse linear regression with heavy-tailed data and outliers.
problem Sparse estimation of linear regression coefficients with heavy-tailed covariates and noises, including outliers.
method Efficient computation of robust estimator with nearly optimal error bound.
result Nearly optimal error bound for robust sparse estimation.
This work compresses heavy-tailed weight matrices for tighter generalization bounds.
problem Empirical evidence linking heavy-tailed weight matrices to test set accuracy but lack of formal relationship with generalization bounds.
method Utilized the compression framework to show that heavy-tailed matrices can be compressed, resulting in sparse weight matrices.
result Demonstrated a non-vacuous generalization bound for compressed networks with heavy-tailed weight matrices.
Proposes a new distribution for robust time series modeling with heavy tails.
problem Robust modeling of time series with heavy-tailed noise.
method Spliced Binned-Pareto distribution for non-stationary time series.
result Accurately models extreme events and captures time dependencies in higher moments.
A simple log-transform fixes heavy-tailed data for generative models.
problem Standard generative models struggle with heavy-tailed data.
method Apply the soft-log transform to data before training and exponentiate samples after generation.
result Log-FM outperforms specialized baselines on multivariate benchmarks.
New class of heavy-tailed distributions shows weighted averages dominate individual variables.
problem Understanding and comparing risks in heavy-tailed distributions.
method Introducing a new class of heavy-tailed distributions and proving stochastic dominance relations.
result Weighted averages of random variables in this class are stochastically larger than individual variables.
New RDP guarantees for heavy-tailed SDEs and SGD.
problem Characterizing differential privacy for heavy-tailed noise in learning algorithms.
method Rényi flow computations and fractional Poincaré inequalities.
result First RDP guarantees for heavy-tailed SDEs with weaker dependence on dimension.
New algorithm tackles multi-agent bandits with heavy-tailed data.
problem Maximizing system performance in multi-agent settings with heavy-tailed data.
method Algorithm exploits hub-like structures and synchronization among clients.
result Regret bound of for homogeneous settings, for heterogeneous.
Machine Learning Panel Data Regressions with Heavy-tailed Dependent Data: Theory and Applicationecon.EM
The paper improves machine learning for heavy-tailed panel data.
problem Improving estimates for financial and economic data with fat tails.
method Sparse-group LASSO regularization and Fuk-Nagaev concentration inequality.
result Oracle inequalities for panel data estimators.
Heavy-tailed distributions are frequently used to enhance the robustness of regression and classification methods to outliers in output space. Often, however, we are confronted with "outliers" in input space, which are isolated observations in sparsely populated regions. We show that heavy-tailed stochastic processes (…
Probability Weighting Meets Heavy Tails: An Econometric Framework for Behavioral Asset Pricingq-fin.MF
Econometric framework integrates heavy-tailed distributions with behavioral probability weighting for better asset pricing.
problem Underestimation of Value-at-Risk by traditional models in asset pricing.
method Developed an econometric framework combining heavy-tailed Student's distributions with behavioral probability weighting.
result Student's specifications outperform Gaussian models in 88.4% of cases, reducing underestimation of Value-at-Risk by 16.5 percentage points.
Heavy-tailed regularization improves deep neural network performance.
problem Improving generalization of deep neural networks.
method Introducing Heavy-Tailed Regularization, using differentiable penalty terms and Bayesian statistics.
result Heavy-tailed regularization outperforms conventional regularization techniques.
Truncated SGD with heavy-tailed noise eliminates sharp local minima.
problem Avoiding sharp local minima in deep learning models.
method Truncated SGD with heavy-tailed gradient noise.
result Truncated SGD can eliminate sharp local minima entirely from its training trajectory.