A new method for exponentially weighted moving models using approximations.
problem Efficiently updating moving averages for time series data.
method Approximates EWMM using a fixed window and quadratic term, solving non-growing problems.
result Approximation produces estimates similar to exact EWMM.
We introduce a covariance matrix estimator that both takes into account the heteroskedasticity of financial returns (by using an exponentially weighted moving average) and reduces the effective dimensionality of the estimation (and hence measurement noise) via techniques borrowed from random matrix theory. We calculate…
A new accelerated method with simpler momentum update rules.
problem Optimizing parameters in machine learning models.
method Proposes a novel accelerated stochastic gradient method with simpler momentum update rules.
result The method outperforms Sgdm and Adam in practical problems.
Extends online learning to metric spaces using exponential weights.
problem Online learning in metric spaces.
method Exponentially weighted average forecaster, barycenters, Jensen's inequality, measure contraction property.
result Results in a statistical learning framework.
The paper extends a prediction method to curved spaces.
problem Prediction with expert advice in curved spaces.
method Geometric considerations and adaptation of online to batch conversion.
result Extension of exponentially weighted average forecaster to non-positive curvature spaces.
We propose an explicit recursive method to approximate a power-law with a finite sum of weighted exponentials. Applications to moving averages with long memory are discussed in relationship with stochastic volatility models.
Paper analyzes sparse aggregation in GLMs with Kullback-Leibler risk bounds.
problem Sparse aggregation in GLMs for parameter approximation.
method Exponential weighted aggregation scheme with Kullback-Leibler risk bounds.
result Sharp oracle inequality for Kullback-Leibler risk with leading constant 1 and minimax-optimal rate of aggregation.
Book covers tools for zeroth-order convex optimisation.
problem Zeroth-order convex optimisation.
method Cutting plane methods, interior point methods, continuous exponential weights, gradient descent, online Newton step.
result Improved existing bounds and algorithms.
New algorithm optimizes online network resource allocation with long-term constraints.
problem Optimal resource reservation in communication networks with job transfers and budget limits.
method Randomized exponentially weighted method for long-term constraints.
result Upper bound for regret and cumulative constraint violations established.
Decentralized algorithm reduces regret and converges to Nash equilibrium in online congestion games.
problem Online congestion games with exponential action sets and strict Nash equilibria.
method CongestEXP algorithm using exponential weights method.
result CongestEXP achieves O ( k F T ) O(kF\sqrt{T}) O ( k F T ) regret bound and almost exponential convergence to strict Nash equilibrium. Introduces Exponentially Weighted Signature for better path representation.
problem Uniform treatment of historical information in signatures.
method Generalizes EFM signature to bounded linear operators, enabling contextualised temporal weighting.
result EWS is the unique solution to a linear controlled differential equation and generalizes state-space models.
Novel Bayesian method for high-dimensional count data prediction.
problem Count data in high-dimensional settings requires feature selection.
method Pseudo-Bayesian framework with scaled Student prior and exponential weights.
result Strong performance compared to Lasso in various settings.
Improved averaging method for noisy observations converges strongly.
problem Noisy observations from random dynamical systems require stable estimates.
method Introduced p p p -EMA, a modified exponential moving average with subharmonic weight decay. result Stochastic convergence guarantees for p p p -EMA under mild assumptions. Researchers extend asymptotic analysis to Bergman projections with Gevrey weights.
problem Analyzing Bergman projections with Gevrey weights.
method Extending direct approach to semiclassical asymptotics to Gevrey weights using Fourier integral operators.
result Gevrey symbol amplitude of asymptotic Bergman projection with Gevrey weights and Gevrey-type growth rate.
The study analyzes how neural reward models learn features for policy optimization in a Gaussian single-index model.
problem Reward modeling in policy optimization and its impact on downstream value.
method Two-stage neural reward model: first learns hidden direction, then fits readout layer.
result For any feature-learning temperature above a dimension-free threshold, a constant fraction of neurons recover the hidden direction.
Paper explores exact recovery of communities in weighted graphs using Gaussian and exponential distributions.
problem Exact recovery of communities in weighted graphs with Gaussian and exponential distributions.
method Introduces a new semi-metric to describe conditions for exact recovery and analyzes conditions for both complete and incomplete graphs.
result Necessary and sufficient conditions for exact recovery are asymptotically tight and applicable to both complete and incomplete graphs.
WildWood improves Random Forest predictions using bootstrap out-of-bag samples.
problem Improving Random Forest predictions for supervised learning.
method Uses bootstrap out-of-bag samples to compute improved predictions by aggregating all possible subtrees with exponential weights.
result WildWood produces faster and more competitive predictions compared to other ensemble methods.
Develops a new fuzzy model using QPs and ewl2 regularization to improve local region behavior.
problem Inability of constant and linear functions to accurately describe local regions in fuzzy models.
method Applied Fuzzy C-Means for structure identification, used QPs as consequents, introduced ewl2 regularization.
result Improved model's ability to describe local regions without overfitting.
Classifying streaming data requires the development of methods which are computationally efficient and able to cope with changes in the underlying distribution of the stream, a phenomenon known in the literature as concept drift. We propose a new method for detecting concept drift which uses an Exponentially Weighted M…
New method improves covariance estimation for weighted samples.
problem Improving covariance estimation for weighted sample data.
method Asymptotic non-linear shrinkage formulas for covariance and precision matrix estimators of weighted sample covariances.
result Asymptotic non-linear shrinkage formulas for covariance and precision matrix estimators of weighted sample covariances.
A standard introduction to online learning might place Online Gradient Descent at its center and then proceed to develop generalizations and extensions like Online Mirror Descent and second-order methods. Here we explore the alternative approach of putting Exponential Weights (EW) first. We show that many standard meth…
We improve deep threshold networks' memorization capacity exponentially.
problem Memorizing datasets with randomized labels using deep neural networks.
method Using Gaussian random weights in the first layer and binary or integer weights in subsequent layers, we prove a new dependence on minimum distance.
result We show that O ~ ( 1 δ + n ) \widetilde{\mathcal{O}}(\frac{1}{\delta} + \sqrt{n}) O ( δ 1 + n ) neurons and O ~ ( d δ + n ) \widetilde{\mathcal{O}}(\frac{d}{\delta} + n) O ( δ d + n ) weights are sufficient. Reweighting training data to better represent new tasks.
problem Deploying machine learning models to new tasks is challenging due to training data distribution.
method Formulate an exponential tilt distribution shift model and learn train data importance weights to minimize KL divergence.
result The learned train data weights improve target performance evaluation, fine-tuning, and model selection.
Paper proposes a new activation function to reduce overfitting and large weight update issues.
problem Overfitting and large weight update problems in neural networks.
method Introduces a new activation function called Thresholded Exponential Rectified Linear Units (TERELU).
result TERELU shows better performance in reducing overfitting and large weight update issues compared to other activation functions.
Abstract result on correlations of pairs in exponentially growing discrete subsets.
problem Pair correlations in exponentially growing discrete subsets with weight functions.
method Proved abstract result on correlations of pairs of elements in an exponentially growing discrete subset with a weight function.
result Distribution function of unscaled differences is t ↦ δ 2 e − ∣ t ∣ t\mapsto\fracδ2\,e^{-|t|} t ↦ 2 δ e − ∣ t ∣ , and pair correlation exhibits Poissonian behavior under certain conditions. Meta-strategy learns tuning parameters for online learning methods.
problem Difficulty in setting tuning parameters for online learning methods.
method Meta-learning approach to learn parameters from past tasks.
result Meta-strategy improves on learning each task in isolation.
Nonuniform tubular neighborhoods of curves in Euclidean n-space are studied by using weighted distance functions and generalizing the normal exponential map. Different notions of injectivity radii are introduced to investigate singular but injective exponential maps. A generalization of the thickness formula is obtaine…
New loss function restores importance weighting in overparameterized models.
problem Restoring importance weighting in overparameterized neural networks.
method Introduced polynomially-tailed losses to restore effects of importance weighting.
result Polynomially-tailed losses improve performance in correcting distribution shift.
Study symmetry groups and curves from sums of exponentials.
problem Understanding the geometry and symmetry of curves from sums of exponentials.
method Analysis of symmetry groups, winding numbers, and parametrization of the unit circle.
result Unified method for constructing curves with specific properties.
Paper develops a new weighted low-rank matrix approximation technique.
problem Matrix completion with missing data.
method Element-wise weighted generalization of low-rank matrix approximation.
result Proposes an algorithm and acceleration techniques for solving the weighted problem.
New aggregation strategy handles unbounded losses with regret bounds.
problem Online optimization with unbounded loss functions.
method Follow The Regularized Leader (FTRL) with φ-divergence.
result Worst regret bound for unbounded losses with alternative divergences.
Annealed Entropic Allocation improves ranking and selection by mitigating hard switching and improving finite-budget discrimination.
problem Sequential budget allocation in ranking and selection
method Annealed weighted soft-min framework
result Surrogate converges uniformly to the hard minimum, soft-min weights concentrate on active challengers, and target allocation map is continuous.
New method compresses non-Gaussian distributions exponentially.
problem Efficiently representing and computing non-Gaussian probability distributions.
method Tensor-Network Fourier Methods using QTT representation.
result Exponential compression of non-Gaussian distributions.
Study addresses RTB model performance drops due to distribution shifts.
problem Distribution shifts between training and target environments in RTB markets.
method Applies Exponential Tilt Reweighting Alignment (ExTRA) algorithm to estimate and correct model weights.
result Demonstrates improved RTB model performance using ExTRA algorithm.
Guyon-Lekeufack model accurately predicts market volatility.
problem Modeling and predicting market volatility accurately.
method Path-dependent volatility model with weighted past price returns and squared volatility.
result Wellposedness of the coupled system of stochastic differential equations for all parameter values.
A new method for averaging model predictions using minimum divergence.
problem Improving model averaging methods, especially in small samples.
method Minimum divergence framework for model weight calculation.
result Empirically outperforms standard model averaging methods.
Exponential Lasso improves Lasso's robustness to outliers and heavy-tailed noise.
problem Lasso's sensitivity to outliers and heavy-tailed noise in high-dimensional statistics.
method Integrates an exponential-type loss function into the Lasso framework.
result Achieves strong statistical convergence rates robust to heavy-tailed contamination.
A new method for robust training under label noise using weighted gradient descent.
problem Overfitting to noisy examples in machine learning.
method Exponentiated gradient reweighting for flexible handling of noisy data.
result Improved generalization in noisy classification and PCA problems.
New method speeds up neural network training by preprocessing weight-data correlation.
problem Slow neural network training due to high time complexity.
method Stores weight-data correlation in a tree structure for quick detection of firing neurons.
result Achieves o ( n m d ) o(nmd) o ( nm d ) time per iteration with only O ( n m d ) O(nmd) O ( nm d ) preprocessing time. AEW estimator achieves optimal risk in expectation for large enough temperatures.
problem Understanding minimax-rate optimality of AEW estimator in model selection aggregation.
method Analyzing AEW estimator with exponential weights for squared loss under random design.
result AEW achieves excess risk T log ( M ) / ( n + 1 ) T \log (M) / (n+1) T log ( M ) / ( n + 1 ) in expectation for large enough temperatures. This work improves polynomial approximations for functions with asymmetric behavior.
problem Efficiently approximating functions with asymmetric behavior, especially those growing unbounded on one side.
method Introduces weighted deep polynomial approximants that combine learnable deep polynomials with one-sided weights.
result Weighted deep polynomial approximants outperform existing methods in approximating functions with asymmetric behavior.
Researchers prove constant solutions for a specific Finslerian equation.
problem Investigating exponentially harmonic functions on Finslerian spaces.
method Analyzing the exponential energy functional and using nonnegative Ricci curvature conditions.
result Any bounded solution to the Finslerian equation is constant.
A new method normalizes flow mixtures for better inference across different data types.
problem Inference failure across diverse posterior geometries in normalizing flows.
method Introduces a two-stage framework with a stable global weighting mechanism based on sEMA.
result Achieves consistent NLL improvements and stable weight trajectories over baselines.
Efficiently optimizes boolean functions using multilinear polynomials and exponential weight updates.
problem Optimizing boolean functions over the boolean hypercube with high computational cost.
method Proposes a computationally efficient algorithm using multilinear polynomials and exponential weight updates.
result Improves computational time up to several orders of magnitude compared to state-of-the-art algorithms.
Paper introduces kernel deformed exponential families for sparse continuous attention.
problem Creating efficient attention mechanisms for sparse data.
method Developed kernel deformed exponential families, theoretically and experimentally.
result Kernel deformed exponential families can attend to multiple compact regions of data.
Robust variable selection for high-dimensional data with missing and measurement errors.
problem Missing data and measurement errors confound data distribution.
method Exponential loss function with inverse probability weighting and additive error models.
result The Atan punishment method improves robust variable selection.
Gradient descent on normalized networks reveals sparsity preferences.
problem Understanding the inductive bias of gradient descent on normalized neural nets.
method Analysis of gradient descent on weight-normalized smooth homogeneous neural nets, focusing on SWN and EWN.
result EWN causes weights to be updated in a way that prefers asymptotic relative sparsity.
New method models aptamer libraries as Boltzmann-weighted graph ensembles for better affinity predictions.
problem Anomalous candidates in SELEX datasets obscure true aptamer-ligand affinity.
method Boltzmann graph ensemble embeddings for thermodynamically parameterized exponential-family random graphs.
result Proposed embedding enables robust community detection and subgraph-level explanations for aptamer ligand affinity.