Tail averaging consists in averaging the last examples in a stream. Common techniques either have a memory requirement which grows with the number of samples to average, are not available at every timestep or do not accomodate growing windows. We propose two techniques with a low constant memory cost that perform tail …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “averaging”
New RL algorithms improve average-reward performance.
problem Improving RL algorithms for average-reward criteria.
method Developed novel algorithms addressing average-reward criterion directly.
result ATRPO significantly outperforms TRPO in challenging MuJuCo environments.
Partial model averaging improves Federated Learning performance.
problem Periodic model averaging causes significant model discrepancy in Federated Learning.
method Proposes a partial model averaging framework that encourages local models to stay close to each other.
result Partial averaging achieves up to 2.2% higher validation accuracy than full averaging.
Paper introduces stability in model averaging and proposes a L2-penalty method.
problem Theoretical properties of model averaging from stability perspective.
method Introduces stability, defines asymptotic empirical risk minimizer, and proposes L2-penalty model averaging method.
result Proposed L2-penalty method ensures stability and consistency under reasonable conditions.
We show Vector Autoregressive Moving Average models with scalar Moving Average components could be estimated by generalized least square (GLS) for each fixed moving average polynomial. The conditional variance of the GLS model is the concentrated covariant matrix of the moving average process. Under GLS the likelihood …
New bounds for average graph distance using curvature and centrality.
problem Finding bounds for average graph distance.
method Using weighted average Ollivier curvature with edge betweenness centrality.
result Equality in bounds achieved for specific reflective graphs.
Paper explores weighted averaging schemes for SGD, achieving asymptotic normality and optimality.
problem Improving convergence of SGD in various settings.
method Develops a general weighted averaging scheme for SGD and establishes asymptotic normality.
result Establishes asymptotic normality and optimality of weighted averaged SGD solutions.
PACE optimizes training for averaged language models, improving performance.
problem How to optimize training for averaged language model iterates.
method Formulated as an optimal-control problem, solved for minimizing error of the average with a penalty on intervention size.
result PACE improves the limiting squared error of the iterate-average estimator by an arbitrarily large factor on some instances.
In mixture model-based clustering applications, it is common to fit several models from a family and report clustering results from only the `best' one. In such circumstances, selection of this best model is achieved using a model selection criterion, most often the Bayesian information criterion. Rather than throw awa…
Two-Tailed Averaging improves generalization by optimizing the number of leading iterates to ignore.
problem Improving generalization in stochastic optimization with limited resources and hyperparameters.
method An anytime adaptive algorithm that balances the number of leading iterates to ignore for better generalization.
result Approximates the optimal tail at all optimization steps, improving generalization without hyperparameters.
The study examines how average scalar curvature influences geometric properties of Riemannian manifolds.
problem Investigating the geometric properties of Riemannian manifolds influenced by average scalar curvature.
method Analyzing the conjugate radius, average area of geodesic spheres, average volume of metric balls, and total volume of closed manifolds.
result Improves the Bishop-Gromov estimate on the average volume of metric balls and proves monotone decreasing properties of certain geometric integrals.
New moving average adapts weight dynamically based on polynomial and wavefunction.
problem Lagging traditional moving averages in adjusting to changes in data.
method Develops a moving average with weight as a polynomial of a wavefunction from an eigenproblem.
result Immediate 'switch' without lag, adapting to changes in data.
New TD method stabilizes average-reward learning.
problem Stability issues in average-reward TD learning.
method Implicit fixed point update for average-reward TD().
result Improved numerical stability and broader step-size range.
In this note we discuss the mathematical tools to define trend indicators which are used to describe market trends. We explain the relation between averages and moving averages on the one hand and the so called exponential moving average (EMA) on the other hand. We present a lot of examples and give the definition of t…
We analyze the question whether sliding window time averages applied to stationary increment processes converge to a limit in probability. The question centers on averages, correlations, and densities constructed via time averages of the increment x(t,T)=x(t+T)-x(t)and the assumption is that the increment is distribute…
Nested model averaging improves high-dimensional linear regression performance.
problem High-dimensional linear regression with predictor ordering impact.
method Combining model averaging with regularized estimators on the solution path.
result Nested model averaging with lasso and SLOPE outperforms competing methods.
Average Oracle outperforms DCC+NLS in portfolio optimization.
problem Optimizing portfolio performance in volatile markets.
method Comparing the Average Oracle to various DCC+NLS variants.
result The Average Oracle consistently yields higher Sharpe ratios.
Study irregular behavior of ball averages for non-amenable group actions on foliations.
problem Exploring irregular behavior of ball averages for non-amenable group actions.
method Introducing a new mechanism based on group structure to analyze irregular behavior.
result First examples of codimension one foliations with non-existent length averages.
Many problems in machine learning and game theory can be formulated as saddle-point problems, for which various first-order methods have been developed and proven efficient in practice. Under the general convex-concave assumption, most first-order methods only guarantee an ergodic convergence rate, that is, the uniform…
WASH trains ensembles with shuffled weights to improve accuracy and reduce communication.
problem Training ensembles for weight averaging leads to models converging to different loss basins.
method WASH randomly shuffles a small percentage of weights during training to keep models within the same basin.
result WASH achieves state-of-the-art image classification accuracy with lower communication costs.
Average signature measures geodesics in Lie groups.
problem Understanding geometric properties of Lie groups through geodesic paths.
method Introducing average signature and using it with trace operation to recover geometric properties.
result Average signature can recover geometric properties like dimension, diameter, volume, and scalar curvature.
Improving optimization for iterate-averaged language models
problem How to optimize the averaged model returned by Language Model pipelines
method Formulating optimizer design as an optimal-control problem
result Proven convergence rate and strict improvement in squared error
Asymptotics for the Discrete-Time Average of the Geometric Brownian Motion and Asian Optionsq-fin.PR
The time average of geometric Brownian motion plays a crucial role in the pricing of Asian options in mathematical finance. In this paper we consider the asymptotics of the discrete-time average of a geometric Brownian motion sampled on uniformly spaced times in the limit of a very large number of averaging time steps.…
In this thesis, we consider the suitability of using the charged cold fluid model in the description of ultra-relativistic beams. The method that we have used is the following. Firstly, the necessary notions of kinetic theory and differential geometry of second order differential equations are explained. Then an averag…
Improved averaging method for noisy observations converges strongly.
problem Noisy observations from random dynamical systems require stable estimates.
method Introduced -EMA, a modified exponential moving average with subharmonic weight decay.
result Stochastic convergence guarantees for -EMA under mild assumptions.
Nonsmooth High-Order Averaging Theory with Application to Extremum Seeking Optimization and Controlmath.OC
New theory for nonsmooth systems helps optimize and control complex functions.
problem Optimizing and controlling systems with nonsmooth functions.
method Higher-order averaging theory with nonsmooth near-identity transformation and lexicographic differentiation.
result Closed formula for nonsmooth first and second-order averaging.
Paper develops bounds for stochastic approximation with averaging.
problem Establish high-probability bounds for averaged stochastic approximation.
method Develops a general framework for non-asymptotic concentration bounds.
result Derives sharp bounds for averaged iterates and tightens existing results.
Revisiting SGD with Increasingly Weighted Averaging: Optimization and Generalization Perspectivescs.LG
This paper analyzes SGD with increasingly weighted averaging for optimization and generalization.
problem Improving optimization and generalization for non-strongly convex objectives.
method Comprehensive analysis of increasingly weighted averaging schemes for convex, strongly convex, and non-convex objectives.
result The weight affects both optimization and generalization errors, revealing a trade-off.
We propose and analyze a variant of the classic Polyak-Ruppert averaging scheme, broadly used in stochastic gradient methods. Rather than a uniform average of the iterates, we consider a weighted average, with weights decaying in a geometric fashion. In the context of linear least squares regression, we show that this …
Group averaging boosts model accuracy without training cost.
problem Challenging training of equivariant models in physics.
method Group averaging at test time, improving accuracy.
result Improves model accuracy by up to 37% in continuous dynamics.
The study analyzes when Bayesian averaging over decision trees is reliable.
problem When do Bayesian model averaging weights over decision trees provide reliable information?
method Closed-form solution for Bayesian decision trees with Catalan-exponential priors.
result Established a complete non-asymptotic theory of rational commitment thresholds.
Derives time-averaged active inference from control principles.
problem Finite-horizon or discounted-surprise problems in active inference.
method Derives infinite-horizon, average-surprise active inference from optimal control principles.
result Unified objective functional for sensorimotor control.
This study uses moving average cluster entropy to analyze financial market dynamics.
problem Understanding long-range dependence in financial markets.
method Moving average cluster entropy approach applied to ARFIMA and FBM processes.
result Long-range positive correlation in financial markets is linked to the cluster entropy behavior.
The paper tackles batch policy learning in Markov Decision Processes, focusing on average reward maximization.
problem Maximizing long-term average reward in Markov Decision Processes with batch learning.
method Doubly robust estimator for average reward, optimization algorithm for optimal policy, finite-sample regret guarantee.
result The proposed method achieves semiparametric efficiency and provides a finite-sample regret guarantee.
Since the pioneering work of Ghys, Langevin and Walczak among others, it has been known that several methods of dynamical systems theory can be adopted to study of foliations. Our aim in this paper is to investigate complexity of foliations, by generalising existence problem of time averages in dynamical systems theory…
Consider a number of workers running SGD independently on the same pool of data and averaging the models every once in a while -- a common but not well understood practice. We study model averaging as a variance-reducing mechanism and describe two ways in which the frequency of averaging affects convergence. For convex…
Improved TD learning with tail averaging and regularization achieves optimal convergence rates.
problem Convergence analysis of TD learning with linear function approximation.
method Tail-averaging and regularization applied to TD learning algorithm.
result Achieves optimal convergence rate in expectation and with high probability.
While stochastic gradient descent (SGD) is one of the major workhorses in machine learning, the learning properties of many practically used variants are poorly understood. In this paper, we consider least squares learning in a nonparametric setting and contribute to filling this gap by focusing on the effect and inter…
Given a Finsler space (M,F), one can define natural average Riemannian metrics on M by averaging on the indicatrix I_x the fundamental tensor g of the Finsler function . In this paper we determine explicitly the Levi-Civita connection for these average Riemannian metrics. We apply the result to the case when (M,F) i…
Geometric Brownian motion (GBM) is a model for systems as varied as financial instruments and populations. The statistical properties of GBM are complicated by non-ergodicity, which can lead to ensemble averages exhibiting exponential growth while any individual trajectory collapses according to its time-average. A com…
Bayesian model averaging improves causal effect estimation by averaging over multiple models.
problem Estimating causal effects under linear Structural Causal Models (SCMs).
method Bayesian model averaging using Gaussian scale mixture distributions for computational efficiency.
result Bayesian model averaging is optimal for causal effect estimation.
Minimal surfaces and average area ratio found to be maximized by hyperbolic metrics.
problem Finding sharp relations between minimal surface entropy and average area ratio.
method Ricci flow with surgery and invariant measures.
result Minimal surface entropy maximized by hyperbolic metrics among metrics with scalar curvature ≥ -6.
A new method for averaging model predictions using minimum divergence.
problem Improving model averaging methods, especially in small samples.
method Minimum divergence framework for model weight calculation.
result Empirically outperforms standard model averaging methods.
This paper improves forecasts for diverse time series by averaging similar ones.
problem Forecasting challenges in heterogeneous time series.
method Dynamic Time Warping to find similar time series, k-Nearest Neighbor averaging.
result Averaging improves forecasts of simple models.
There are non-vanishing price responses across different stocks in correlated financial markets. We further study this issue by performing different averages, which identify active and passive cross-responses. The two average cross-responses show different characteristic dependences on the time lag. The passive cross-r…
In classification, the de facto method for aggregating individual losses is the average loss. When the actual metric of interest is 0-1 loss, it is common to minimize the average surrogate loss for some well-behaved (e.g. convex) surrogate. Recently, several other aggregate losses such as the maximal loss and average t…
This work proposes a model averaging method for SVM that avoids redundant covariates and achieves asymptotic optimality.
problem Redundant covariates impair SVM performance in high-dimensional settings.
method Frequentist model averaging procedure for SVM using cross-validation to select optimal weights.
result The proposed method achieves asymptotic optimality in SVM model averaging.
Unified analysis of finite weight averaging methods in deep learning.
problem Theoretical explanation of finite weight averaging methods in deep learning.
method Generalized SGD and LAWA, establishing convergence and generalization bounds.
result FWA achieves faster convergence and better generalization compared to SGD.