A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Improved generalization with iterate averaging and adaptive algorithms.
problem Enhancing model generalization performance in high-dimensional settings.
method Gaussian process perturbation model, combine IA with large learning rates and regularisation, less frequent averaging, adaptive gradient methods.
result Iterate averaging with adaptive algorithms significantly outperforms SGD on various datasets.
Random Feature (RF) models are used as efficient parametric approximations of kernel methods. We investigate, by means of random matrix theory, the connection between Gaussian RF models and Kernel Ridge Regression (KRR). For a Gaussian RF model with P features, N data points, and a ridge λ, we show that the avera…
In this paper we investigate the scaling behavior of the average daily exchange rate returns of the Indian Rupee against four foreign currencies namely US Dollar, Euro, Great Britain Pound and Japanese Yen. Average daily exchange rate return of the Indian Rupee against US Dollar is found to exhibit a persistent scaling…
In mixture model-based clustering applications, it is common to fit several models from a family and report clustering results from only the `best' one. In such circumstances, selection of this best model is achieved using a model selection criterion, most often the Bayesian information criterion. Rather than throw awa…
A new Bayesian modeling method is proposed by combining the maximization of the marginal likelihood with a momentum-space renormalization group transformation for Gaussian graphical models. Moreover, we present a scheme for computint the statistical averages of hyperparameters and mean square errors in our proposed met…
Sornette et al. claimed that the optimal supply does not agree with the average demand, by analyzing a bakery model where a daily demand fluctuates with a uniform distribution. In this note, we extend the model to general probability distributions, and obtain the formula of the optimal supply for Gaussian distribution,…
The posterior variance of Gaussian processes is a valuable measure of the learning error which is exploited in various applications such as safe reinforcement learning and control design. However, suitable analysis of the posterior variance which captures its behavior for finite and infinite number of training data is …
Deep neural networks learn by averaging fast variables, revealing a Gaussian process.
problem Analyzing the complex behavior of deep neural networks (DNNs) with billions of parameters.
method Identifying slow variables that average the erratic behavior of fast microscopic variables in fully trained DNNs.
result DNN layers couple only through the second moment (kernels) of their activations and pre-activations, which fluctuate in a nearly Gaussian manner.
We propose SWA-Gaussian (SWAG), a simple, scalable, and general purpose approach for uncertainty representation and calibration in deep learning. Stochastic Weight Averaging (SWA), which computes the first moment of stochastic gradient descent (SGD) iterates with a modified learning rate schedule, has recently been sho…
We show that if F is a convex class of functions that is L-subgaussian, the error rate of learning problems generated by independent noise is equivalent to a fixed point determined by `local' covering estimates of the class, rather than by the gaussian averages. To that end, we establish new sharp upper and lower e…
Deep Gaussian Processes with polynomial kernels can collapse rapidly without proper hyperparameter tuning.
problem The collapse of Deep Gaussian Processes with polynomial kernels without careful hyperparameter tuning.
method Analysis using the Berry-Esseen Theorem and observation of prior behavior.
result The prior of a Deep Gaussian Process collapses rapidly towards zero or places negligible mass on low norm functions without proper hyperparameter tuning.
This paper develops several average-case reduction techniques to show new hardness results for three central high-dimensional statistics problems, implying a statistical-computational gap induced by robustness, a detection-recovery gap and a universality principle for these gaps. A main feature of our approach is to ma…
Andreas Maurer in the paper "A vector-contraction inequality for Rademacher complexities" extended the contraction inequality for Rademacher averages to Lipschitz functions with vector-valued domains; He did it replacing the Rademacher variables in the bounding expression by arbitrary idd symmetric and sub-gaussian var…
This article is concerned with Gaussian process quadratures, which are numerical integration methods based on Gaussian process regression methods, and sigma-point methods, which are used in advanced non-linear Kalman filtering and smoothing algorithms. We show that many sigma-point methods can be interpreted as Gaussia…
The random matrix theory method of planar Gaussian diagrammatic expansion is applied to find the mean spectral density of the Hermitian equal-time and non-Hermitian time-lagged cross-covariance estimators, firstly in the form of master equations for the most general multivariate Gaussian system, secondly for seven part…
The method to derive uniform bounds with Gaussian and Rademacher complexities is extended to the case where the sample average is replaced by a nonlinear statistic. Tight bounds are obtained for U-statistics, smoothened L-statistics and error functionals of l2-regularized algorithms.
We derive an explicit formula for likelihood function for Gaussian VARMA model conditioned on initial observables where the moving-average (MA) coefficients are scalar. For fixed MA coefficients the likelihood function is optimized in the autoregressive variables Φ's by a closed form formula generalizing regression c…
We examine two different techniques for parameter averaging in GAN training. Moving Average (MA) computes the time-average of parameters, whereas Exponential Moving Average (EMA) computes an exponentially discounted sum. Whilst MA is known to lead to convergence in bilinear settings, we provide the -- to our knowledge …