A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
New method corrects missing data bias in dimension reduction.
We study the problem of empirical minimization for variance-type functionals over functional classes. Sharp non-asymptotic bounds for the excess variance are derived under mild conditions. In particular, it is shown that under some restrictions imposed on the functional class fast convergence rates can be achieved incl…
We study optimization algorithms based on variance reduction for stochastic gradient descent (SGD). Remarkable recent progress has been made in this direction through development of algorithms like SAG, SVRG, SAGA. These algorithms have been shown to outperform SGD, both theoretically and empirically. However, asynchro…
A new framework for dimension reduction using ensemble of random projections.
New method improves scalability of SGD for large datasets.
Paper proposes an algorithm to reduce hypothesis space for faster convergence in high-dimensional settings.
Paper develops efficient algorithms for robust optimization across multiple groups.
We propose a new randomized ensemble technique with a provable security guarantee against black-box transfer attacks. Our proof constructs a new security problem for random binary classifiers which is easier to empirically verify and a reduction from the security of this new model to the security of the ensemble classi…
Unified framework for DR and clustering using Gromov-Wasserstein.
New sparsity operator reduces variance reduction methods' computational cost.
Modeling data as being sampled from a union of independent subspaces has been widely applied to a number of real world applications. However, dimensionality reduction approaches that theoretically preserve this independence assumption have not been well studied. Our key contribution is to show that projection vect…
We introduce a novel data-driven order reduction method for nonlinear control systems, drawing on recent progress in machine learning and statistical dimensionality reduction. The method rests on the assumption that the nonlinear system behaves linearly when lifted into a high (or infinite) dimensional feature space wh…
A new deep neural network tackles nonlinear functional regression with improved dimensionality reduction.
We consider a composite convex minimization problem associated with regularized empirical risk minimization, which often arises in machine learning. We propose two new stochastic gradient methods that are based on stochastic dual averaging method with variance reduction. Our methods generate a sparser solution than the…
Efficiently reduces rank of non-negative matrices with quadratic time complexity.
POTD estimates SDR subspace using optimal transport for binary response.
New PG method derived from IS, improving variance reduction.
We show how to reduce the process of predicting general order statistics (and the median in particular) to solving classification. The accompanying theoretical statement shows that the regret of the classifier bounds the regret of the quantile regression under a quantile loss. We also test this reduction empirically ag…
New method turns optimization algorithms into uniformly stable learning algorithms for non-Euclidean norms.
Adaptive importance sampling for stochastic optimization is a promising approach that offers improved convergence through variance reduction. In this work, we propose a new framework for variance reduction that enables the use of mixtures over predefined sampling distributions, which can naturally encode prior knowledg…
Paper uses non-linear dimension reduction for better economic forecasting.
A new framework for bilevel optimization tackles stochastic and global variance reduction.
The study reveals how synaptic correlations promote dimension reduction in neural networks.
We introduce a data-driven order reduction method for nonlinear control systems, drawing on recent progress in machine learning and statistical dimensionality reduction. The method rests on the assumption that the nonlinear system behaves linearly when lifted into a high (or infinite) dimensional feature space where ba…
Ensembles improve classifier performance by reducing bias, not variance.
Optimizes differentially private kernel learning with random projection.
Reduces complexity of financial contagion dynamics on networks.
Deep neural networks are vulnerable to adversarial examples, i.e., carefully-perturbed inputs aimed to mislead classification. This work proposes a detection method based on combining non-linear dimensionality reduction and density estimation techniques. Our empirical findings show that the proposed approach is able to…
In this paper, we study randomized reduction methods, which reduce high-dimensional features into low-dimensional space by randomized methods (e.g., random projection, random hashing), for large-scale high-dimensional classification. Previous theoretical results on randomized reduction methods hinge on strong assumptio…
Improved variance reduction for Riemannian non-convex optimization with adaptive batch size.
New optimizer MARS-M combines variance reduction with Muon for faster LLM training.
Paper reduces movement primitive dimensionality in parameter space.
In this paper we propose the macroblock scaling (MBS) algorithm, which can be applied to various CNN architectures to reduce their model size. MBS adaptively reduces each CNN macroblock depending on its information redundancy measured by our proposed effective flops. Empirical studies conducted with ImageNet and CIFAR-…
Paper introduces S-SSE for stable sparse subspace embedding.
Improves MARS for nonparametric multivariate regression with dimension reduction.
SVRN accelerates Newton methods by reducing variance and improving performance.
Unified convergence analysis of alpha-SVRG under strong convexity.
In this work, we revisit fast dimension reduction approaches, as with random projections and random sampling. Our goal is to summarize the data to decrease computational costs and memory footprint of subsequent analysis. Such dimension reduction can be very efficient when the signals of interest have a strong structure…
Variance reduction (VR) methods boost the performance of stochastic gradient descent (SGD) by enabling the use of larger, constant stepsizes and preserving linear convergence rates. However, current variance reduced SGD methods require either high memory usage or an exact gradient computation (using the entire dataset)…
In our work, we propose a novel formulation for supervised dimensionality reduction based on a nonlinear dependency criterion called Statistical Distance Correlation, Szekely et. al. (2007). We propose an objective which is free of distributional assumptions on regression variables and regression model assumptions. Our…
This paper investigates the hedging effectiveness of a dynamic moving window OLS hedging model, formed using wavelet decomposed time-series. The wavelet transform is applied to calculate the appropriate dynamic minimum-variance hedge ratio for various hedging horizons for a number of assets. The effectiveness of the dy…
New method for spatiotemporal data regression using Gaussian processes.
TriMap improves data visualization by preserving global structure better than existing methods.
Communication on heterogeneous edge networks is a fundamental bottleneck in Federated Learning (FL), restricting both model capacity and user participation. To address this issue, we introduce two novel strategies to reduce communication costs: (1) the use of lossy compression on the global model sent server-to-client;…
U-statistics improve gradient estimation in importance-weighted variational inference.
We present Matrix Krasulina, an algorithm for online k-PCA, by generalizing the classic Krasulina's method (Krasulina, 1969) from vector to matrix case. We show, both theoretically and empirically, that the algorithm naturally adapts to data low-rankness and converges exponentially fast to the ground-truth principal su…
The variance reduction class of algorithms including the representative ones, SVRG and SARAH, have well documented merits for empirical risk minimization problems. However, they require grid search to tune parameters (step size and the number of iterations per inner loop) for optimal performance. This work introduces `…
SMAVE optimizes SDR by projecting onto a low-dimensional subspace on a Riemannian manifold.