One-pass algorithm finds small subset for ℓp subspace approximation with additive error.
problem Finding a small subset of data points for ℓp subspace approximation. method One-pass subset selection with additive approximation guarantee for p∈[1,∞). result First one-pass algorithm with additive error for ℓp subspace approximation. Bayesian online learning algorithm for one-pass data, achieving frequentist validity and uncertainty quantification.
problem Theoretical limitations in Bayesian online learning, especially in the one-pass setting.
method Proposed a new Bayesian online learning algorithm with a warm-start phase for the one-pass regime, establishing convergence rates and valid uncertainty quantification.
result The sequentially updated posterior attains optimal convergence rates and valid uncertainty quantification without diverging mini-batch sample sizes.
SGMM clusters data in one pass, reducing dimensionality.
problem Clustering high-dimensional data efficiently.
method One-pass sparsification of Gaussian mixture model.
result SGMM achieves clustering accuracy with reduced computational cost.
Full-batch GD outperforms one-pass SGD in learning a single-index model with quadratic activation.
problem Learning a single-index model with quadratic activation using gradient descent.
method Full-batch gradient descent compared to one-pass stochastic gradient descent (SGD) on a correlation loss.
result Full-batch GD requires only n≃d samples for strong recovery, while one-pass SGD requires n≳dlogd samples. One-pass private sketch supports various machine learning tasks.
problem Efficiently supporting multiple machine learning tasks with differential privacy.
method Randomized contingency tables indexed with locality-sensitive hashing, constructed in one pass.
result Competitive error bounds for DP kernel density estimation, faster than existing methods.
One-pass optimisation for high-dimensional hyperparameters.
problem Efficient optimisation of hyperparameters in machine learning models.
method Approximate hypergradient-based optimisation for any continuous hyperparameter, requiring only one training episode.
result Competitive performance on various datasets without hyperparameter restarts.
New algorithm achieves nearly optimal regret with one-pass updates for GLB problems.
problem Generalized linear bandits with non-linear reward distributions.
method Jointly efficient algorithm using OMD estimator with one-pass updates.
result Nearly optimal regret bound with O(1) time and space complexities per round. Efficiently tests discrete distributions with limited memory and communication.
problem Testing discrete distributions with constraints on memory and communication.
method Developed efficient algorithms for uniformity/identity and closeness testing in streaming and distributed models.
result Nearly-tight lower bounds on sample complexity and communication cost for uniformity testing.
One-pass SGD dynamics in overparameterized quadratic networks show slow escape from poor solutions.
problem Slow escape from poor generalization solutions in overparameterized neural networks.
method Analysis of one-pass SGD dynamics using ordinary differential equations for overlap matrices.
result Overparameterization only modestly accelerates escape from poor solutions.
ORFit trains models on streaming data with one pass, minimizing memory and computational costs.
problem Training large models on a stream of data without retraining on previous data.
method Orthogonal Recursive Fitting (ORFit) using orthogonal gradient descent and recursive least-squares.
result ORFit updates parameters orthogonally to past gradients, leading to efficient memory and computational usage.
DFOP learns online without knowing distribution changes.
problem Learning with changing distributions and limited data scans.
method Distribution-free one-pass learning approach.
result Estimate error decreases until convergence with high probability.
One-pass SGD converges in overparametrized neural networks with random data.
problem Understanding convergence of SGD in neural networks with streaming data.
method Overparameterized two-layer neural networks, one-pass SGD, random initialization, NTK eigen-decomposition, VC dimension, McDiarmid's inequality.
result Prediction error converges in expectation under one-pass SGD in overparametrized neural networks.
A new method for clustering multi-view data with missing views.
problem Clustering multi-view datasets with missing views is challenging.
method OPIMC framework using regularized and weighted matrix factorization.
result OPIMC efficiently clusters multi-view datasets with missing views.
Consider a sequence of closed, orientable surfaces of fixed genus g in a Riemannian manifold M with uniform upper bounds on mean curvature and area. We show that on passing to a subsequence and choosing appropriate parametrisations, the inclusion maps converge in C0 to a map from a surface of genus g to M. W…
In this paper, we propose a one-pass algorithm on MapReduce for penalized linear regression \[f_λ(α, β) = \|Y - α\mathbf{1} - Xβ\|_2^2 + p_λ(β)\] where α is the intercept which can be omitted depending on application; β is the coefficients and pλ is the penalized function with penalizing parameter λ. $f_λ(α, β…
New method for RLHF reduces costs by integrating new data in one pass.
problem Continuous integration and re-optimization of models in RLHF leads to high computational and storage costs.
method Proposes a one-pass reward modeling method using online mirror descent with a tailored local norm.
result Achieves constant-time updates per iteration, enhancing both statistical and computational efficiency.
A new data-oblivious sketch for logistic regression reduces data size while maintaining approximation accuracy.
problem Efficiently solving logistic regression in one pass over a data stream.
method Data-oblivious sketching approach that reduces data size to poly(μdlog n) weighted points.
result Sketching reduces data size significantly and provides approximation guarantees.
Efficiently learns gFM on streaming data with linear convergence.
problem Learning generalized Factorization Machine on streaming data.
method Alternating framework with CI-RIP condition.
result Linear convergence and O(ε) recovery error after retrieving O(k3dlog(1/ε)) training instances. End-to-end neural network clusters data in one pass.
problem Clustering high-dimensional data like images and speech.
method Trains a neural network to directly output cluster assignments based on perceptual similarity.
result Demonstrates promising performance on COIL-100 and TIMIT datasets.
Stochastic Gradient Descent can overfit after just a few passes, contrary to initial expectations.
problem Understanding the out-of-sample performance of multi-pass SGD in stochastic convex optimization.
method Analysis of multi-pass SGD in the stochastic convex optimization model.
result Multi-pass SGD can lead to significant overfitting after just a few passes, contrary to initial expectations.
Two new approaches for point prediction in streaming data, showing consistency and performance.
problem Predicting points in streaming data without a true model.
method Count-Min sketch and Gaussian process priors with random bias.
result CMS-based estimates are consistent under i.i.d. samples assumption.
The equivariant Gromov--Hausdorff convergence of metric spaces is studied. Where all isometry groups under consideration are compact Lie, it is shown that an upper bound on the dimension of the group guarantees that the convergence is by Lie homomorphisms. Additional lower bounds on curvature and volume strengthen this…
Polyak-Ruppert CLT for SA-Adam with momentum and non-convergent adaptive preconditioning
problem Adaptive optimizers combining momentum and non-convergent preconditioning
method Proving positive drift stability and a non-autonomous Polyak-Ruppert CLT for SA-Adam
result The iterate-marginal covariance is exactly the plain stochastic gradient descent (SGD) sandwich
A new feature selection method using attention for neural networks.
problem Feature selection for neural networks with a budget constraint.
method Sequential Attention: greedy forward selection with attention weights.
result Achieves state-of-the-art results for neural networks.
The paper analyzes how repeating epochs affects data scaling in linear regression.
problem Understanding how to scale data for multi-epoch training in linear regression.
method Theoretical analysis of effective reuse rate (E(K, N)) under strong convexity or Zipf-distributed data.
result The effective reuse rate E(K, N) plateaus at a problem-dependent value that grows with N, indicating diminishing marginal gains.
We present a streaming model for large-scale classification (in the context of ℓ2-SVM) by leveraging connections between learning and computational geometry. The streaming model imposes the constraint that only a single pass over the data is allowed. The ℓ2-SVM is known to have an equivalent formulation in …
New algorithm reduces heavy-tailed linear bandits' computational cost.
problem Stochastic linear bandits with heavy-tailed noise.
method One-pass online mirror descent with adaptive Huber regression.
result Near-optimal regret bound with reduced computational cost.
Study Higgs bundles and their Langlands duals on surfaces.
problem Matching components of Higgs bundle moduli spaces under Langlands duality.
method Defined antiholomorphic involutions on Higgs bundle moduli spaces and analyzed fixed point loci.
result Components of fixed point loci match under Langlands duality for specific groups.
We describe many vantage points on the Baire metric and its use in clustering data, or its use in preprocessing and structuring data in order to support search and retrieval operations. In some cases, we proceed directly to clusters and do not directly determine the distances. We show how a hierarchical clustering can …
A new algorithm reduces memory usage for long token attention in streaming applications.
problem Memory inefficiency in computing attention for long documents.
method One-pass streaming algorithm using sublinear space storage.
result Super-efficient memory usage for long token attention.
New algorithm for dynamic k-means clustering in high dimensions.
problem Dynamic clustering of high-dimensional data points.
method One-pass coreset construction algorithm for k-means. result Nearly optimal space usage for dynamic k-means. A new randomized method reduces memory usage for kernel clustering.
problem High memory usage in kernel-based clustering methods.
method Randomized approximation followed by standard K-means.
result Significantly less memory usage compared to standard methods.
Scalable methods for maximizing regularized submodular functions with improved memory and communication complexity.
problem Maximizing submodular functions with negative values and constraints.
method Developed one-pass streaming and distributed algorithms for maximizing regularized submodular functions.
result Improved memory and communication complexity by a factor of O(1/ε) compared to existing work.
A new framework for efficient large-scale learning using sketching of moments.
problem Efficiently learning from large datasets with limited computational resources.
method Compressing the training data into a low-dimensional sketch and solving a nonlinear least squares problem.
result Sufficient sketch sizes to control the generalization error of the procedure.
New algorithm improves mean field inference in probabilistic models.
problem Improving mean field inference in probabilistic models.
method DR-DoubleGreedy algorithm for continuous DR-submodular maximization with box-constraints.
result Achieves optimal 1/2 approximation ratio for continuous DR-submodular maximization.
Cubic-Spline Flows improve autoregressive flow performance in density estimation.
problem Improving the performance of flow-based models in density estimation.
method Stacking a new coupling transform based on monotonic cubic splines with LU-decomposed linear layers.
result Cubic-Spline Flows close the gap with autoregressive flows on density-estimation tasks.
The paper calculates indices for families of Fredholm operators and their extensions.
problem Calculating indices for families of Fredholm operators and their extensions.
method Passing from a Fredholm operator to its graph, deforming the horizontal subspace.
result Index formulas for families of Fredholm realizations and self-adjoint extensions.
This paper addresses how well we can recover a data matrix when only given a few of its elements. We present a randomized algorithm that element-wise sparsifies the data, retaining only a few its elements. Our new algorithm independently samples the data using sampling probabilities that depend on both the squares ($\e…
FFJORD models generate complex distributions efficiently with unbiased density estimation.
problem Efficiently generating complex distributions with unbiased density estimation.
method FFJORD uses continuous-time invertible neural networks with Hutchinson's trace estimator for unbiased log-density estimation.
result FFJORD achieves state-of-the-art performance in high-dimensional density estimation, image generation, and variational inference.
VR methods improve SGD for faster machine learning.
problem Efficiency in stochastic optimization for machine learning.
method Variance reduction techniques for stochastic optimization.
result VR methods achieve faster convergence than SGD.
Study compares adaptive vs fixed query learning methods.
problem Comparing adaptive and fixed query learning methods for task approximation.
method Examined in-context and agentic learning in two settings: unrestricted and realizable.
result Adaptivity does not hinder performance in unrestricted setting but can in realizable setting.
Tensor Switching networks improve neural network expressiveness without vanishing gradients.
problem Improving neural network expressiveness and avoiding vanishing gradients.
method Copies input vector to different tensor locations based on hidden unit activity; uses various training methods.
result TS network learns faster and is more expressive than standard ReLU networks.
A streaming algorithm for matrix completion with noisy data.
problem Matrix completion with limited memory and noisy observed entries.
method A streaming algorithm that estimates the original matrix with minimal memory and computational complexity.
result The algorithm produces an estimate with vanishing mean square error using linear memory and proportional to the number of non-zero entries.
Improved COD algorithm reduces streaming AMM errors and uses less space.
problem Efficiently approximate matrix multiplication with limited memory.
method Tighter error bound for COD, space optimality, sparse matrix variant.
result Improved COD is space optimal and more efficient for sparse matrices.
Paper presents an efficient approach for integrating LSTM language models in LVCSR systems.
problem Difficulty in incorporating LSTM language models efficiently into LVCSR systems.
method First-pass decoding with LSTM-LM followed by lattice rescoring and hypothesis recombination.
result Competitive results on Hub5'00 and Librispeech corpora with runtime better than real-time.
A new algorithm for streaming submodular maximization with memory constraints.
problem Maximizing a submodular function in a stream with memory removal.
method STAR-T algorithm based on partitioning and thresholding.
result A simple greedy algorithm achieves constant-factor approximation after memory removal.
The study analyzes deep linear networks from random initialization, capturing dynamics and hyperparameter effects.
problem Understanding training dynamics in deep linear networks from random initialization.
method Theoretical analysis of gradient descent dynamics in deep linear networks with random initialization and large data.
result Captures the 'wider is better' effect and hyperparameter transfer effects, contrasting with neural-tangent parameterization.
New method models matrix time series using tensor CP-decomposition.
problem Modeling matrix time series with reduced complexity.
method One-pass estimation via generalized eigenanalysis and refined projection.
result Component coefficient vectors estimated consistently with certain rates.