arXiv research

ORFit trains models on streaming data with one pass, minimizing memory and computational costs.

We study distribution testing with communication and memory constraints in the following computational models: (1) The {\em one-pass streaming model} where the goal is to minimize the sample complexity of the protocol subject to a memory constraint, and (2) A {\em distributed model} where the data samples reside at mul…

One-pass SGD converges in overparametrized neural networks with random data.

A new data-oblivious sketch for logistic regression reduces data size while maintaining approximation accuracy.

Two new approaches for point prediction in streaming data, showing consistency and performance.

We present a streaming model for large-scale classification (in the context of $\ell_2$-SVM) by leveraging connections between learning and computational geometry. The streaming model imposes the constraint that only a single pass over the data is allowed. The $\ell_2$-SVM is known to have an equivalent formulation in …

We present a one-pass sparsified Gaussian mixture model (SGMM). Given $N$ data points in $P$ dimensions, $X$, the model fits $K$ Gaussian distributions to $X$ and (softly) classifies each point to these clusters. After paying an up-front cost of $\mathcal{O}(NP\log P)$ to precondition the data, we subsample $Q$ entries…

We consider the $k$-means clustering problem in the dynamic streaming setting, where points from a discrete Euclidean space $\{1, 2, \ldots, Δ\}^d$ can be dynamically inserted to or deleted from the dataset. For this problem, we provide a one-pass coreset construction algorithm using space $\tilde{O}(k\cdot \mathrm{pol…

A new algorithm reduces memory usage for long token attention in streaming applications.

We study the classical problem of maximizing a monotone submodular function subject to a cardinality constraint k, with two additional twists: (i) elements arrive in a streaming fashion, and (ii) m items from the algorithm's memory are removed after the stream is finished. We develop a robust submodular algorithm STAR-…

Streaming algorithms are generally judged by the quality of their solution, memory footprint, and computational complexity. In this paper, we study the problem of maximizing a monotone submodular function in the streaming setting with a cardinality constraint $k$. We first propose Sieve-Streaming++, which requires just…

In this paper, we consider the streaming memory-limited matrix completion problem when the observed entries are noisy versions of a small random fraction of the original entries. We are interested in scenarios where the matrix size is very large so the matrix is very hard to store and manipulate. Here, columns of the o…

Improved COD algorithm reduces streaming AMM errors and uses less space.

One-pass algorithm finds small subset for $\ell_p$ subspace approximation with additive error.

Bayesian online learning algorithm for one-pass data, achieving frequentist validity and uncertainty quantification.

Scalable methods for maximizing regularized submodular functions with improved memory and communication complexity.

Full-batch GD outperforms one-pass SGD in learning a single-index model with quadratic activation.

One-pass private sketch supports various machine learning tasks.

One-pass optimisation for high-dimensional hyperparameters.

New algorithm achieves nearly optimal regret with one-pass updates for GLB problems.

We consider streaming, one-pass principal component analysis (PCA), in the high-dimensional regime, with limited memory. Here, $p$-dimensional samples are presented sequentially, and the goal is to produce the $k$-dimensional subspace that best approximates these points. Standard algorithms require $O(p^2)$ memory; mea…

One-pass SGD dynamics in overparameterized quadratic networks show slow escape from poor solutions.

Sequential quantile estimation refers to incorporating observations into quantile estimates in an incremental fashion thus furnishing an online estimate of one or more quantiles at any given point in time. Sequential quantile estimation is also known as online quantile estimation. This area is relevant to the analysis …

We develop algorithms to learn non-linear dynamical systems without mixing assumptions.

Consider a sequence of closed, orientable surfaces of fixed genus $g$ in a Riemannian manifold $M$ with uniform upper bounds on mean curvature and area. We show that on passing to a subsequence and choosing appropriate parametrisations, the inclusion maps converge in $C^0$ to a map from a surface of genus $g$ to $M$. W…

We develop an efficient alternating framework for learning a generalized version of Factorization Machine (gFM) on steaming data with provable guarantees. When the instances are sampled from $d$ dimensional random Gaussian vectors and the target second order coefficient matrix in gFM is of rank $k$, our algorithm conve…

In this paper, we propose a one-pass algorithm on MapReduce for penalized linear regression \[f_λ(α, β) = \|Y - α\mathbf{1} - Xβ\|_2^2 + p_λ(β)\] where $α$ is the intercept which can be omitted depending on application; $β$ is the coefficients and $p_λ$ is the penalized function with penalizing parameter $λ$. $f_λ(α, β…

In many large-scale machine learning applications, data are accumulated with time, and thus, an appropriate model should be able to update in an online paradigm. Moreover, as the whole data volume is unknown when constructing the model, it is desired to scan each data item only once with a storage independent with the …

New method for RLHF reduces costs by integrating new data in one pass.

Real data are often with multiple modalities or from multiple heterogeneous sources, thus forming so-called multi-view data, which receives more and more attentions in machine learning. Multi-view clustering (MVC) becomes its important paradigm. In real-world applications, some views often suffer from instances missing…

Stochastic Gradient Descent can overfit after just a few passes, contrary to initial expectations.

A new method for estimating SW from streaming data.

Data stream classification methods demonstrate promising performance on a single data stream by exploring the cohesion in the data stream. However, multiple data streams that involve several correlated data streams are common in many practical scenarios, which can be viewed as multi-task data streams. Instead of handli…

Data stream clustering tackles real-time data processing challenges.

The equivariant Gromov--Hausdorff convergence of metric spaces is studied. Where all isometry groups under consideration are compact Lie, it is shown that an upper bound on the dimension of the group guarantees that the convergence is by Lie homomorphisms. Additional lower bounds on curvature and volume strengthen this…

Given a compact Riemann surface $X$ and a complex reductive Lie group $G$ equipped with real structures, we define antiholomorphic involutions on the moduli space of $G$-Higgs bundles over $X$. We investigate how the various components of the fixed point locus match up, as one passes from $G$ to its Langlands dual $^LG…

Bayesian tensor train method recovers streaming data with high accuracy.

stream-learn is a Python library for analyzing data streams with various drift types.

Polyak-Ruppert CLT for SA-Adam with momentum and non-convergent adaptive preconditioning

A new feature selection method using attention for neural networks.

PySAD offers a unified Python framework for efficient streaming anomaly detection.

A new method detects anomalies in multivariate streams without unit dependence.

New metrics needed for streaming ML due to delayed labels.

Nowadays, every device connected to the Internet generates an ever-growing stream of data (formally, unbounded). Machine Learning on unbounded data streams is a grand challenge due to its resource constraints. In fact, standard machine learning techniques are not able to deal with data whose statistics is subject to gr…

CROC identifies the earliest-changing stream as the root cause in multi-stream data.

The paper analyzes how repeating epochs affects data scaling in linear regression.

SDF adapts Deep Forest for evolving data streams with active learning.

Financial fraud detection in digital banking requires reasoning over multiple heterogeneous event streams.