arXiv research

New algorithm accelerates single-pass SGD for generalized linear prediction.

SPACR trains uncertainty-aware regressors directly within a single pass, improving efficiency and validity.

Improved single-pass streaming MAB regret bound to O(K^1/3T^2/3).

We consider stochastic gradient descent (SGD) for least-squares regression with potentially several passes over the data. While several passes have been widely reported to perform practically better in terms of predictive performance on unseen data, the existing theoretical analysis of SGD suggests that a single pass i…

A new single-pass algorithm improves sparse PCA under limited computational resources.

A new classifier encodes local neighborhoods for each class using Fly Bloom Filters.

New framework models uncertainty in classification debates.

We derive a single pass algorithm for computing the gradient and Fisher information of Vecchia's Gaussian process loglikelihood approximation, which provides a computationally efficient means for applying the Fisher scoring algorithm for maximizing the loglikelihood. The advantages of the optimization techniques are de…

Proposes PSCs for UQ in deep nets without retraining.

Single-pass method estimates neural network uncertainty.

In this paper we present a new algorithm for computing a low rank approximation of the product $A^TB$ by taking only a single pass of the two matrices $A$ and $B$. The straightforward way to do this is to (a) first sketch $A$ and $B$ individually, and then (b) find the top components using PCA on the sketch. Our algori…

PySAD offers a unified Python framework for efficient streaming anomaly detection.

Generative networks trained with sketching reduce computational burden.

Improved algorithm reduces stochastic gradient complexity for large-scale learning problems.

We develop and analyze a procedure for gradient-based optimization that we refer to as stochastically controlled stochastic gradient (SCSG). As a member of the SVRG family of algorithms, SCSG makes use of gradient estimates at two scales, with the number of updates at the faster scale being governed by a geometric rand…

Efficient streaming algorithms for robust statistics with near-optimal memory.

We develop efficient algorithms for robust PCA that handle outliers.

Improved DP optimization for nonconvex, nonsmooth objectives with reduced sample complexity.

We consider the optimization of a quadratic objective function whose gradients are only accessible through a stochastic oracle that returns the gradient at any given point plus a zero-mean finite variance random error. We present the first algorithm that achieves jointly the optimal prediction error rates for least-squ…

In many estimation problems, e.g. linear and logistic regression, we wish to minimize an unknown objective given only unbiased samples of the objective function. Furthermore, we aim to achieve this using as few samples as possible. In the absence of computational constraints, the minimizer of a sample average of observ…

Two-layer networks learn faster with batch reuse, overcoming information and leap exponents.

We present a streaming model for large-scale classification (in the context of $\ell_2$-SVM) by leveraging connections between learning and computational geometry. The streaming model imposes the constraint that only a single pass over the data is allowed. The $\ell_2$-SVM is known to have an equivalent formulation in …

Optimizes private learning with differential privacy for LASSO problems.

New algorithms minimize regret in streaming MAB with memory constraints.

Given a collection of categorical data, we want to find the parameters of a Dirichlet distribution which maximizes the likelihood of that data. Newton's method is typically used for this purpose but current implementations require reading through the entire dataset on each iteration. In this paper, we propose a modific…

DP-PCA improves privacy in PCA computations with optimal statistical error.

The brain optimizes memory by forgetting what's predictable, improving generalization.

FinReflectKG builds a comprehensive financial knowledge graph from SEC filings, improving extraction quality.

AUC (Area under the ROC curve) is an important performance measure for applications where the data is highly imbalanced. Learning to maximize AUC performance is thus an important research problem. Using a max-margin based surrogate loss function, AUC optimization problem can be approximated as a pairwise rankSVM learni…

Online structure learning approaches, such as those stemming from Statistical Relational Learning, enable the discovery of complex relations in noisy data streams. However, these methods assume the existence of fully-labelled training data, which is unrealistic for most real-world applications. We present a novel appro…

A scalable method for heavy-tailed gradients using cheap stochastic sub-processes.

FinReflectKG - EvalBench benchmarks financial KG extraction from SEC 10-K filings.

We present SDA-Bayes, a framework for (S)treaming, (D)istributed, (A)synchronous computation of a Bayesian posterior. The framework makes streaming updates to the estimated posterior according to a user-specified approximation batch primitive. We demonstrate the usefulness of our framework, with variational Bayes (VB) …

This note explores probabilistic sampling weighted by uncertainty in active learning. This method has been previously used and authors have tangentially remarked on its efficacy. The scheme has several benefits: (1) it is computationally cheap, (2) it can be implemented in a single-pass streaming fashion which is a ben…

Numerous algorithms are used for nonnegative matrix factorization under the assumption that the matrix is nearly separable. In this paper, we show how to make these algorithms efficient for data matrices that have many more rows than columns, so-called "tall-and-skinny matrices". One key component to these improved met…

Paper analyzes SGD in kernel regression, showing it outperforms offline methods.

Improved image synthesis with user scribbles and text prompts.

Improved uncertainty estimation in neural networks with VBLL.

Kernel approximation using randomized feature maps has recently gained a lot of interest. In this work, we identify that previous approaches for polynomial kernel approximation create maps that are rank deficient, and therefore do not utilize the capacity of the projected feature space effectively. To address this chal…

New algorithms for online MAP inference and learning for NDPPs.

Hybrid model improves sequential data prediction by combining neural and time series models.

This paper provides fast estimates for complex option types.

New MCMC algorithm reduces subset selection passes to 2 for optimal $k$-dimensional subspace approximation.

This paper introduces a novel online inference method for high-dimensional GLMs.

The Denoising Autoencoder (DAE) enhances the flexibility of the data stream method in exploiting unlabeled samples. Nonetheless, the feasibility of DAE for data stream analytic deserves an in-depth study because it characterizes a fixed network capacity that cannot adapt to rapidly changing environments. Deep evolving …

Itô maps provide a method for any-step SDE integration.

COMET is a single-pass MapReduce algorithm for learning on large-scale data. It builds multiple random forest ensembles on distributed blocks of data and merges them into a mega-ensemble. This approach is appropriate when learning from massive-scale data that is too large to fit on a single machine. To get the best acc…

This paper analyzes multi-pass SGD for least squares, improving generalization bounds.