arXiv research

Extends online linear regression to handle multivariate data.

OOMP selects features online for sparse linear regression.

Study on online regression with noise, achieving near-optimal regret bounds.

Paper proposes an online sparse linear regression method for streaming data.

Study on deleting user data in linear regression models to maintain limited memory.

New CLT for SGD in high-dimensional regression provides online inference.

Paper addresses online identification and clustering for mixed linear regression models.

Optimal online linear regression in dynamic environments using discounted Vovk-Azoury-Warmuth forecaster.

There has been a growing interest in using non-parametric regression methods like Gaussian Process (GP) regression for system identification. GP regression does traditionally have three important downsides: (1) it is computationally intensive, (2) it cannot efficiently implement newly obtained measurements online, and …

Proposes an online method for high-dimensional streaming data.

Study online linear regression with paid noise reduction.

Paper studies Gaussian approximation in linear regression with rates derived.

Robust multivariate linear regression methods for online and offline use.

Proposes online debiasing estimators for adaptive linear regression.

We establish optimal rates for online regression for arbitrary classes of regression functions in terms of the sequential entropy introduced in (Rakhlin, Sridharan, Tewari, 2010). The optimal rates are shown to exhibit a phase transition analogous to the i.i.d./statistical learning case, studied in (Rakhlin, Sridharan,…

We consider the problem of online linear regression on arbitrary deterministic sequences when the ambient dimension d can be much larger than the number of time rounds T. We introduce the notion of sparsity regret bound, which is a deterministic online counterpart of recent risk bounds derived in the stochastic setting…

Online learning of linear operators between infinite-dimensional spaces is possible but with limitations.

We consider the problem of online active learning to collect data for regression modeling. Specifically, we consider a decision maker with a limited experimentation budget who must efficiently learn an underlying linear population model. Our main contribution is a novel threshold-based algorithm for selection of most i…

Online active regression algorithms minimize label queries for efficient data regression.

FOLKLORE algorithm speeds up online multiclass logistic regression.

G-GLN extends GLNs to multiple regression and density modeling.

Online Passive-Aggressive (PA) learning is a class of online margin-based algorithms suitable for a wide range of real-time prediction tasks, including classification and regression. PA algorithms are formulated in terms of deterministic point-estimation problems governed by a set of user-defined hyperparameters: the a…

SA algorithms control dynamic regret in non-stationary settings with strong convexity or exp-concavity.

New method tackles endogeneity in online learning with improved regret bounds.

Robust method estimates state, input, and parameters of linear systems online.

Study shows pretraining and finetuning can effectively tackle covariate shift in linear regression.

Algorithm adapts to shifting domains with minimal label queries.

We present a framework for supervised subspace tracking, when there are two time series $x_t$ and $y_t$, one being the high-dimensional predictors and the other being the response variables and the subspace tracking needs to take into consideration of both sequences. It extends the classic online subspace tracking work…

This paper establishes minimax rates for online regression with arbitrary classes of functions and general losses. We show that below a certain threshold for the complexity of the function class, the minimax rates depend on both the curvature of the loss function and the sequential complexities of the class. Above this…

New bounds on self-normalized martingales improve online linear regression performance.

Polynomial-time RL algorithm for constant actions under linear Bellman completeness.

SRNNs achieve LSTMs' online regression performance in less time.

Designs efficient algorithms for online and sliding window models of subspace embeddings for all p.

Algorithm minimizes regret while adhering to unknown safety constraints.

Fast detection of changepoints in linear regression models.

Stochastic gradient descent on l1 loss converges to true parameter in online robust regression.

Online algorithm for probabilistic forecasting of conditional moments.

Linear regression is arguably the most prominent among statistical inference methods, popular both for its simplicity as well as its broad applicability. On par with data-intensive applications, the sheer size of linear regression problems creates an ever growing demand for quick and cost efficient solvers. Fortunately…

The article describe the model, derivation, and implementation of variational Bayesian inference for linear and logistic regression, both with and without automatic relevance determination. It has the dual function of acting as a tutorial for the derivation of variational Bayesian inference for simple models, as well a…

Develops algorithms for CCBs with non-linear costs, improving safety and performance.

We consider the setting of online linear regression for arbitrary deterministic sequences, with the square loss. We are interested in the aim set by Bartlett et al. (2015): obtain regret bounds that hold uniformly over all competitor vectors. When the feature sequence is known at the beginning of the game, they provide…

As an automatic method of determining model complexity using the training data alone, Bayesian linear regression provides us a principled way to select hyperparameters. But one often needs approximation inference if distribution assumption is beyond Gaussian distribution. In this paper, we propose a Bayesian linear reg…

New analysis improves SGD for robust and quantile regression with sub-quadratic convergence.

New algorithms robust to adversarial data achieve optimal performance.

Coordinate descent (CD) algorithms have become the method of choice for solving a number of optimization problems in machine learning. They are particularly popular for training linear models, including linear support vector machine classification, LASSO regression, and logistic regression. We consider general CD with …

We consider the problem of online linear regression on individual sequences. The goal in this paper is for the forecaster to output sequential predictions which are, after $T$ time rounds, almost as good as the ones output by the best linear predictor in a given $\ell^1$-ball in $\\R^d$. We consider both the cases wher…

GD outperforms ridge regression and SGD in linear regression problems.

We study online linear regression problems in a distributed setting, where the data is spread over a network. In each round, each network node proposes a linear predictor, with the objective of fitting the \emph{network-wide} data. It then updates its predictor for the next round according to the received local feedbac…