arXiv research

Improved online Lasso reduces regret in sparse linear contextual bandits.

Improved online penalty selection for time series models.

Data driven soft sensor design has recently gained immense popularity, due to advances in sensory devices, and a growing interest in data mining. While partial least squares (PLS) is traditionally used in the process literature for designing soft sensors, the statistical literature has focused on sparse learners, such …

Online algorithm for probabilistic forecasting of conditional moments.

Scalable Bayesian LASSO using variational inference for large p and n.

Adaptive collection of data is commonplace in applications throughout science and engineering. From the point of view of statistical inference however, adaptive data collection induces memory and correlation in the samples, and poses significant challenge. We consider the high-dimensional linear regression, where the s…

We introduce a recursive adaptive group lasso algorithm for real-time penalized least squares prediction that produces a time sequence of optimal sparse predictor coefficient vectors. At each time index the proposed algorithm computes an exact update of the optimal $\ell_{1,\infty}$-penalized recursive least squares (R…

ALEXP improves model selection in linear bandits with exponential regret improvement.

Paper proposes a new dynamic pricing method with always-valid online statistical learning.

New online method for multivariate probabilistic electricity price forecasting.

Joint sparsity offers powerful structural cues for feature selection, especially for variables that are expected to demonstrate a "grouped" behavior. Such behavior is commonly modeled via group-lasso, multitask lasso, and related methods where feature selection is effected via mixed-norms. Several mixed-norm based spar…

Online graph learning from matrix-valued time series data.

New method optimizes hyperparameters for non-smooth problems efficiently.

Current online learning methods suffer issues such as lower convergence rates and limited capability to select important features compared to their offline counterparts. In this paper, a novel framework for online learning based on running averages is proposed. Many popular offline regularized methods such as Lasso, El…

Study shows high-dimensional sparse RL hardness and Lasso Q-iteration's nearly dimension-free regret.

Despite the recent progress towards efficient multiple kernel learning (MKL), the structured output case remains an open research front. Current approaches involve repeatedly solving a batch learning problem, which makes them inadequate for large scale scenarios. We propose a new family of online proximal algorithms fo…

A new algorithm reduces regret in high-dimensional online learning problems.

Paper proposes an online sparse linear regression method for streaming data.

New algorithms for batch decision-making with high-dimensional user data.

We solve ElasticNet regularization tuning across multiple instances with provable guarantees.

Spike sorting is a fundamental preprocessing step in neuroscience that is central to access simultaneous but distinct neuronal activities and therefore to better understand the animal or even human brain. But numerical complexity limits studies that require processing large scale datasets in terms of number of electrod…

A new algorithm reduces communication costs for collaborative decision-making across clients.

Unified framework for high-dimensional bandit problems with low-dimensional structures.

Coordinate descent (CD) algorithms have become the method of choice for solving a number of optimization problems in machine learning. They are particularly popular for training linear models, including linear support vector machine classification, LASSO regression, and logistic regression. We consider general CD with …

In this article we propose a new supervised ensemble learning method called Data Shared Adaptive Bootstrap Aggregated (AdaBag) Lasso for capturing low dimensional useful features for word based sentiment analysis and mining problems. The literature on ensemble methods is very rich in both statistics and machine learnin…

Online method learns sparse models efficiently in large scale settings.

This paper introduces a novel online inference method for high-dimensional GLMs.

A winning method for day-ahead electricity demand forecasting during and after the COVID-19 pandemic.

PLS-Lasso integrates dimension reduction into regression for financial index tracking.

The paper examines Adaptive Lasso and Transfer Lasso, highlighting their differences and proposing a new method.

KTVGL models tensor time series data for interpretable dynamic network estimation.

Develops a gradient-enhanced approach for online estimation in high-dimensional generalized linear models with streaming data.

DFR reduces the computational cost of sparse-group lasso and adaptive sparse-group lasso.

Estimates shared parameters across related learning problems using robust statistics and LASSO.

A fast method for Lasso and Logistic Lasso problems.

We leverage recent advances in high-dimensional statistics to derive new L2 estimation upper bounds for Lasso and Group Lasso in high-dimensions. For Lasso, our bounds scale as $(k^*/n) \log(p/k^*)$---$n\times p$ is the size of the design matrix and $k^*$ the dimension of the ground truth $\boldsymbolβ^*$---and match t…

We introduce an application of the group lasso to design of experiments. Note that we are NOT trying to explain experimental design for the group lasso. Conversely, we explain how we can use the idea of the group lasso in experimental design, showing that the problem of constructing an optimal design matrix can be tran…

Multi-Task Learning (MTL) can enhance a classifier's generalization performance by learning multiple related tasks simultaneously. Conventional MTL works under the offline or batch setting, and suffers from expensive training cost and poor scalability. To address such inefficiency issues, online learning techniques hav…

New method for tuning Graphical Lasso hyperparameters.

The Bayesian Lasso is constructed in the linear regression framework and applies the Gibbs sampling to estimate the regression parameters. This paper develops a new sparse learning model, named the Bayesian Lasso Sparse (BLS) model, that takes the hierarchical model formulation of the Bayesian Lasso. The main differenc…

Excessive computational cost for learning large data and streaming data can be alleviated by using stochastic algorithms, such as stochastic gradient descent and its variants. Recent advances improve stochastic algorithms on convergence speed, adaptivity and structural awareness. However, distributional aspects of thes…

LLM-Lasso uses LLMs to improve feature selection in Lasso regression.

The solution path of the 1D fused lasso for an $n$-dimensional input is piecewise linear with $\mathcal{O}(n)$ segments (Hoefling et al. 2010 and Tibshirani et al 2011). However, existing proofs of this bound do not hold for the weighted fused lasso. At the same time, results for the generalized lasso, of which the wei…

Proposes MM-DUST for efficient generalized lasso solution paths.

Bayesian approach improves network lasso for multi-task learning.

We propose a method for finding alternate features missing in the Lasso optimal solution. In ordinary Lasso problem, one global optimum is obtained and the resulting features are interpreted as task-relevant features. However, this can overlook possibly relevant features not selected by the Lasso. With the proposed met…

This review summarizes five Lasso optimization algorithms.

In high dimensional settings, sparse structures are crucial for efficiency, both in term of memory, computation and performance. It is customary to consider $\ell_1$ penalty to enforce sparsity in such scenarios. Sparsity enforcing methods, the Lasso being a canonical example, are popular candidates to address high dim…