arXiv research

Proposes FARM model combining latent factor and sparse regression.

Sparse linear models improve neural network debuggability.

In machine learning and data mining, linear models have been widely used to model the response as parametric linear functions of the predictors. To relax such stringent assumptions made by parametric linear models, additive models consider the response to be a summation of unknown transformations applied on the predict…

A new estimator learns sparse linear models with context-dependent coefficients.

IDS improves sparse linear bandits by balancing information and regret.

NGSLL combines DNN accuracy with linear model interpretability.

Sparse generalized additive models (GAMs) are an extension of sparse generalized linear models which allow a model's prediction to vary non-linearly with an input variable. This enables the data analyst build more accurate models, especially when the linearity assumption is known to be a poor approximation of reality. …

In this paper we consider sparse and identifiable linear latent variable (factor) and linear Bayesian network models for parsimonious analysis of multivariate data. We propose a computationally efficient method for joint parameter and model inference, and model comparison. It consists of a fully Bayesian hierarchy for …

Fast classification for sparse models, even with correlated features.

This paper explores loss landscapes of sparse neural networks, finding unique characteristics compared to dense networks.

Study shows inefficiency of sparse linear regression learning with fewer than Ω(k^2) samples.

Efficient algorithms recover two sparse models from a mix of linear queries.

We present a signal representation framework called the sparse manifold transform that combines key ideas from sparse coding, manifold learning, and slow feature analysis. It turns non-linear transformations in the primary sensory signal space into linear interpolations in a representational embedding space while maint…

PopArt efficiently solves sparse linear bandits with tighter recovery guarantees.

We analyze the computational complexity of Quantum Sparse Support Vector Machine, a linear classifier that minimizes the hinge loss and the $L_1$ norm of the feature weights vector and relies on a quantum linear programming solver instead of a classical solver. Sparse SVM leads to sparse models that use only a small fr…

FLIPHAT addresses joint differential privacy for high-dimensional sparse linear bandits.

Many conventional statistical procedures are extremely sensitive to seemingly minor deviations from modeling assumptions. This problem is exacerbated in modern high-dimensional settings, where the problem dimension can grow with and possibly exceed the sample size. We consider the problem of robust estimation of sparse…

Study on recovering supports of multiple sparse vectors from mixed linear measurements.

New CRM models for sparse networks with linear edge growth.

Proposes EM for sparse horseshoe estimation.

New method discovers causal relationships in sparse linear data.

OOMP selects features online for sparse linear regression.

Pruning method removes less important features in linear models.

The generalized linear model (GLM) plays a key role in regression analyses. In high-dimensional data, the sparse GLM has been used but it is not robust against outliers. Recently, the robust methods have been proposed for the specific example of the sparse GLM. Among them, we focus on the robust and sparse linear regre…

New bounds for high-dimensional sparse linear bandits, balancing information and regret.

Study finds exact limits for sparse regression with fewer observations than usual.

Is it possible to find the sparsest vector (direction) in a generic subspace $\mathcal{S} \subseteq \mathbb{R}^p$ with $\mathrm{dim}(\mathcal{S})= n < p$? This problem can be considered a homogeneous variant of the sparse recovery problem, and finds connections to sparse dictionary learning, sparse PCA, and many other …

Proposes $\ell_0$-CCA for sparse CCA with improved representation learning.

RFM reduces feature space for linear models, improving sparse recovery.

A new framework for interpretable models using sparse linear layers.

High dimensional sparse learning has imposed a great computational challenge to large scale data analysis. In this paper, we are interested in a broad class of sparse learning approaches formulated as linear programs parametrized by a {\em regularization factor}, and solve them by the parametric simplex method (PSM). O…

Random non-linear Fourier features have recently shown remarkable performance in a wide-range of regression and classification applications. Motivated by this success, this article focuses on a sparse non-linear Fourier feature (NFF) model. We provide a characterization of the sufficient number of data points that guar…

PROBE algorithm efficiently solves sparse high-dimensional linear regression.

Efficiently finds sparse solutions to max-plus equations for convex regression.

New model leads to optimal test loss in sparse linear regression.

Flexible Bayesian approach for generalized linear models, especially for sparse logistic regression.

Paper analyzes sparse aggregation in GLMs with Kullback-Leibler risk bounds.

The importance of interpretability of machine learning models has been increasing due to emerging enterprise predictive analytics, threat of data privacy, accountability of artificial intelligence in society, and so on. Piecewise linear models have been actively studied to achieve both accuracy and interpretability. Th…

Linear Mixed Models (LMMs) are important tools in statistical genetics. When used for feature selection, they allow to find a sparse set of genetic traits that best predict a continuous phenotype of interest, while simultaneously correcting for various confounding factors such as age, ethnicity and population structure…

We propose a nonparametric method for detecting nonlinear causal relationship within a set of multidimensional discrete time series, by using sparse additive models (SpAMs). We show that, when the input to the SpAM is a $β$-mixing time series, the model can be fitted by first approximating each unknown function with a …

New method reduces SBL complexity from cubic to linear, improving scalability.

In a recent work (arXiv:0910.2517), for nonlinear models with sparse underlying linear structures, we studied the error bounds of $\ell_0$-regularized estimation. In this note, we show that $\ell_1$-regularized estimation in some important cases can achieve the same order of error bounds as those in the aforementioned …

The paper analyzes sparse high-dimensional linear regression with random design and unknown error variance, providing adaptiveness and concentration rates.

This paper proposes a fast and accurate method for sparse regression in the presence of missing data. The underlying statistical model encapsulates the low-dimensional structure of the incomplete data matrix and the sparsity of the regression coefficients, and the proposed algorithm jointly learns the low-dimensional s…

New method for linear bandits with unknown sparsity, improving sparse regret bounds.

Study iterative regularization for linear models with convex bias, improving robust sparse recovery.

Paper develops an efficient method for conformal prediction in sparse linear models.

Study improves materials discovery for high-entropy alloys using sparse linear models.