arXiv research

Develops algorithms for sparse signal reconstruction without needing signal sparsity or noise variance.

We consider the Orthogonal Least-Squares (OLS) algorithm for the recovery of a $m$-dimensional $k$-sparse signal from a low number of noisy linear measurements. The Exact Recovery Condition (ERC) in bounded noisy scenario is established for OLS under certain condition on nonzero elements of the signal. The new result a…

Orthogonal matching pursuit (OMP) is a widely used compressive sensing (CS) algorithm for recovering sparse signals in noisy linear regression models. The performance of OMP depends on its stopping criteria (SC). SC for OMP discussed in literature typically assumes knowledge of either the sparsity of the signal to be e…

OMP improves text classification accuracy with sparse models.

This paper improves OMP-based sparse subspace clustering with data-adaptive capability.

Distributed-OMP recovers sparse vectors with low communication costs.

Sparsity-based subspace clustering algorithms have attracted significant attention thanks to their excellent performance in practical applications. A prominent example is the sparse subspace clustering (SSC) algorithm by Elhamifar and Vidal, which performs spectral clustering based on an adjacency matrix obtained by sp…

New technique RRT improves OMP performance without knowing sparsity or noise.

A new algorithm improves SSC clustering accuracy with low complexity.

We propose a novel application of the Simultaneous Orthogonal Matching Pursuit (S-OMP) procedure for sparsistant variable selection in ultra-high dimensional multi-task regression problems. Screening of variables, as introduced in \cite{fan08sis}, is an efficient and highly scalable way to remove many irrelevant variab…

In this paper, we present new results on using orthogonal matching pursuit (OMP), to solve the sparse approximation problem over redundant dictionaries for complex cases (i.e., complex measurement vector, complex dictionary and complex additive white Gaussian noise (CAWGN)). A sufficient condition that OMP can recover …

Many models for sparse regression typically assume that the covariates are known completely, and without noise. Particularly in high-dimensional applications, this is often not the case. This paper develops efficient OMP-like algorithms to deal with precisely this setting. Our algorithms are as efficient as OMP, and im…

Paper proposes an accelerated algorithm for sparse subspace clustering.

A fast feature selection method using OLS and SOCC for classification.

The OLS estimator optimally identifies stable linear systems with a finite number of samples.

This study examines the relationship between PLS and OLS regression using eigenvalue distributions.

OLS predictions are shown to be similar to attention mechanisms in models.

Every 4-dimensional infrasolvmanifold $M$ with $β_1(M;\mathbb{Q})>0$ or which is flat or has one of the geometries $\mathbb{N}il^4$, $\mathbb{S}ol_{m,n}^4$, or $\mathbb{S}ol_0^4$ bounds. However there are non-orientable $\mathbb{S}ol_1^4$-manifolds which do not bound. The question remains open for $\mathbb{N}il^3\times…

Feature selection and regularization are becoming increasingly prominent tools in the efforts of the reinforcement learning (RL) community to expand the reach and applicability of RL. One approach to the problem of feature selection is to impose a sparsity-inducing form of regularization on the learning method. Recent …

We show that if $M$ is an orientable 4-dimensional infrasolvmanifold and either $β=β_1(M;\mathbb{Q})\geq2$ or $M$ is a $\mathbb{S}ol_0^4$- or a $\mathbb{S}ol_{m,n}^4$-manifold (with $m\not=n$) then $M$ is parallelizable. There are non-parallelizable examples with $β=1$ for each of the other solvable Lie geometries $\ma…

Paper proves noise-tolerant SSC using greedy methods under coherence conditions.

Develops methods for reliable inference on batched bandit data.

Polynomial Chaos Expansion improves operator learning for PDEs.

New techniques for compressive sensing without noise or signal statistics.

The performance of Orthogonal Matching Pursuit (OMP) for variable selection is analyzed for random designs. When contrasted with the deterministic case, since the performance is here measured after averaging over the distribution of the design matrix, one can have far less stringent sparsity constraints on the coeffici…

We show that $\mathbb{S}ol^3\times\mathbb{E}^1$-manifolds are Seifert fibred, with general fibre the torus, and base one of the seven flat 2-orbifolds $T, Kb, \mathbb{A}, \mathbb{M}b, S(2,2,2,2), P(2,2)$ or $\mathbb{D}(2,2)$, and outline a classification of such 4-manifolds.

PCA-based dimensionality reduction improves robustness in overparameterized linear models.

Unified framework for online learning in click prediction for search ads.

Lower bounds show OLS outperforms basis pursuit in overparameterized linear regression.

GRRT recovers sparse signals without prior sparsity or noise variance knowledge.

Orthogonal Matching Pursuit (OMP) has long been considered a powerful heuristic for attacking compressive sensing problems; however, its theoretical development is, unfortunately, somewhat lacking. This paper presents an improved Restricted Isometry Property (RIP) based performance guarantee for T-sparse signal reconst…

In this paper, we consider the problem of compressed sensing where the goal is to recover almost all the sparse vectors using a small number of fixed linear measurements. For this problem, we propose a novel partial hard-thresholding operator that leads to a general family of iterative algorithms. While one extreme of …

We consider new formulations and methods for sparse quantile regression in the high-dimensional setting. Quantile regression plays an important role in many applications, including outlier-robust exploratory analysis in gene selection. In addition, the sparsity consideration in quantile regression enables the explorati…

ARHT algorithm improves sparsity guarantees in convex optimization.

When the design matrix has orthonormal columns, "soft thresholding" the ordinary least squares (OLS) solution produces the Lasso solution [Tibshirani, 1996]. If one uses the Puffer preconditioned Lasso [Jia and Rohe, 2012], then this result generalizes from orthonormal designs to full rank designs (Theorem 1). Theorem …

We study a robust optimal stopping problem with respect to a set $\cP$ of mutually singular probabilities. This can be interpreted as a zero-sum controller-stopper game in which the stopper is trying to maximize its pay-off while an adverse player wants to minimize this payoff by choosing an evaluation criteria from $\…

Improved privacy-preserving linear regression via iterative Hessian mixing.

New algorithm selects variables from large datasets.

The paper tackles system identification via Hankel nuclear norm regularization, improving estimation rates and singular value gaps.

We compare the random group model of Gromov and the model of generic groups of Arzhantseva and Ol'shanskii.

OLS is a special case of Transformer, revealing its linear nature.

Ordinary least squares (OLS) is the default method for fitting linear models, but is not applicable for problems with dimensionality larger than the sample size. For these problems, we advocate the use of a generalized version of OLS motivated by ridge regression, and propose two novel three-step algorithms involving l…

Enhanced LSTM predicts equity trends, outperforming traditional methods.

We consider the question of learning in general topological vector spaces. By exploiting known (or parametrized) covariance structures, our Main Theorem demonstrates that any continuous linear map corresponds to a certain isomorphism of embedded Hilbert spaces. By inverting this isomorphism and extending continuously, …

We consider the task of robust non-linear regression in the presence of both inlier noise and outliers. Assuming that the unknown non-linear function belongs to a Reproducing Kernel Hilbert Space (RKHS), our goal is to estimate the set of the associated unknown parameters. Due to the presence of outliers, common techni…

Study combines SEM, OLS, and DML for robustness checks in survey-based research.

This work develops fast and accurate ROMs for AM models using OL methods.

The purpose of this note is to present several criteria for essential self-adjointness. The method is based on ideas due to Shubin. This note is divided into two parts. The first part deals with symmetric first order systems on the line in the most general setting. Such a symmetric first order system of differential eq…