arXiv research

Theoretical guarantees for STE, a robust subspace recovery method.

Paper proves linear convergence of SCMS algorithm for directional data.

We investigate a Gaussian mixture model (GMM) with component means constrained in a pre-selected subspace. Applications to classification and clustering are explored. An EM-type estimation algorithm is derived. We prove that the subspace containing the component means of a GMM with a common covariance matrix also conta…

Unified framework for clustering with sparse convex combinations.

Unified framework for structured principal subspace estimation with bounds and rates.

Study shows how varying levels of supervision and orthonormality constraints affect generalization errors in subspace fitting.

High-dimensional data often lie in low-dimensional subspaces corresponding to different classes they belong to. Finding sparse representations of data points in a dictionary built using the collection of data helps to uncover low-dimensional subspaces and address problems such as clustering, classification, subset sele…

Modern information processing relies on the axiom that high-dimensional data lie near low-dimensional geometric structures. This paper revisits the problem of data-driven learning of these geometric structures and puts forth two new nonlinear geometric models for data describing "related" objects/phenomena. The first o…

Reconstruction based subspace clustering methods compute a self reconstruction matrix over the samples and use it for spectral clustering to obtain the final clustering result. Their success largely relies on the assumption that the underlying subspaces are independent, which, however, does not always hold in the appli…

Study geodesics in constrained curve spaces, including elastic curves and concentric circles.

New method solves saddle-point problems faster than existing methods.

Multi-view clustering is an important and fundamental problem. Many multi-view subspace clustering methods have been proposed, and most of them assume that all views share a same coefficient matrix. However, the underlying information of multi-view data are not fully exploited under this assumption, since the coefficie…

We study sparse principal components analysis in high dimensions, where $p$ (the number of variables) can be much larger than $n$ (the number of observations), and analyze the problem of estimating the subspace spanned by the principal eigenvectors of the population covariance matrix. We introduce two complementary not…

Most existing approaches address multi-view subspace clustering problem by constructing the affinity matrix on each view separately and afterwards propose how to extend spectral clustering algorithm to handle multi-view data. This paper presents an approach to multi-view subspace clustering that learns a joint subspace…

While the topology of the space of all smooth immersed curves on the $2$-sphere $\mathbb{S}^2$ that start and end at given points in given directions is well known, it is an open problem to understand the homotopy type of its subspaces consisting of the curves whose geodesic curvatures are constrained to a prescribed p…

FMDP-BF algorithm improves RL in factored MDPs with exponential regret reduction.

Subspace clustering is a growing field of unsupervised learning that has gained much popularity in the computer vision community. Applications can be found in areas such as motion segmentation and face clustering. It assumes that data originate from a union of subspaces, and clusters the data depending on the correspon…

New method prevents forgetting in LLMs by dynamically identifying task-specific subspaces.

In this paper, we present GASG21 (Grassmannian Adaptive Stochastic Gradient for $L_{2,1}$ norm minimization), an adaptive stochastic gradient algorithm to robustly recover the low-rank subspace from a large matrix. In the presence of column outliers, we reformulate the batch mode matrix $L_{2,1}$ norm minimization with…

Bayesian SHMM models speech units from unannotated speech.

Modern machine learning algorithms have been adopted in a range of signal-processing applications spanning computer vision, natural language processing, and artificial intelligence. Many relevant problems involve subspace-structured features, orthogonality constrained or low-rank constrained objective functions, or sub…

A problem of considerable importance within the field of uncertainty quantification (UQ) is the development of efficient methods for the construction of accurate surrogate models. Such efforts are particularly important to applications constrained by high-dimensional uncertain parameter spaces. The difficulty of accura…

Unified framework for nonconvex matrix completion with linearly parameterized factors.

Paper unifies subspace identification and DMD for dynamical systems.

The paper proposes methods to find a shared active subspace for multivariate vector-valued functions.

New algorithm improves multitask learning across diverse agents.

Clustering in high-dimensional spaces is nowadays a recurrent problem in many scientific domains but remains a difficult task from both the clustering accuracy and the result understanding points of view. This paper presents a discriminative latent mixture (DLM) model which fits the data in a latent orthonormal discrim…

Study shows contractibility of certain metrics on 3-manifolds.

RSAC improves lightweight continuous learning efficiency.

We prove that the critical points of various energies such as the area, the Willmore energy, the frame energy for tori...etc among possibly branched immersions constrained to evolve within a smooth sub-manifold of the Teichmüller space satisfy the corresponding constrained Euler Lagrange equation. We deduce that critic…

SBMs learn manifold-like structures by mixing samples with a non-conservative field.

New algorithm for nonconvex optimization on constrained Riemannian manifolds converges quickly.

A new diffusion sampling method combines Krylov subspace and diffusion models for faster and more efficient inverse problems.

LR-EDNN reduces PDE solver complexity by limiting network weights to low-rank subspace.

The techniques and analysis presented in this thesis provide new methods to solve optimization problems posed on Riemannian manifolds. These methods are applied to the subspace tracking problem found in adaptive signal processing and adaptive control. A new point of view is offered for the constrained optimization prob…

SCOPE iteratively optimizes sparsity-constrained problems without tuning hyperparameters.

We will develop simple relations between the arc-lengths of a pair of geodesics that share common end-points. The two geodesics differ only by the requirement that one is constrained to lie in a subspace of the parent manifold. We will present two applications of our results. In the first example we explore the converg…

New methods reduce computational cost for Gaussian Markov Random Fields with sparse constraints.

Subspace learning and matrix factorization problems have great many applications in science and engineering, and efficient algorithms are critical as dataset sizes continue to grow. Many relevant problem formulations are non-convex, and in a variety of contexts it has been observed that solving the non-convex problem d…

It has been observed in a variety of contexts that gradient descent methods have great success in solving low-rank matrix factorization problems, despite the relevant problem formulation being non-convex. We tackle a particular instance of this scenario, where we seek the $d$-dimensional subspace spanned by a streaming…

Weakly-supervised RL identifies meaningful tasks, improving performance in complex environments.

MCD offers a complete model understanding for high-stake decisions.

We study the problem of switching-constrained online convex optimization (OCO), where the player has a limited number of opportunities to change her action. While the discrete analog of this online learning task has been studied extensively, previous work in the continuous setting has neither established the minimax ra…

Improved private AdaGrad achieves faster convergence rates for convex functions.

Representation of human actions as a sequence of human body movements or action attributes enables the development of models for human activity recognition and summarization. We present an extension of the low-rank representation (LRR) model, termed the clustering-aware structure-constrained low-rank representation (CS…

Robust high-dimensional data processing has witnessed an exciting development in recent years, as theoretical results have shown that it is possible using convex programming to optimize data fit to a low-rank component plus a sparse outlier component. This problem is also known as Robust PCA, and it has found applicati…

New method learns shared structures in non-linear tasks.

Paper tackles distribution shifts in prediction models with unobserved confounding.