Estimates Markov chains from data with a non-convex rank-constrained approach.
problem Estimating low-rank Markov chains from empirical data.
method Rank-constrained likelihood maximization and DC programming.
result The proposed estimator achieves better empirical performance than other methods.
New conditions ensure Dantzig-Wolfe relaxation matches rank-constrained optimization problems.
problem Rank-constrained optimization problems with linear matrix inequalities.
method Investigates Dantzig-Wolfe relaxation and develops conditions for exactness.
result Conditions for extreme point, convex hull, and objective exactness.
Optimizing the acquisition matrix is useful for compressed sensing of signals that are sparse in overcomplete dictionaries, because the acquisition matrix can be adapted to the particular correlations of the dictionary atoms. In this paper a novel formulation of the optimization problem is proposed, in the form of a ra…
Proximal splitting methods solve rank-constrained convex problems locally.
problem Solving optimization problems with rank constraints.
method Proximal splitting algorithms with conditions on rank constraint convex envelopes.
result Proximal splitting methods converge locally to solutions under convex relaxation conditions.
New algorithm reduces rank constrained optimization problems.
problem Rank constrained optimization problems in machine learning and statistics.
method Recursive Importance Sketching (RISRO) algorithm.
result RISRO offers clear advantages over existing algorithms and converges efficiently.
A number of statistical estimation problems can be addressed by semidefinite programs (SDP). While SDPs are solvable in polynomial time using interior point methods, in practice generic SDP solvers do not scale well to high-dimensional problems. In order to cope with this problem, Burer and Monteiro proposed a non-conv…
Improved understanding of low-rank solutions in SDPs via smoothed analysis.
problem Finding low-rank solutions to semidefinite programs efficiently.
method Penalty function formulation and smoothed analysis to avoid worst-case matrices.
result All approximate local optima are global optima for rank-constrained SDPs under certain conditions.
Large CNNs have delivered impressive performance in various computer vision applications. But the storage and computation requirements make it problematic for deploying these models on mobile devices. Recently, tensor decompositions have been used for speeding up CNNs. In this paper, we further develop the tensor decom…
Supervised linear feature extraction can be achieved by fitting a reduced rank multivariate model. This paper studies rank penalized and rank constrained vector generalized linear models. From the perspective of thresholding rules, we build a framework for fitting singular value penalized models and use it for feature …
This work studies the linear approximation of high-dimensional dynamical systems using low-rank dynamic mode decomposition (DMD). Searching this approximation in a data-driven approach is formalised as attempting to solve a low-rank constrained optimisation problem. This problem is non-convex and state-of-the-art algor…
This paper studies simultaneous feature selection and extraction in supervised and unsupervised learning. We propose and investigate selective reduced rank regression for constructing optimal explanatory factors from a parsimonious subset of input features. The proposed estimators enjoy sharp oracle inequalities, and w…
We tackle binary tensor decomposition with a multilinear model and likelihood-based estimation.
problem Decomposing binary tensors with probabilistic models.
method Multilinear Bernoulli model, rank-constrained likelihood estimation, alternating optimization.
result The estimation error bound is established and shown to be minimax optimal.
Paper unifies subspace identification and DMD for dynamical systems.
problem Estimating dynamical models from data.
method Unified optimization and regression problems for SID and DMD.
result Proves equivalence of SID and DMD for optimal model construction.
The paper identifies universal features for high-dimensional data inference.
problem Identifying universal low-dimensional features from high-dimensional data for inference tasks.
method Introduces natural notions of universality and shows a local equivalence among them, using information geometry.
result Reveals the complementary roles of various data analysis techniques.
New method estimates and completes tensors from ordinal data, improving accuracy and efficiency.
problem Estimating and completing tensors from incomplete, ordinal observations.
method Multi-linear cumulative link model with rank-constrained M-estimator.
result The proposed estimator achieves faster convergence and is minimax optimal.
Introduces Grassmannian learning to incorporate geometric features in machine learning.
problem Subspace-structured features, orthogonality constraints, and low-rank constraints in machine learning.
method Studies the Grassmann manifold to solve mathematical problems in shallow and deep learning.
result Improvements in performance of classic and deep learning algorithms using Grassmannian learning.
Optimization problems with rank constraints appear in many diverse fields such as control, machine learning and image analysis. Since the rank constraint is non-convex, these problems are often approximately solved via convex relaxations. Nuclear norm regularization is the prevailing convexifying technique for dealing …
Efficient tensor decomposition for count data models achieves near-optimal multiway analysis.
problem Efficient tensor decomposition for count data models.
method Rank-constrained maximum-likelihood estimator for tensor decomposition.
result Achieves multiway analysis with variance matching Cramér-Rao Lower Bound up to constants and logarithmic factors.
Proposes a method to identify subgroup structure and estimate covariate effects for multivariate response data.
problem Identifying subgroup structure and estimating covariate effects in multivariate response data.
method Joint heterogeneity and reduced-rank learning framework using rank-constrained pairwise fusion penalization.
result Established the asymptotic properties of the estimators and proposed a predictive information criterion for rank selection.
We solve a challenging factor analysis problem using ML principle and scalable algorithms.
problem Estimating the maximum likelihood in low-rank factor analysis.
method Reformulated as a nonlinear nonsmooth semidefinite optimization problem, solved with DC optimization.
result Our approach is scalable, guarantees computational efficiency, and adapts to various constraints.
A new approach to learn weights for multi-view clustering.
problem Measuring the importance of each view in multi-view clustering.
method Proposes a re-weighted approach to learn intrinsic weights for multi-view clustering.
result The proposed approach is effective and practical for multi-view clustering.
New method learns shared structures in non-linear tasks.
problem Learning shared linear representations in non-linear tasks.
method Convex optimization with structural assumptions.
result Rank and clustered estimators recover shared structures under certain conditions.
New method GSAT improves robustness against structured perturbations.
problem Structured perturbations in biological data.
method Formulates GSAT as a non-convex concave minimax optimization problem and solves it with GDADMM.
result Improves robustness against group-sparse and rank-constrained perturbations.
Paper develops methods for non-quadratic loss low-rank matrix recovery.
problem Recovery of low-rank matrices with non-quadratic losses.
method Projected gradient method with a regularity projection oracle.
result Projected gradient method converges globally and linearly.
New method improves image and signal processing with nonconvex rank surrogates and dual momentum.
problem Optimizing nonconvex rank minimization problems in image processing.
method Proposes a novel nonconvex rank surrogate, uses ADMM with dual momentum trick.
result Effective in image and signal processing applications, outperforming state-of-the-art methods.
Paper introduces a new approach for matrix completion with improved convergence guarantees.
problem Matrix completion problems, especially in the presence of probabilistic dependency.
method Leave-one-out analysis combined with PGD and NNM for convergence analysis.
result Established convergence guarantees for PGD and optimal sample complexity for NNM.
The paper decomposes unsupervised learning's generalization error into model, data, and variance components.
problem Understanding the components of unsupervised learning's generalization error.
method Information-geometric decomposition of the Kullback-Leibler generalization error.
result The optimal rank in ε-PCA is the noise floor, balancing model-error gain and data-bias cost. We consider the classic problem of establishing a statistical ranking of a set of n items given a set of inconsistent and incomplete pairwise comparisons between such items. Instantiations of this problem occur in numerous applications in data analysis (e.g., ranking teams in sports data), computer vision, and machine …
CALDERA compresses large language models by approximating weight matrices with low-rank, low-precision factors.
problem The large sizes of Large Language Models (LLMs) hinder deployment on edge devices.
method CALDERA approximates weight matrices W as Q+LR, where L and R are low-rank factors quantized to low precision. result CALDERA achieves better zero-shot performance than existing techniques, especially with low bit precision.