We consider the problem of estimating a low-rank matrix from a noisy observed matrix. Previous work has shown that the optimal method depends crucially on the choice of loss function. In this paper, we use a family of weighted loss functions, which arise naturally for problems such as submatrix denoising, denoising wit…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “matrix loss”
On the Loss Landscape Geometry of Regularized Deep Matrix Factorization: Uniqueness and Sharpnessstat.ML
The paper studies the loss landscape of regularized deep matrix factorization, revealing unique and sharp minimizers.
problem Understanding the loss landscape and minimizers of regularized deep matrix factorization problems.
method Theoretical analysis of -regularized deep matrix factorization/deep linear network training problems with squared-error loss.
result The unique end-to-end minimizer exists for all target matrices except for a set of Lebesgue measure zero.
Paper proposes a novel method to improve matrix completion with median loss for large datasets.
problem Matrix completion with absolute deviation loss for large-scale data.
method Proposes a refinement step using pseudo data to improve inefficient estimators of median matrix completion.
result Turns inefficient estimators into a rate (near-)optimal matrix completion procedure.
New method proves asymptotic normality for matrix sensing problems.
problem Proving asymptotic normality for matrix sensing under general convex losses.
method Riemannian geometry to handle degeneracy of the Hessian due to rotational symmetry.
result Proves as .
Random Matrix Theory explains loss surface Hessians in neural networks.
problem Understanding the loss surfaces of neural networks.
method Investigation of local spectral statistics of neural network Hessians.
result Excellent agreement with Gaussian Orthogonal Ensemble statistics.
We study consistency properties of surrogate loss functions for general multiclass learning problems, defined by a general multiclass loss matrix. We extend the notion of classification calibration, which has been studied for binary and multiclass 0-1 classification problems (and for certain other specific learning pro…
Study on deep matrix factorization with Bures-Wasserstein loss, focusing on critical points and convergence.
problem Analyzing critical points and convergence of generative deep linear networks trained with Bures-Wasserstein loss.
method Characterization of critical points and minimizers of Bures-Wasserstein distance, analysis of Hessian at low-rank matrices, convergence results for gradient flow and descent.
result Established convergence results for gradient flow and finite step size gradient descent under certain assumptions.
Unified framework for non-negative matrices and tensors using Wasserstein loss.
problem Finding low-dimensional representations of high-dimensional datasets with non-negative constraints.
method Unified mathematical framework with a smoothed Wasserstein loss, convex dual formulation for efficient computation.
result Efficient solution for non-negative matrix and tensor factorisations with Wasserstein loss.
Paper develops methods for non-quadratic loss low-rank matrix recovery.
problem Recovery of low-rank matrices with non-quadratic losses.
method Projected gradient method with a regularity projection oracle.
result Projected gradient method converges globally and linearly.
Unified approach for robust low rank matrix estimation with adversaries.
problem Robust low rank matrix estimation in the presence of adversaries.
method Unified approach combining Huber loss and nuclear norm penalization.
result Sharp estimation error bounds for matrix compressed sensing and completion.
A new framework for deep matrix factorizations improves model consistency and flexibility.
problem Inconsistent loss functions in deep matrix factorizations.
method Introduces two new loss functions and a generic optimization framework.
result Demonstrates improved model performance on synthetic and real data.
This paper improves compression of large NLP models using doped Kronecker Products.
problem Accuracy loss when compressing large NLP tasks with Kronecker Products.
method Doping Kronecker Products with an overlay matrix to recover accuracy, and a new regularization scheme called co matrix dropout regularization (CMR).
result Compression of a large language model with LSTM layers of size 25 MB by 25x with 1.4% loss in perplexity score.
Paper finds exact Hessian sharpness in deep matrix factorization.
problem Understanding the geometry of loss landscapes in deep matrix factorization.
method Presented the first exact expression for Hessian maximum eigenvalue.
result Spectral-norm balance is a sufficient condition for flatness in deep matrix factorization.
Algorithm learns a better sketch matrix for low-rank approximations.
problem Efficiently compute low-rank approximations of large matrices.
method Uses a learned sketch matrix instead of random matrix for optimization.
result Learned sketch matrix reduces approximation loss significantly compared to random matrix.
A Flexible Optimization Framework for Regularized Matrix-Tensor Factorizations with Linear Couplingscs.LG
Flexible framework for CMTF with ADMM for various constraints and couplings.
problem Challenges in data fusion from multiple sources with varying characteristics.
method Flexible algorithmic framework using AO and ADMM for various constraints, loss functions, and couplings.
result Accurate and computationally efficient results for various loss functions, including KL divergence.
The flare Package for High Dimensional Linear Regression and Precision Matrix Estimation in Rstat.ML
A new R package for high-dimensional regression and precision matrix estimation.
problem High-dimensional linear regression and precision matrix estimation challenges.
method flare package implements various regression methods and extensions for sparse precision matrix estimation.
result The flare package is efficient and scalable for large problems.
Flat minima lead to better generalization in low-rank matrix recovery models.
problem Understanding why flat minima generalize well in overparameterized models.
method Analysis of overparameterized matrix and bilinear sensing, robust PCA, covariance matrix estimation, and neural networks with quadratic activation functions.
result Flat minima, measured by the trace of the Hessian, exactly recover the ground truth in low-rank matrix recovery models under standard statistical assumptions.
In this paper, we propose two new algorithms for transduction with Matrix Completion (MC) problem. The joint MC and prediction tasks are addressed simultaneously to enhance the accuracy, i.e., the label matrix is concatenated to the data matrix forming a stacked matrix. Assuming the data matrix is of low rank, we propo…
New robust loss functions improve matrix completion accuracy.
problem Outliers in data corrupting matrix completion accuracy.
method Developed nonconvex M-estimator functions to down-weight outliers.
result Proposed methods outperform competitors in recovery accuracy and runtime.
Laplacian Constrained Precision Matrix Estimation: Existence and High Dimensional Consistencystat.ML
Paper proves conditions for estimating precision matrices with Laplacian constraints.
problem Estimating high-dimensional precision matrices with Laplacian constraints.
method Minimizing Stein's loss with conditions on graph connectivity and Laplacian constraints.
result High-dimensional consistency achieved with Laplacian constraints, independent of graph structure.
SCOPE: Spectral Concentration by Distributionally Robust Joint Covariance-Precision Estimationstat.ML
SCOPE estimator improves covariance and precision matrix estimation.
problem Estimating covariance and precision matrices accurately.
method Distributionally robust optimization with convex spectral divergence.
result SCOPE estimator reduces spectral bias and improves condition number.
Paper tackles robust matrix completion with heavy-tailed noise.
problem Estimating a low-rank matrix from noisy incomplete data.
method Adaptive Huber loss for robustness, nonconvex algorithm with spectral initialization.
result Achieves minimax-optimal statistical estimation error under bounded second moment condition.
This paper studies noisy low-rank matrix completion: given partial and noisy entries of a large low-rank matrix, the goal is to estimate the underlying matrix faithfully and efficiently. Arguably one of the most popular paradigms to tackle this problem is convex relaxation, which achieves remarkable efficacy in practic…
Study generalizes matrix completion with side info in low noise settings.
problem Matrix completion with side information in low noise conditions.
method Inductive matrix completion with i.i.d. subgaussian noise, uniform sampling, and side information.
result Generalization bounds with noise scaling, convergence to zero, and logarithmic dependence on matrix size.
In this paper, we introduce a novel and robust approach to Quantized Matrix Completion (QMC). First, we propose a rank minimization problem with constraints induced by quantization bounds. Next, we form an unconstrained optimization problem by regularizing the rank function with Huber loss. Huber loss is leveraged to c…
The paper analyzes error bounds and KL properties for noisy matrix recovery problems.
problem Noisy low-rank matrix recovery problems.
method Squared F-norm regularization, accelerated alternating minimization method.
result Established error bounds and KL properties for critical points and global minimizers.
We estimate generic statistical properties of a structural credit risk model by considering an ensemble of correlation matrices. This ensemble is set up by Random Matrix Theory. We demonstrate analytically that the presence of correlations severely limits the effect of diversification in a credit portfolio if the corre…
Extends matrix factorization for deviance-based losses with GLM theory.
problem Improving data loss models beyond squared error.
method Adapts GLM theory to matrix factorization for deviance losses.
result Strong consistency and robustness of the proposed decomposition.
We study the column subset selection problem with respect to the entrywise -norm loss. It is known that in the worst case, to obtain a good rank- approximation to a matrix, one needs an arbitrarily large number of columns to obtain a -approximation to the best entrywise -norm low ra…
Transformers exhibit abrupt learning in matrix completion tasks.
problem Understanding abrupt learning in Transformers for matrix completion.
method Formulated matrix completion as MLM task, trained BERT model, analyzed model components.
result Sudden drop in loss despite no changes in training procedure or hyper-parameters.
We use matricial free energy to regularize autoencoders, producing Gaussian-like codes.
problem Generating Gaussian-like codes for autoencoders.
method Define a differentiable loss function based on singular values of the code matrix, minimizing matricial free energy.
result Minimizing matricial free energy results in Gaussian-like codes that generalize.
Ridge regression CV loss may have multiple local optima.
problem Can we globally optimize cross-validation loss in ridge regression?
method Analyzing quasiconvexity of CV loss in ridge regression.
result CV loss may fail to be quasiconvex and have multiple local optima.
Due to challenging applications such as collaborative filtering, the matrix completion problem has been widely studied in the past few years. Different approaches rely on different structure assumptions on the matrix in hand. Here, we focus on the completion of a (possibly) low-rank matrix with binary entries, the so-c…
The notion of developing statistical methods in machine learning which are robust to adversarial perturbations in the underlying data has been the subject of increasing interest in recent years. A common feature of this work is that the adversarial robustification often corresponds exactly to regularization methods whi…
Sign-RIP improves robust low-rank matrix recovery by preserving norms even with corrupted measurements.
problem Robust low-rank matrix recovery in the presence of corrupted measurements.
method Proposed Sign-RIP, a robust restricted isometry property.
result Sign-RIP guarantees uniform convergence of subdifferentials in robust low-rank matrix recovery.
Improved SVD for shifted matrices without explicit matrix construction.
problem Efficiently estimating SVD of shifted matrices.
method Shifted Randomized SVD algorithm.
result More efficient matrix factorization and low-rank approximation.
We consider the problem of recovering a low-rank matrix from its clipped observations. Clipping is conceivable in many scientific areas that obstructs statistical analyses. On the other hand, matrix completion (MC) methods can recover a low-rank matrix from various information deficits by using the principle of low-ran…
Non-negative matrix factorization (NMF) minimizes the Euclidean distance between the data matrix and its low rank approximation, and it fails when applied to corrupted data because the loss function is sensitive to outliers. In this paper, we propose a Truncated CauchyNMF loss that handle outliers by truncating large e…
HERA: Partial Label Learning by Combining Heterogeneous Loss with Sparse and Low-Rank Regularizationcs.LG
Partial Label Learning (PLL) aims to learn from the data where each training instance is associated with a set of candidate labels, among which only one is correct. Most existing methods deal with such problem by either treating each candidate label equally or identifying the ground-truth label iteratively. In this pap…
Study connects covariance cleaning theory to information theory for heavy-tailed distributions.
problem Optimizing covariance matrices for heavy-tailed distributions using information theory.
method Minimizing Frobenius norm and information loss between true and estimated covariance matrices.
result Asymptotic regime of large matrices minimizes information loss for Student's t distributions.
KL property of exponent of -norm and DC regularized factorizations for low-rank matrix recoverymath.OC
This paper is concerned with the factorization form of the rank regularized loss minimization problem. To cater for the scenario in which only a coarse estimation is available for the rank of the true matrix, an -norm regularized term is added to the factored loss function to reduce the rank adaptively; and…
Computationally Efficient and Statistically Optimal Robust Low-rank Matrix and Tensor Estimationmath.ST
A new algorithm improves both computational efficiency and statistical optimality for robust low-rank matrix and tensor estimation.
problem Challenges in low-rank matrix estimation under heavy-tailed noise, both computationally and statistically.
method Riemannian sub-gradient (RsGrad) algorithm, which is computationally efficient and statistically optimal.
result RsGrad achieves linear convergence and statistical optimality for robust loss functions under Gaussian and heavy-tailed noise.
The speed at which one can minimize an expected loss using stochastic methods depends on two properties: the curvature of the loss and the variance of the gradients. While most previous works focus on one or the other of these properties, we explore how their interaction affects optimization speed. Further, as the ulti…
EAST aligns neural network classifiers with user-defined evaluation metrics.
problem Mismatch between neural network training and evaluation metrics leads to suboptimal performance.
method EAST uses dynamic thresholding, soft-set confusion matrix, and annealing to align neural network predictions with target evaluation metrics.
result EAST improves alignment between training objectives and evaluation metrics, outperforming existing methods.
Preconditioned SGD accelerates convergence for ill-conditioned huge-scale matrix completion.
problem Recovering a low-rank matrix from incomplete data with high condition number.
method Preconditioned Stochastic Gradient Descent (SGD) for huge-scale online optimization.
result Preconditioned SGD converges to ε-accuracy in O(log(1/ε)) iterations, compared to O(κlog(1/ε)) for unpreconditioned SGD.
LDA-GO improves LDA for high-dimensional data via gradient optimization.
problem LDA struggles in high-dimensional settings due to unreliable covariance matrix estimation.
method LDA-GO learns a low-rank precision matrix via gradient optimization, automatically selecting between Gaussian likelihood and cross-entropy loss.
result LDA-GO outperforms other LDA variants in sparse-signal high-dimensional regimes.
Boosts neural network performance by improving weight separability.
problem Improving the separability of weight vectors in neural networks.
method Proposes a new evaluation metric and feed-backward reconstruction loss to encourage weight separability.
result Improves visual recognition performance across various tasks.
Boosting theory extended to handle cost-sensitive and multi-objective losses.
problem Real-world prediction problems with different error costs.
method Developed a comprehensive theory of cost-sensitive and multi-objective boosting.
result Established a dichotomy for binary classification and a more intricate landscape for multiclass settings.