Bayesian HMF integrates multiple datasets for in/out-of-matrix prediction.
problem Data integration across different entity types and sparsity levels.
method Bayesian hybrid matrix factorisation model combining multiple methods.
result Consistently better in-matrix and out-of-matrix predictions compared to state-of-the-art methods.
Improved prediction accuracy in matrix factorization using graph-based priors.
problem Graph side-information may not align with latent-feature relations in matrix completion.
method Identify and remove 'contested' edges using graphical lasso approximation, maintaining linear scalability.
result Improved prediction accuracy with fewer graph edges, demonstrating the often inaccurate nature of graph side-information.
We provide a proof of backpropagation algorithm in matrix notation.
problem The lack of a full induction proof of backpropagation algorithm in matrix notation.
method We provide a full induction proof of the BP algorithm in matrix notation, situating it in the framework of matrix differential calculus.
result We prove the validity of the backpropagation algorithm in inductive form.
GD with large init shows incremental learning in matrix factorization.
problem Understanding GD's behavior with large initial values in matrix factorization.
method Signal-to-noise ratio concepts and inductive arguments.
result Uncovering an incremental learning phenomenon in GD with large initialization.
Mirror descent algorithm recovers low-rank matrices in matrix sensing.
problem Matrix sensing with low-rank matrices under certain conditions.
method Discrete-time mirror descent applied to empirical risk with Bregman divergence analysis.
result Mirror descent converges to a matrix minimizing a specific nuclear norm-related quantity.
Enhances NMF for better time series recovery and prediction using side information.
problem Reconstruct and predict electricity consumption time series.
method Extends NMF with side information, proposes HALSX algorithm.
result Improved recovery and prediction performance validated on various datasets.
We describe novel subgradient methods for a broad class of matrix optimization problems involving nuclear norm regularization. Unlike existing approaches, our method executes very cheap iterations by combining low-rank stochastic subgradients with efficient incremental SVD updates, made possible by highly optimized and…
Non-convex gradient descent accelerates convergence in matrix factorization models.
problem Non-convex optimization in matrix factorization models.
method Factored gradient descent with acceleration.
result Acceleration leads to linear convergence rate in non-convex settings.
Deep model tackles matrix completion issues.
problem Matrix completion problems in signal processing and machine learning.
method Deep matrix factorization model with a generic discretization operator.
result Efficacy demonstrated on a real movie rating dataset.
New method predicts and optimizes matrix recovery from noisy measurements.
problem Recovering rank-1 matrices from Gaussian measurements with noise.
method Stochastic prox-linear iterative algorithm with trajectory predictions.
result The method converges linearly with accurate predictions of error.
New principle in online learning: Regret can be expressed using sufficient statistics and a Burkholder function.
problem Achieving optimal online learning performance with limited memory.
method Introducing a Burkholder function that depends only on sufficient statistics, not the entire data sequence.
result Developed novel online strategies for matrix prediction and parameter-free supervised learning.
Paper proposes a framework for disease prediction from EHRs with missing data.
problem Missing data in EHRs for disease prediction.
method Two-stage framework including missing data imputation and disease prediction using GANs and stacked autoencoders.
result Significantly improved disease prediction accuracy with AC-GAN and stacked autoencoder.
Paper bounds subspace estimator error from noisy projections.
problem Estimating subspaces from noisy data.
method Derives perturbation bound on optimal subspace estimator.
result Fundamental result with implications in matrix completion and clustering.
LNMC improves link prediction on social networks by considering log-normal degree distributions.
problem Link prediction in social networks with log-normal degree distributions.
method Log-Normal Matrix Completion (LNMC) using Alternating Direction Method of Multipliers.
result Up to 5% AUC increase over non-structured sparsity based methods.
Minimal submanifolds in matrix spaces proven for specific ranks.
problem Minimal submanifolds in matrix spaces.
method Proving semialgebraic sets of matrices are minimal.
result Rectangular, skew-symmetric, and symmetric matrices with prescribed eigenvalues are minimal.
Paper proposes a deep learning method for better covariance matrix forecasting.
problem Suboptimal predictive performance in traditional matrix volatility forecasting.
method Riemannian-geometry-aware deep learning framework for symmetric positive definite matrices.
result Our method outperforms traditional approaches in predictive accuracy.
Minimal cones defined by rank conditions in matrix spaces.
problem Characterizing minimal cones in matrix spaces.
method Proving determinantal varieties are minimal cones with regular strata.
result Determinantal varieties are minimal cones with regular submanifolds.
The paper analyzes how to improve matrix completion with prior information.
problem Improving matrix completion accuracy and extrapolation of missing entries.
method Using Reproducing Kernel Hilbert spaces to incorporate prior information.
result Analyzes the generalization error of matrix completion methods.
Gradient descent on matrix factorization leads to minimum nuclear norm solution.
problem Optimizing underdetermined quadratic objectives over matrices.
method Gradient descent on a full dimensional factorization of the matrix.
result Gradient descent converges to the minimum nuclear norm solution.
In this paper, we study the Poisson equation and heat equation in a model matrix geometry Mn. Our main results are about the Poisson equation and global behavior of the heat equation on Mn. We can show that if c0 is the initial positive definite matrix in Mn, then c(t) exists for all time and is positive …
The subdifferential of convex functions of the singular spectrum of real matrices has been widely studied in matrix analysis, optimization and automatic control theory. Convex analysis and optimization over spaces of tensors is now gaining much interest due to its potential applications to signal processing, statistics…
Transformers exhibit abrupt learning in matrix completion tasks.
problem Understanding abrupt learning in Transformers for matrix completion.
method Formulated matrix completion as MLM task, trained BERT model, analyzed model components.
result Sudden drop in loss despite no changes in training procedure or hyper-parameters.
New algorithm improves game learning with randomised optimism.
problem Learning in matrix games with unknown payoffs and bandit feedback.
method Integrates evolutionary algorithms into bandit framework for randomised optimism.
result Achieves sublinear regret, outperforming classical methods.
New insights into how deep models generalize, focusing on matrix factorization.
problem Understanding how deep models generalize and why they work well.
method Using Morse functions and dynamical systems to study implicit regularization.
result Solved a conjecture on implicit regularization in matrix factorization.
Paper proposes an efficient algorithm for nonnegative binary matrix factorization.
problem Decomposing binary data using matrix factorization.
method Majorization-minimization algorithm with Beta prior for improved performance.
result Proposed algorithm offers excellent trade-off between performance, complexity, and interpretability.
New coordinates show Toda flow is Morse-Smale.
problem Toda flow on full flags of Lie algebras.
method Introduced atlas and local coordinates.
result Toda flow on full flags is Morse-Smale.
New method clusters matrix-variate data with outliers.
problem Clustering matrix-variate data with outliers.
method Iterative approach using subset log-likelihoods.
result Extends OCLUST algorithm to matrix-variate normal data.
Long term optimal investment problems are studied in a factor model with matrix valued state variables. Explicit parameter restrictions are obtained under which, for an isoelastic investor, the finite horizon value function and optimal strategy converge to their long-run counterparts as the investment horizon approache…
New norms derived from box-norm improve multitask learning performance.
problem Improving multitask learning performance in matrix completion and prediction.
method Derived new norms (box-norm, spectral k-support, spectral box-norm) and improved algorithms to compute them.
result New norms provide state-of-the-art performance in matrix completion and multitask learning.
Weight normalization speeds up matrix sensing problems.
problem Matrix sensing with overparameterization.
method Generalized weight normalization with Riemannian optimization.
result WN achieves linear convergence, improving speed and complexity.
The paper studies vanishing cycles in matrix singularities.
problem Understanding the topology of singular Milnor fibers of matrix families.
method Definition and analysis of vanishing cycles, proof of conjectures, study of monodromy.
result Proof of an extended Damon-Pike μ=τ conjecture for special matrix families.
This paper considers the problem of matrix completion when some number of the columns are completely and arbitrarily corrupted, potentially by a malicious adversary. It is well-known that standard algorithms for matrix completion can return arbitrarily poor results, if even a single column is corrupted. One direct appl…
We obtain a family of matrix integrals which decompose to a product of Gamma-functions (they have some relations with S.G.Gindikin 'Beta', but generally speaking essentially differ from it). We obtain Plancherel formula for Berezin representations for all series of classical groups (for large values of parameters of re…
We recall the main facts about the odd Laplacian acting on half-densities on an odd symplectic manifold and discuss a homological interpretation for it suggested recently by P. {Š}evera. We study the relationship of odd symplectic geometry with classical objects. We show that the Berezinian of a canonical transformatio…
Gradient descent with large steps leads to chaotic parameter space and unpredictable outcomes.
problem Understanding the behavior of gradient descent with large step sizes in matrix factorization.
method Analyzing the fractal structure of the parameter space and deriving critical step sizes for convergence.
result Gradient descent with large steps exhibits chaotic behavior and sensitivity to initialization, creating a fractal boundary between converging and diverging minimizers.
The paper analyzes how over-parameterization affects GD convergence in matrix sensing problems.
problem Matrix sensing problem with over-parameterized gradient descent.
method Analyzes symmetric and asymmetric parameterizations, provides lower bounds and convergence rates.
result Over-parameterization slows down GD convergence, but asymmetric parameterization can speed up convergence.
This paper considers the recovery of a low-rank matrix from an observed version that simultaneously contains both (a) erasures: most entries are not observed, and (b) errors: values at a constant fraction of (unknown) locations are arbitrarily corrupted. We provide a new unified performance guarantee on when the natura…
Paper calculates the exact error of LDA models.
problem Bayesian generalization error in Latent Dirichlet Allocation (LDA).
method Theoretical analysis of learning coefficient using algebraic geometry.
result Exact asymptotic form of LDA's generalization error.
Recovery of low-rank matrices has recently seen significant activity in many areas of science and engineering, motivated by recent theoretical results for exact reconstruction guarantees and interesting practical applications. A number of methods have been developed for this recovery problem. However, a principled meth…
Bayesian parametric matrix models provide uncertainty quantification for spectral learning.
problem Uncertainty quantification in spectral learning for safety-critical applications.
method Bayesian parametric matrix models (B-PMMs) that extend PMMs to provide uncertainty estimates.
result B-PMMs achieve exceptional uncertainty calibration (ECE < 0.05) while maintaining favorable scaling.
New algorithms improve on bandit feedback in matrix games with unknown payoff matrices.
problem Improving performance in matrix games with unknown payoff matrices and bandit feedback.
method Regret analyses of variants of UCB and K-learning.
result New algorithms achieve lower regret compared to adversarial bandit algorithms.
This paper concerns the problem of matrix completion, which is to estimate a matrix from observations in a small subset of indices. We propose a calibrated spectrum elastic net method with a sum of the nuclear and Frobenius penalties and develop an iterative algorithm to solve the convex minimization problem. The itera…
New method uses spectral geometry to improve matrix completion with geometric relations.
problem Matrix completion problems with underlying geometric or topological relations.
method Interprets DMF through spectral geometry to incorporate explicit regularization.
result DMF models can exploit geometric relations, improving performance on real benchmarks.
Proposes a neural network for recognizing 3D skeleton-based interactions.
problem Recognizing two-person interactions from 3D skeleton sequences.
method Uses Gaussian distributions and Riemannian geometry of SPD matrices and matrix groups.
result Achieves competitive results on three benchmarks for 3D human activity understanding.
Gradient descent in deep matrix factorization favors low-rank solutions, improving recovery accuracy.
problem Understanding the generalization in deep learning models.
method Study of gradient descent over deep linear neural networks for matrix completion and sensing.
result Adding depth enhances an implicit tendency towards low-rank solutions, leading to more accurate recovery.
Extended wMEM approach for MEG inverse problem using wavelet and spatial filters.
problem Infer brain activity from full space-time data in MEG.
method Wavelet decomposition, spatial filters, Kronecker product modeling, numerical optimization.
result Smooth numerical optimization problem solved with reasonable dimensionality.
Unweighted matrix factorization can match or outperform weighted methods in recommender systems.
problem Improving recommendation performance with matrix factorization on implicit feedback data.
method Systematic study of various weighting schemes and matrix factorization algorithms.
result Training with unweighted data can perform comparably to, and sometimes outperform, training with weighted data.
We extend the theory of matrix completion to the case where we make Poisson observations for a subset of entries of a low-rank matrix. We consider the (now) usual matrix recovery formulation through maximum likelihood with proper constraints on the matrix M, and establish theoretical upper and lower bounds on the rec…