Paper restricts non-negative matrix factorization to stochastic matrices for data analysis.
problem Analyzing unstructured data like topic models and face storage retrieval.
method Necessary and sufficient conditions for unique factorization, natural bounds on parameters, consistent estimator.
result Unique factorization conditions and parameter bounds for observed data.
Improved NMF using variance-reduced MU rule.
problem Slow convergence of multiplicative update in NMF.
method Introduces variance-reduced stochastic multiplicative update.
result Robustly outperforms state-of-the-art algorithms.
Variational inference improves neural network matrix factorization for stochastic blockmodels.
problem Improving predictive performance of neural network matrix factorization for stochastic blockmodels.
method Construct Bayesian neural networks and fit with variational inference.
result Variational inference can achieve equivalent performance to neural networks on Movielens data.
Paper shows LDA and SMF have similar generalization errors.
problem LDA and SMF's generalization performance is unknown.
method Algebraic and geometric method to show equivalence of LDA and SMF.
result LDA and SMF have asymptotically same Bayesian generalization error.
Paper improves efficiency in matrix computations for Gaussian processes.
problem Efficiency in matrix computations for Gaussian processes.
method Variance reduction via matrix factorization.
result Factorized estimator can be up to 1,000 times more efficient.
An ADRC-incorporated SGD algorithm improves latent factor analysis speed and accuracy.
problem Slow convergence in standard SGD for HDI matrix analysis.
method Incorporates ADRC principles to refine historical and future learning error states.
result Empirically outperforms state-of-the-art LFA models in HDI matrix prediction.
We present a method based on the orthogonal symmetric non-negative matrix tri-factorization of the normalized Laplacian matrix for community detection in complex networks. While the exact factorization of a given order may not exist and is NP hard to compute, we obtain an approximate factorization by solving an optimiz…
New method combines simulated annealing and Levy distribution for fast matrix factorization.
problem High complexity and difficulty in parallelizing matrix factorization for large matrices.
method Combining simulated annealing with Levy distribution for matrix factorization.
result Achieves good solutions in acceptable time with low computations.
Develops polynomial diffusion models for multi-factor commodity futures dynamics.
problem Modeling futures prices using latent state variables for short and long-term stochastic factors.
method Polynomial diffusion models to incorporate non-linear effects, two filtering methods for estimation.
result Accurate estimation of futures prices despite parameter identification issues in polynomial diffusion models.
In this paper, we propose an online algorithm to compute matrix factorizations. Proposed algorithm updates the dictionary matrix and associated coefficients using a single observation at each time. The algorithm performs low-rank updates to dictionary matrix. We derive the algorithm by defining a simple objective funct…
Unified framework speeds up SMF algorithms via variance reduction.
problem Improving convergence speed and accuracy in stochastic matrix factorization.
method Unified framework using variance reduction for SMF.
result Consistently faster convergence and more accurate output.
Despite having various attractive qualities such as high prediction accuracy and the ability to quantify uncertainty and avoid over-fitting, Bayesian Matrix Factorization has not been widely adopted because of the prohibitive cost of inference. In this paper, we propose a scalable distributed Bayesian matrix factorizat…
We improve private training accuracy with learning rate schedules and matrix factorizations.
problem Private training with learning rate schedules and correlated noise.
method General upper and lower bounds for learning rate schedules, memory-efficient constructions, and schedule-aware factorizations.
result Schedule-aware factorizations improve accuracy in private training.
The paper analyzes DeepWalk and node2vec for community detection in stochastic blockmodels.
problem Theoretical understanding of community detection in stochastic blockmodels.
method Matrix factorization perspective on DeepWalk and node2vec.
result Node2vec and DeepWalk can perfectly recover community memberships in sparse graphs.
Algorithm for fast matrix factorization of large datasets.
problem Factorizing huge matrices with sparse or dense factors.
method Subsampling and iterative learning of matrix factors.
result Significant speed-ups on large datasets.
Optimizes Bayesian priors for matrix factorization without posterior inference.
problem Selecting optimal priors for Bayesian models in machine learning.
method Prior predictive distribution and virtual statistics matching user-provided or observed data statistics.
result Analytically determines hyperparameters for Poisson factorization models.
sgdGMF efficiently estimates generalized matrix factorization models for single-cell RNA sequencing data.
problem Challenges in dimensionality reduction for large single-cell RNA sequencing datasets.
method Scalable adaptive stochastic gradient descent algorithm for generalized matrix factorization models.
result sgdGMF outperforms existing methods in scalability and accuracy for large datasets.
We find multi-factor CIR models can exhibit unspanned stochastic volatility.
problem Unspanned stochastic volatility in fixed income markets.
method Formal review and necessary/sufficient conditions for multi-factor CIR models.
result We construct three-factor CIR models that exhibit unspanned stochastic volatility.
Beta process is the standard nonparametric Bayesian prior for latent factor model. In this paper, we derive a structured mean-field variational inference algorithm for a beta process non-negative matrix factorization (NMF) model with Poisson likelihood. Unlike the linear Gaussian model, which is well-studied in the non…
NeuralIF uses neural networks to improve preconditioning for faster CG convergence.
problem Improving convergence of conjugate gradient method for large-scale sparse systems.
method Data-driven approach using graph neural networks to generate incomplete factorization.
result Data-driven preconditioners accelerate convergence of conjugate gradient method.
Proposes a new matrix factorization model for interval-valued matrices.
problem Matrix factorization for matrices with entries in a given interval.
method Bounded simplex-structured matrix factorization (BSSMF) with fast algorithm for missing data.
result BSSMF provides a unique decomposition under certain conditions.
Stochastic gradient descent (SGD) on a low-rank factorization is commonly employed to speed up matrix problems including matrix completion, subspace tracking, and SDP relaxation. In this paper, we exhibit a step size scheme for SGD on a low-rank least-squares problem, and we prove that, under broad sampling conditions,…
A new optimization method reduces memory and compute requirements for deep learning.
problem Memory and compute constraints in second-order stochastic optimizers for deep learning.
method Proposes KrAD, a novel factorization to approximate inverse Fisher matrix without inversion, leading to KrADagrad.
result Improves performance over Shampoo for 32-bit precision and comparable/generalization on real datasets.
FOCUS method forecasts counterfactuals in panel data with time series dynamics.
problem Forecasting unobserved potential outcomes in causal inference with missing entries and latent factors.
method FOCUS extends matrix completion methods by leveraging time series dynamics of latent factors.
result FOCUS method outperforms existing benchmarks in predicting future counterfactuals.
Sparse coding--that is, modelling data vectors as sparse linear combinations of basis elements--is widely used in machine learning, neuroscience, signal processing, and statistics. This paper focuses on the large-scale matrix factorization problem that consists of learning the basis set, adapting it to specific data. V…
New methods for community detection in multi-layer networks improve upon existing techniques.
problem Estimating a consensus community structure in multi-layer networks.
method Spectral clustering and matrix factorization methods for low-rank matrix optimization.
result Consistency properties of intermediate fusion techniques under multi-layer stochastic blockmodel.
Model-based collaborative filtering analyzes user-item interactions to infer latent factors that represent user preferences and item characteristics in order to predict future interactions. Most collaborative filtering algorithms assume that these latent factors are static, although it has been shown that user preferen…
This article provides the mathematical foundation for stochastically continuous affine processes on the cone of positive semidefinite symmetric matrices. This analysis has been motivated by a large and growing use of matrix-valued affine processes in finance, including multi-asset option pricing with stochastic volatil…
OptComplete efficiently completes matrices with side information, providing insights.
problem Matrix completion with interpretability for side information.
method Binary convex optimization reformulation and stochastic cutting planes.
result OptComplete outperforms state-of-the-art methods in scalability and accuracy.
A new method for learning Bayesian neural networks using layerwise inference.
problem Learning Bayesian neural networks efficiently and accurately.
method Bayesian layerwise inference, treating neural networks as stacked Bayesian linear models, with pseudo-targets defined by backpropagated gradients.
result The method converges quickly and performs well on various benchmarks.
Paper shows robustness of gradient descent in matrix sensing despite perturbations.
problem Understanding robustness of gradient descent in matrix sensing.
method Developed perturbed gradient flow to capture noise and improve robustness.
result Gradient descent is robust to perturbations in matrix sensing.
Method selects number of communities in weighted networks.
problem Selecting the number of communities in weighted networks.
method Proposes a novel weighted DCSBM and uses a sequential testing framework with spectral clustering and matrix scaling.
result Method is consistent in estimating the true number of communities under mild conditions.
We show that there are no spurious local minima in the non-convex factorized parametrization of low-rank matrix recovery from incoherent linear measurements. With noisy measurements we show all local minima are very close to a global optimum. Together with a curvature bound at saddle points, this yields a polynomial ti…
A new matrix factorization model learns and weights data deviations for better model performance.
problem Stochastic noise causes unreliable data points, leading to suboptimal model fitting.
method Deviation-driven matrix factorization model that learns and weights data deviations.
result Our model outperforms state-of-the-art models in accuracy and efficiency.
We propose an efficient method for approximating natural gradient descent in neural networks which we call Kronecker-Factored Approximate Curvature (K-FAC). K-FAC is based on an efficiently invertible approximation of a neural network's Fisher information matrix which is neither diagonal nor low-rank, and in some cases…
New methods improve analysis of single cell RNA sequencing data.
problem High dimensionality and complexity of scRNA-seq data.
method Topological Nonnegative Matrix Factorization (TNMF) and Robust Topological NMF (rTNMF).
result TNMF and rTNMF significantly outperform other NMF-based methods.
The article develops a method to learn sparse and low rank PARAFAC decomposition robust to noise.
problem Learning sparse and low rank PARAFAC decomposition for tensors with missing values.
method Bayesian model with elastic net regularization, efficient algorithms for large scale problems.
result The method finds true rank and sparse factor matrix robust to noise.
Framework captures missing data in sparse data sets.
problem Capturing missing data in extremely sparse data sets.
method Coupled compound Poisson factorization with stochastic variational inference.
result Explicitly modeling missing data improves results in clustering, prediction, and matrix factorization.
We describe novel subgradient methods for a broad class of matrix optimization problems involving nuclear norm regularization. Unlike existing approaches, our method executes very cheap iterations by combining low-rank stochastic subgradients with efficient incremental SVD updates, made possible by highly optimized and…
Simplified proof shows SGD optimality for least squares.
problem Optimizing SGD for least squares efficiency.
method Analyzing SGD as a stochastic process, characterizing stationary covariance matrix.
result Statistical minimax optimality of SGD for least squares.
Noise in linear networks minimizes sharpness and leads to shrinkage-thresholding.
problem Minimizing sharpness in diagonal linear networks.
method Stochastic sharpness-aware minimization (SAM) with isotropic noise.
result Noise forces shrinkage-thresholding of true parameters.
We develop a unified and systematic framework for performing online nonnegative matrix factorization under a wide variety of important divergences. The online nature of our algorithm makes it particularly amenable to large-scale data. We prove that the sequence of learned dictionaries converges almost surely to the set…
Newsroom in online ecosystem is difficult to untangle. With prevalence of social media, interactions between journalists and individuals become visible, but lack of understanding to inner processing of information feedback loop in public sphere leave most journalists baffled. Can we provide an organized view to charact…
Paper tackles joint community detection and phase synchronization in stochastic block models.
problem Jointly recover cluster structure and phase angles in stochastic block models.
method Proposes two algorithms: a spectral method based on multi-frequency QR factorization and an iterative multi-frequency generalized power method.
result Proposed algorithms significantly improve recovery of cluster structure and phase angles compared to existing methods.
We introduce a multivariate stochastic volatility model for asset returns that imposes no restrictions to the structure of the volatility matrix and treats all its elements as functions of latent stochastic processes. When the number of assets is prohibitively large, we propose a factor multivariate stochastic volatili…
NIMFA is a Python library for nonnegative matrix factorization.
problem Efficiently factorizing nonnegative matrices for various applications.
method Unified interface, state-of-the-art methods, initialization approaches, quality scoring, supports dense and sparse matrices.
result Unified and efficient implementation of nonnegative matrix factorization methods.
Paper predicts travel costs across regions using neural networks.
problem Predicting travel costs in sparse, stochastic OD matrices.
method Recurrent Multi-Graph Neural Networks (R-MGNN) for sparse, stochastic OD matrix forecasting.
result Framework effectively predicts future OD matrices without empty elements.
A new algorithm reduces online eigenvector computation time while maintaining optimal performance.
problem Online learning of top eigenvectors in both adversarial and stochastic settings.
method Follow the Compressed Leader (FTCL) framework, compressing the matrix strategy to dimensions 3 (adversarial) and 1 (stochastic).
result Achieves optimal regret without sacrificing running time, resolving open questions.