Study on signal-plus-noise decomposition in nonlinear spiked random matrices.
problem Nonlinear spiked random matrix models with rank-one signal and noise.
method Signal-plus-noise decomposition and phase transition analysis.
result Identified precise phase transitions in signal components at critical thresholds.
ADMM algorithm solves nonlinear matrix decompositions efficiently.
problem Nonlinear matrix decompositions for various applications.
method Alternating Direction Method of Multipliers (ADMM) for nonlinear matrix factorization.
result The method efficiently solves diverse nonlinear matrix decompositions.
New algorithms accelerate solving nonlinear matrix decomposition with ReLU.
problem Nonlinear matrix decomposition with ReLU function.
method Two new algorithms: A-NMD and 3B-NMD, with adaptive extrapolation and block parametrization.
result Effective algorithms accelerate solving ReLU-NMD problems.
New matrix approximation method using RBF components for better memory efficiency.
problem Efficiently approximate any real matrix without being symmetric or positive definite.
method Formulate as an optimization problem with gradient descent methods.
result Significantly reduces memory usage for various matrix types.
A novel algorithm converges for solving a specific matrix decomposition problem.
problem Nonlinear matrix decomposition with ReLU function for sparse data.
method Introduced a reparametrization of the Latent-RMD model and developed eBCD for convergence proof.
result eBCD converges and outperforms state-of-the-art methods on various data sets.
IKD uses eigen-decomposition for nonlinear dimensionality reduction.
problem Lack of sophisticated and nonlinear dimensionality reduction methods.
method Inverse Kernel Decomposition (IKD) based on eigen-decomposition of sample covariance matrix.
result IKD achieves comparable performance to optimization-based methods with faster running speeds.
In this paper we consider general rank minimization problems with rank appearing in either objective function or constraint. We first establish that a class of special rank minimization problems has closed-form solutions. Using this result, we then propose penalty decomposition methods for general rank minimization pro…
Paper proves optimal decomposition for matrix fields, reducing convex integration steps.
problem Optimizing decomposition of symmetric matrix fields for convex integration.
method Algebraic geometry and topology applications to prove optimality.
result Optimal decomposition with fewer rank-one terms, improving Hölder regularity.
Improved modeling of chaotic systems using time-delay embeddings and Frenet-Serret frame.
problem Identifying effective coordinate systems for nonlinear dynamical systems.
method Developed a new algorithm to identify more stable and accurate models from less data, leveraging the connection between HAVOK and Frenet-Serret frame.
result The sub- and super-diagonal entries of the linear model correspond to intrinsic curvatures in Frenet-Serret frame.
Many machine learning applications use latent variable models to explain structure in data, whereby visible variables (= coordinates of the given datapoint) are explained as a probabilistic function of some hidden variables. Finding parameters with the maximum likelihood is NP-hard even in very simple settings. In rece…
Proposes neural dynamic mode decomposition for end-to-end modeling of nonlinear dynamics.
problem Understanding and modeling nonlinear dynamical systems.
method Trains neural networks to minimize forecast error based on spectral decomposition in the lifted space.
result Demonstrates effectiveness in eigenvalue estimation and forecast performance.
Article presents QR and LQ decomposition algorithms for various matrix sizes and ranks.
problem Solving least squares problems in machine learning and computer vision.
method Developed novel matrix backpropagation algorithms for QR and LQ decompositions of different matrix sizes and ranks.
result Numerical stability and computational efficiency of the proposed methods.
Biological neurons learn tensor decompositions of higher-order correlations using nonlinear Hebbian plasticity.
problem Learning higher-order correlations in biological neurons.
method Introduce and study generalized nonlinear Hebbian learning rules.
result Neurons can learn tensor eigenvectors of higher-order input correlation tensors.
Modeling dynamic user interests using neural matrix factorization.
problem Challenging extraction of valuable insights from unstructured, high-dimensional, and dynamic online content data.
method Combines matrix factorization with neural networks to model nonlinear user and content factors.
result Accurately identifies nuanced and coherent consumption patterns of Boston Globe readers over five years.
Interpretability has become an important issue in the machine learning field, along with the success of layered neural networks in various practical tasks. Since a trained layered neural network consists of a complex nonlinear relationship between large number of parameters, we failed to understand how they could achie…
DeepTensor uses deep networks to efficiently decompose tensors with improved performance and robustness.
problem Efficiently decomposing tensors with deep learning to capture nonlinear structures.
method Low-rank tensor decomposition using deep generative networks trained to minimize approximation error.
result DeepTensor outperforms classical methods like SVD and PCA in various applications, including image denoising and 3D MRI.
Robust method learns nonlinear structures robustly to noise.
problem Learning nonlinear structures in noisy data.
method Robust Non-Linear Matrix Factorization (RNLMF).
result RNLMF achieves noticeable improvements in denoising and clustering.
In this paper, we study a type of reflected BSDE with a constraint and introduce a new kind of nonlinear expectation via BSDE with a constraint and prove the Doob-Meyer decomposition with respect to the super(sub)martingale introduced by this nonlinear expectation. We then apply the results to the pricing of American o…
New method cleans cross-covariance matrices for better financial forecasting.
problem Asymptotically optimal cross-covariance cleaners fail in real-world, time-varying markets.
method Physics-informed neural network that learns from empirical singular values.
result Trained model outperforms analytical cleaners in out-of-sample cross-covariance prediction.
We orthogonalize the NSS model to condition and diagnose its ill-conditioned parameters.
problem The ill-conditioning of the NSS model's design matrix.
method Exact orthogonal reparametrization via QR decomposition.
result Orthogonalization isolates the conditioning structure and maintains fit uncertainty.
New method models matrix time series using tensor CP-decomposition.
problem Modeling matrix time series with reduced complexity.
method One-pass estimation via generalized eigenanalysis and refined projection.
result Component coefficient vectors estimated consistently with certain rates.
On the basis of loop group decompositions (Birkhoff decompositions), we give a discrete version of the nonlinear d'Alembert formula, a method of separation of variables of difference equations, for discrete constant negative Gauss curvature (pseudospherical) surfaces in Euclidean three space. We also compute two exampl…
Tensor decomposition methods are widely used for model compression and fast inference in convolutional neural networks (CNNs). Although many decompositions are conceivable, only CP decomposition and a few others have been applied in practice, and no extensive comparisons have been made between available methods. Previo…
New method solves robust matrix completion using nonlinear equations.
problem Recover low rank and sparse matrices from incomplete observations.
method Transforms problem into solving a system of nonlinear equations, then uses the alternative direction method.
result Algorithm converges linearly to the true solution under proper assumptions.
Proposes D-CDLF for multi-view data decomposition.
problem Uncorrelatedness between common and distinctive latent factors.
method Decomposes data into common, distinctive, and noise components.
result Effective uncorrelatedness between distinctive latent factors from different views.
Efficient algorithm for Hadamard decomposition of matrices.
problem Decomposing matrices into low-rank factors efficiently.
method Alternating optimization with SVD-inspired initialization and momentum.
result Significantly improved performance compared to existing methods.
This paper proposes a boosting-based solution addressing metric learning problems for high-dimensional data. Distance measures have been used as natural measures of (dis)similarity and served as the foundation of various learning methods. The efficiency of distance-based learning methods heavily depends on the chosen d…
Matrix decomposition is a popular and fundamental approach in machine learning and data mining. It has been successfully applied into various fields. Most matrix decomposition methods focus on decomposing a data matrix from one single source. However, it is common that data are from different sources with heterogeneous…
New deep learning model for matrix completion combining linear and nonlinear relationships.
problem Matrix completion considering only linear or nonlinear relations, ignoring latent relationships.
method Combines linear and nonlinear models in a latent variables framework, using a deep neural network with two branches for columns and rows, and manifold learning as an auxiliary task.
result Experimental results show the proposed method outperforms state-of-the-art matrix completion methods.
New decompositions misattribute differences between populations, even when outcomes are identical.
problem Misattribution of differences between populations using common functional decompositions.
method Extending the Kitagawa-Oaxaca-Blinder decomposition to nonlinear functional decompositions.
result Functional ANOVA and Accumulated Local Effects can misattribute differences even when outcomes are identical in two populations.
Fast matrix algorithms have become the fundamental tools of machine learning in big data era. The generalized matrix regression problem is widely used in the matrix approximation such as CUR decomposition, kernel matrix approximation, and stream singular value decomposition (SVD), etc. In this paper, we propose a fast …
Noise-robust Koopman operator framework for control with improved stability and performance.
problem Developing a stable and noise-robust Koopman operator for control tasks.
method Proposes a learning framework using Hankel matrix and neural network approximations for system dynamics, ensuring long-term stability and noise robustness.
result Demonstrates improved model performance and noise robustness in control tasks compared to existing methods.
Most recent results in matrix completion assume that the matrix under consideration is low-rank or that the columns are in a union of low-rank subspaces. In real-world settings, however, the linear structure underlying these models is distorted by a (typically unknown) nonlinear transformation. This paper addresses the…
A new method models financial returns by separating sign and magnitude, improving forecasting accuracy.
problem Capturing nonlinear predictability in financial return dynamics.
method Decomposes returns into sign and magnitude components, using a joint distribution model.
result Significantly outperforms traditional linear models in forecasting U.S. stock market returns.
Suppose a given observation matrix can be decomposed as the sum of a low-rank matrix and a sparse matrix (outliers), and the goal is to recover these individual components from the observed sum. Such additive decompositions have applications in a variety of numerical problems including system identification, latent var…
Algorithm learns weight matrix from single trajectory of nonlinear dynamical system.
problem Learning weight matrix from a single trajectory of nonlinear dynamical system.
method Algorithm uses global stability and well-conditioned covariance to recover weight matrix.
result Algorithm recovers weight matrix with optimal sample complexity and linear running time.
New algorithms for interpreting complex multivariate functions.
problem Hard interpretation of multivariate functions due to many parameters.
method Filtered tensor decompositions of derivative information.
result Nonparametric estimates of smooth decoupled functions.
Spectral decomposition of the Koopman operator is attracting attention as a tool for the analysis of nonlinear dynamical systems. Dynamic mode decomposition is a popular numerical algorithm for Koopman spectral analysis; however, we often need to prepare nonlinear observables manually according to the underlying dynami…
New method preserves spectral clustering performance under aggressive sparsification and quantization.
problem Maintaining spectral clustering performance with sparse and quantized data.
method Random matrix theory applied to eigenspectrum changes under sparsification and quantization.
result Spectral clustering performance is preserved even with aggressive sparsification and quantization.
We propose a novel Riemannian manifold preconditioning approach for the tensor completion problem with rank constraint. A novel Riemannian metric or inner product is proposed that exploits the least-squares structure of the cost function and takes into account the structured symmetry that exists in Tucker decomposition…
Nonnegative matrix factorization (NMF) is a powerful class of feature extraction techniques that has been successfully applied in many fields, namely in signal and image processing. Current NMF techniques have been limited to a single-objective problem in either its linear or nonlinear kernel-based formulation. In this…
A new algorithm speeds up matrix operations in Neural Networks.
problem Time-consuming matrix operations in Neural Networks.
method An algorithm that increases the degree of parallelism of matrix multiplication.
result The algorithm speeds up several matrix operations in Neural Networks.
Framework for robust matrix estimation with side information.
problem High-dimensional matrix estimation with restrictive structure.
method Flexible framework decomposing matrix into four components: interaction, row, column, and residual.
result Improved imputation accuracy and treatment-effect estimation with side information.
Derives adjoint formulas for matrix operations and applies them to specific cases.
problem Computing adjoints for matrix operations and specific matrix types.
method Derives adjoint formulas for matrix operations and applies them to specific cases.
result Closed-form expressions for adjoints in specific matrix types.
New method identifies key genes affecting phenotypes in biological systems.
problem Identifying genes that drive specific phenotypes in complex biological systems.
method Data-driven observability decomposition using Koopman operators.
result Koopman operator representation identifies genes that drive phenotypes.
Principal component analysis (PCA) is largely adopted for chemical process monitoring and numerous PCA-based systems have been developed to solve various fault detection and diagnosis problems. Since PCA-based methods assume that the monitored process is linear, nonlinear PCA models, such as autoencoder models and kern…
Paper studies nonnegative Tucker decomposition identifiability with sparsity conditions.
problem Identify nonnegative Tucker decomposition factors uniquely.
method Adapting NMF identifiability results, derive procedures using tensor unfoldings or slices.
result Nonnegative Tucker decomposition factors are identifiable under certain sparsity conditions.
The CUR matrix decomposition is an important extension of Nyström approximation to a general matrix. It approximates any data matrix in terms of a small number of its columns and rows. In this paper we propose a novel randomized CUR algorithm with an expected relative-error bound. The proposed algorithm has the advanta…