A new method reduces high-dimensional filtering to quadratic complexity.
problem High-dimensional dynamical systems inference and simulation.
method Low-rank Kalman filtering using dynamical low-rank integrator.
result The method reproduces exact Kalman filter in low-rank limit.
Proposes a model for identifying edges in low-rank dynamical networks.
problem Inability of conventional methods to handle low-rank dynamical networks.
method Low rank dynamical network model with causal Wiener filtering.
result Consistent method for estimating all network edges.
Improved GCNs for non-sparse graphs with low-rank filters.
problem Training and evaluation of GCNs on large non-sparse graphs is computationally expensive.
method Introduced low-rank filters and a reduced-order GCN architecture.
result Significant runtime acceleration and improved accuracy achieved.
This paper considers a new framework to detect communities in a graph from the observation of signals at its nodes. We model the observed signals as noisy outputs of an unknown network process, represented as a graph filter that is excited by a set of unknown low-rank inputs/excitations. Application scenarios of this m…
New algorithm learns low-rank matrices with linear number of samples.
problem Learning low-rank matrices efficiently in latent-variable applications.
method Proposed algorithm that uses linear number of samples in high dimension.
result Learning kimesk, rank-r, matrices requires $Ω( rac{kr}{ε^2})$ samples. Low-rank modeling plays a pivotal role in signal processing and machine learning, with applications ranging from collaborative filtering, video surveillance, medical imaging, to dimensionality reduction and adaptive filtering. Many modern high-dimensional data and interactions thereof can be modeled as lying approximat…
Efficiently learns neural network parameters from streaming data.
problem Online learning of neural networks from non-stationary data streams.
method Low-rank extended Kalman filtering for approximate Bayesian inference.
result Significantly faster learning and adaptation to changing distributions.
A new EnKF method for elliptic PDEs reduces dimensionality for accurate state estimation.
problem Elliptic PDEs in fluid flows make traditional EnKF regularization ineffective.
method Low-rank factorization of the Kalman gain based on the Jacobian spectrum.
result Inference can be performed in a low-dimensional subspace of the state space.
Low-rank modeling generally refers to a class of methods that solve problems by representing variables of interest as low-rank matrices. It has achieved great success in various fields including computer vision, data mining, signal processing and bioinformatics. Recently, much progress has been made in theories, algori…
Bayesian framework for sequential learning tasks with low-rank approximations.
problem Balancing knowledge retention and adaptability in sequential neural networks.
method Bayesian framework with diagonal plus low-rank approximations of the precision matrix.
result Unlocking capabilities to encode task relationships and incorporate prior knowledge from later tasks.
We address the problem of minimizing a convex function over the space of large matrices with low rank. While this optimization problem is hard in general, we propose an efficient greedy algorithm and derive its formal approximation guarantees. Each iteration of the algorithm involves (approximately) finding the left an…
New graph convolution captures local features on non-Euclidean grids.
problem Capturing local features on irregular, coarse non-Euclidean grids.
method Low-rank learnable local filters in graph convolutions.
result Proves more expressive than previous spectral graph convolution methods.
ALF reduces network parameters and operations by 70% and 61%, respectively, on embedded hardware.
problem Efficient deployment of deep learning models on resource-constrained hardware.
method Autoencoder-based low-rank filter-sharing technique.
result ALF achieves significant compression with minimal accuracy loss.
Develops novel techniques for collaborative filtering and multi-label classification.
problem Information overload and categorization of data objects.
method Hierarchical bi-level maximum margin matrix factorization and piecewise-linear embedding method.
result Effective multi-label classification and collaborative filtering techniques developed.
New nonconvex regularizer speeds up low-rank matrix completion.
problem Low-rank matrix completion with good theoretical and empirical performance.
method Proposes a new nonconvex regularizer with adaptive shrinkage, scalable, and fast optimization.
result Proposed method achieves state-of-the-art recovery performance and is the fastest.
New method relaxes spatial invariance in locally connected layers, improving accuracy.
problem Improving classification accuracy with locally connected layers.
method Designing a low-rank locally connected layer with varying spatially varying combining weights.
result Relaxing spatial invariance improves classification accuracy over convolution and locally connected layers.
Recommender systems are widely used to recommend the most appealing items to users. These recommendations can be generated by applying collaborative filtering methods. The low-rank matrix completion method is the state-of-the-art collaborative filtering method. In this work, we show that the skewed distribution of rati…
The annihilating filter-based low-rank Hankel matrix approach (ALOHA) is one of the state-of-the-art compressed sensing approaches that directly interpolates the missing k-space data using low-rank Hankel matrix completion. The success of ALOHA is due to the concise signal representation in the k-space domain thanks to…
Predicting the response of cancer cells to drugs is an important problem in pharmacogenomics. Recent efforts in generation of large scale datasets profiling gene expression and drug sensitivity in cell lines have provided a unique opportunity to study this problem. However, one major challenge is the small number of sa…
New method improves Kalman filtering and smoothing for large state spaces.
problem High computational cost and uncertainty in large-scale Kalman filtering.
method Probabilistic numerical method leveraging GPU acceleration and tunable trade-off.
result Mitigates scaling issues and provides more accurate uncertainty estimates.
New method estimates and completes tensors from ordinal data, improving accuracy and efficiency.
problem Estimating and completing tensors from incomplete, ordinal observations.
method Multi-linear cumulative link model with rank-constrained M-estimator.
result The proposed estimator achieves faster convergence and is minimax optimal.
Low-rank matrix recovery has found many applications in science and engineering such as machine learning, signal processing, collaborative filtering, system identification, and Euclidean embedding. But the low-rank matrix recovery problem is an NP hard problem and thus challenging. A commonly used heuristic approach is…
Filtered conformal ellipsoids for graph-native time series
problem Joint prediction sets for multivariate time series
method Filtered conformal ellipsoids
result Sharper at-target ellipsoids than static-covariance and non-filter baselines
This work formulates a novel song recommender system as a matrix completion problem that benefits from collaborative filtering through Non-negative Matrix Factorization (NMF) and content-based filtering via total variation (TV) on graphs. The graphs encode both playlist proximity information and song similarity, using …
New method for online low-rank matrix completion with improved regret.
problem Designing an efficient algorithm for online recommendation systems with low regret.
method Explore-then-commit (ETC) approach and iterative user clustering (OCTAL) for rank-1 setting.
result Nearly optimal regret bounds for online low-rank matrix completion.
This paper examines the problem of locating outlier columns in a large, otherwise low-rank, matrix. We propose a simple two-step adaptive sensing and inference approach and establish theoretical guarantees for its performance; our results show that accurate outlier identification is achievable using very few linear sum…
Ens-CGP synthesizes ensemble-based inference with Gaussian processes.
problem Ensemble-based inference and Gaussian process modeling.
method Formulates Ens-CGP as a conditional Gaussian process for ensemble moments.
result Ens-CGP provides a unified probabilistic foundation for Kalman-type methods.
We tackle the problem of collaborative filtering (CF) with side information, through the lens of Gaussian Process (GP) regression. Driven by the idea of using the kernel to explicitly model user-item similarities, we formulate the GP in a way that allows the incorporation of low-rank matrix factorisation, arriving at o…
New method decomposes corrupted data matrices into sparse and low-rank components.
problem Decomposing corrupted data matrices into sparse and low-rank components.
method Discrete optimization approach with alternating minimization, semidefinite relaxation, and branch-and-bound algorithm.
result High-quality solutions and meaningful bounds for SLR problems.
SVD training reduces DNN rank and computation load without SVD per step.
problem High memory and computational load in deep neural networks.
method Explicitly achieves low-rank DNNs during training without SVD per step, using orthogonality regularization and sparsity-inducing regularizers.
result Significantly reduces DNN rank and computation load compared to existing methods.
The paper proposes methods for predicting missing values in mixed data matrices.
problem Matrix completion for mixed data types (continuous, binary, ordinal).
method Generalized latent factor models for low-rank matrix estimation with entrywise consistency.
result Tight probabilistic error bounds for the proposed estimators.
Spectral algorithm reduces samples needed for multitask regression.
problem Jointly recover shared and task-specific components in low-rank multitask regression.
method Common mechanism regression (CMR) model with a non-iterative spectral algorithm.
result Provable non-convex bi-linear structure is overcome with spectral algorithm.
Paper develops an online EM algorithm for graph signal inference from streaming data.
problem Joint inference and clustering of graph signals with non-white excitation.
method Mixture model with low-rank plus sparse prior, online EM algorithm.
result Proposed online EM algorithm converges to MAP solution.
A new method for finding efficient neural interaction functions in collaborative filtering.
problem Finding consistent good performance for complex interactions in collaborative filtering.
method Proposes a search algorithm for simple neural interaction functions (SIF) in CF, using a structured multi-layer perceptron.
result Demonstrates much better prediction performance and distinct IFCs for different data sets and tasks.
Given a matrix M of low-rank, we consider the problem of reconstructing it from noisy observations of a small, random subset of its entries. The problem arises in a variety of applications, from collaborative filtering (the `Netflix problem') to structure-from-motion and positioning. We study a low complexity algorithm…
DCCNNs reduce computational overhead and ambiguity in convolutional neural networks.
problem Reducing computational overhead and ambiguity in convolutional neural networks.
method Introducing a primal learning problem and constructing a dual convex training program, using Fenchel conjugates and Karush-Kuhn-Tucker conditions.
result Eliminates ambiguity and reduces computational overhead in constructing a large kernel matrix.
The paper classifies links with low rank knot Floer and Khovanov homologies.
problem Detecting and classifying links with low rank knot Floer and Khovanov homologies.
method Generalized link Floer homology, used to obtain rank bounds and classify links.
result Knot Floer homology detects T(2,8) and T(2,10). Unified framework for nonconvex matrix completion with linearly parameterized factors.
problem Matrix completion with improved accuracy using linearly parameterized factors.
method Unified nonconvex optimization framework with Correlated Parametric Factorization condition.
result Uniform upper bounds for low-rank estimation at any local minimum.
TASC improves synthetic control for time-series data with trends.
problem Inability of existing SC methods to fully utilize temporal structure in time-series data.
method TASC uses a state-space model with a constant trend and Kalman filter for counterfactual inference.
result TASC offers advantages in settings with strong temporal trends and high observation noise.
StatFEM uses low-rank approximations to scale Bayesian statFEM for high-dimensional problems.
problem Model misspecification and scalability in high-dimensional physical systems.
method Low-rank approximation of covariance matrix, Bayesian filtering, sparse data reconstruction.
result Reconstructs sparsely observed data-generating processes with minimal loss of information.
Paper compares Bayesian and de-biased estimators for low-rank matrix completion.
problem Predict missing entries in partially observed matrices.
method Bayesian and de-biased estimators comparison.
result De-biased estimator performs similarly to Bayesian estimators but is more stable and can outperform in small samples.
Subspace recovery from corrupted and missing data is crucial for various applications in signal processing and information theory. To complete missing values and detect column corruptions, existing robust Matrix Completion (MC) methods mostly concentrate on recovering a low-rank matrix from few corrupted coefficients w…
AdaptiveLRF adapts low-rank factorization for neural networks to improve generalization without sacrificing accuracy.
problem Overfitting in neural networks, especially in shallow and deep models.
method Adaptive Low-Rank Factorization (LRF) applied to neural network layers based on their complexity.
result AdaptiveLRF improves generalization without significantly decreasing training speed or accuracy.
Paper proposes DeCEF layers to reduce CNN complexity.
problem Reduces complexity of CNNs without pre-trained models.
method Develops Depthwise Convolutional Eigen-Filter (DeCEF) layers.
result Achieves similar or higher accuracy with 2/3 parameters and 2/3 FLOPs.
Boolean matrix factorization and Boolean matrix completion from noisy observations are desirable unsupervised data-analysis methods due to their interpretability, but hard to perform due to their NP-hardness. We treat these problems as maximum a posteriori inference problems in a graphical model and present a message p…
New method improves image denoising with fewer parameters and less data.
problem Image denoising requires large datasets and supervised settings, limiting practical applications.
method Self-supervised framework using Tucker low-rank tensor approximation.
result Improves model generalizability and reduces data acquisition costs.
A new GCN variant tackles large eigengaps in dense graphs and hypergraphs.
problem Large eigengaps in dense graphs and hypergraphs hinder popular GCN architectures.
method Uses pseudoinverse of the Laplacian and low-rank approximation for efficient computation.
result Improves runtime and accuracy in various experiments with real-world datasets.
GEnBP combines EnKF and GaBP for efficient high-dimensional inference.
problem Efficient inference in high-dimensional models.
method Gaussian Ensemble Belief Propagation algorithm combining EnKF and GaBP.
result GEnBP outperforms existing methods in accuracy and efficiency.