A framework estimates multiple precision matrices with shared structures.
problem Estimating multiple precision matrices with shared structures.
method Penalized likelihood framework with iterative algorithm alternating between convex and clustering problems.
result The method outperforms competitors and performs similarly to methods using prior information.
New bounds improve minimax estimation of banded precision matrices.
problem Estimating banded precision matrices with optimal rates.
method Inverting wider blocks of empirical covariance matrices to estimate subblocks of precision matrices.
result Minimax rate matches for banded covariance matrices, improving previous bounds.
Sparse precision matrices in Gaussian variational approximations for high-dimensional models.
problem Learning posterior distributions with high-dimensional parameters and conditional independence structures.
method Sparse precision matrix parameterization and efficient stochastic gradient optimization methods.
result Flexibility and parsimony in Gaussian variational distributions achieved through sparsity in precision matrices.
The paper improves Bayesian precision matrix estimation for high-dimensional sparse data.
problem Estimating sparse precision matrices in high-dimensional settings.
method Tempered posterior with fully specified horseshoe prior.
result Concentration results and theoretical oracle inequality for posterior.
Conjugate gradient methods improve efficiency for high-dimensional GLMMs.
problem Efficiency bottleneck in computing high-dimensional GLMM precision matrices.
method Combining spectral analysis and random graph theory with conjugate gradient methods.
result CG-based methods achieve linear scaling in cost with model parameters and observations.
We develop a class of rules spanning the range between quadratic discriminant analysis and naive Bayes, through a path of sparse graphical models. A group lasso penalty is used to introduce shrinkage and encourage a similar pattern of sparsity across precision matrices. It gives sparse estimates of interactions and pro…
A new algorithm improves GLasso for sparse precision matrix estimation.
problem Efficiently estimating sparse precision matrices in high-dimensional data.
method A new reparametrization and iterative block coordinate descent algorithm.
result Improved performance comparable to DP-GLasso with a simpler optimization target.
High-dimensional inference for sparse spectral precision matrices
problem Inference on the spectral precision matrix at a fixed frequency
method Full likelihood-based inference using neighboring discrete Fourier transforms
result Simultaneous control of regularization, finite-sample truncation, and smoothing biases
SpInGP speeds up Gaussian process computations with sparse matrices.
problem Efficiently computing Gaussian processes for large datasets.
method Sparse precision Gaussian process formulation and parallelizable matrix routines.
result The parallelized SpInGP reduces time complexity to sublinear.
Simplified optimization for structured matrices in deep learning.
problem Computational challenges in Riemannian submanifold optimization for structured symmetric positive-definite matrices.
method Proposed a generalized Riemannian normal coordinates that dynamically orthonormalizes the metric and converts the problem into an unconstrained Euclidean space problem.
result Simplified existing approaches for structured covariances and developed matrix-inverse-free 2nd-order optimizers for deep learning with low precision.
ISEE method efficiently estimates large precision matrices in Gaussian graphical models.
problem Estimating large precision matrices in ultra-large Gaussian graphical models.
method ISEE method combines sparse modeling and large covariance matrix estimation.
result ISEE method can recover graphical structure with significant probability and efficient estimation of link strengths.
Bayesian method estimates sparse precision matrices with unequal shrinkage.
problem Estimating high-dimensional sparse precision matrices with unequal shrinkage.
method Bayesian framework with mixture of Laplace priors and EM algorithm for computation.
result Optimal error rates and selection consistency for sparse structure recovery.
We learn sparse precision matrices from compressed data sketches.
problem Learning a graph from high-dimensional data with limited storage.
method Estimate a sparse precision matrix from a sketch of the data using non-linear random features.
result It is possible to estimate a sparse precision matrix from a sketch of size $m=Ω\left((d+2k)\log(d)
ight)$.
Improved method for estimating precision matrices without knowing variable order.
problem Estimating precision matrices without knowing the order of variables.
method Combining multiple permutations and thresholding for sparse structure.
result Consistent property established under weak conditions, superior performance in simulations and real data.
New GPU kernels boost deep learning speed and memory efficiency.
problem Sparse deep learning matrices are not well-suited for existing sparse kernels.
method Identified favorable properties of sparse matrices from deep learning, developed high-performance GPU kernels for sparse matrix operations.
result 27% of single-precision peak performance on Nvidia V100 GPUs achieved with new kernels.
Motivated by a sampling problem basic to computational statistical inference, we develop a nearly optimal algorithm for a fundamental problem in spectral graph theory and numerical analysis. Given an n×n SDDM matrix M, and a constant −1≤p≤1, our algorithm gives efficient access to a…
This paper establishes information-theoretic limits in estimating a finite field low-rank matrix given random linear measurements of it. These linear measurements are obtained by taking inner products of the low-rank matrix with random sensing matrices. Necessary and sufficient conditions on the number of measurements …
This paper proposes a new method for estimating sparse precision matrices in the high dimensional setting. It has been popular to study fast computation and adaptive procedures for this problem. We propose a novel approach, called Sparse Column-wise Inverse Operator, to address these two issues. We analyze an adaptive …
Sparse DNNs simplify DNN mathematics and reveal exact solutions.
problem Sparse DNNs to overcome memory limitations in large, complex models.
method Associative array algebra to simplify DNN mathematics and construct exact solutions.
result Construct exact solutions and perturbation models for ReLU DNN equations.
New method trains sparse Gaussian processes without matrix inversion.
problem Costly training of Gaussian processes at scale.
method Inverse-free approach using matmul-only natural-gradient updates.
result Significantly improved stability and convergence in training.
Convex optimization with expander matrices improves sparse recovery efficiency.
problem Sparse recovery from linear measurements using expander matrices.
method Use of expander matrices for linear sketches in convex optimization to recover block-sparse matrices.
result The recovery error can be expressed in terms of the model-based norm, ensuring the solution is within the model.
Proposes a new method for estimating sparse precision matrices in GMRF-MM models.
problem Difficulty in learning GMMs with large parameters and limited data.
method Restricts GMM to GMRF-MM, proposes efficient optimization for sparse precision matrices, and debiases the estimates.
result Debiasing approach outperforms GLASSO in single-GMRF and GMRF-MM cases.
Approximating Gaussian Whittle-Matern Fields over Well-Centered Triangulations of Riemannian Manifolds
problem Convergent approximation of Gaussian Whittle-Matern fields on Riemannian manifolds
method Finite Element approximation of SPDEs
result Universal approximation of precision and covariance matrices
Recently, there has been focus on penalized log-likelihood covariance estimation for sparse inverse covariance (precision) matrices. The penalty is responsible for inducing sparsity, and a very common choice is the convex l1 norm. However, the best estimator performance is not always achieved with this penalty. The …
Diagonal transformations preserve independence structures in non-Gaussian distributions.
problem Preserving independence structures in non-Gaussian distributions.
method Diagonal nonlinear transformations of multivariate normal variables.
result Independence structures are preserved in non-Gaussian distributions under diagonal transformations.
RadiX-Net generates diverse sparse neural topologies.
problem Sparse neural networks require more efficient storage and training.
method Deterministically generates RadiX-Nets from sparse topologies.
result RadiX-Nets can train to the same precision as dense DNNs at lower cost.
Framework estimates precision matrices for heterogeneous populations.
problem Estimating precision matrices in populations with subpopulations.
method Laplacian shrinkage penalty, ADMM algorithm, hierarchical clustering.
result Consistent estimation of precision matrices in heterogeneous populations.
Denise learns a function to quickly decompose covariance matrices robustly.
problem Robustly decomposing covariance matrices for feature extraction.
method Deep learning for symmetric positive semidefinite matrices.
result Denise achieves state-of-the-art performance in decomposition quality and speed.
A new RPCA method is faster and more scalable than AltProj.
problem Recovering low-rank matrices from data matrices with sparse parts.
method Factorization-based RPCA model with complexity O(kdn). result Our method is about 4 times faster than AltProj and more scalable.
Ranky solves SVD for large sparse matrices in distributed systems.
problem Rank problem in large sparse matrices for SVD.
method Distributed approach to solve rank problem.
result Recovers SVD with negligible error for large sparse matrices.
Proposes a method for forecasting large-scale interval-valued time series.
problem Modeling and forecasting large-scale interval-valued time series.
method Feature extraction procedure involving auto-segmentation, clustering, and precision matrix estimation.
result The method enhances forecasting performance for large-scale interval-valued time series.
The Gaussian graphical model, a popular paradigm for studying relationship among variables in a wide range of applications, has attracted great attention in recent years. This paper considers a fundamental question: When is it possible to estimate low-dimensional parameters at parametric square-root rate in a large Gau…
Efficiently estimates sparse inverse covariance matrices in distributed systems.
problem Communication issues in distributed systems for estimating inverse covariance matrices.
method Proposes a method where each machine transfers a small subset of the entries of the inverse covariance matrix in a single round of communication.
result Error rates comparable with non-distributed settings, and correct model selection possible.
This paper solves quadratic systems with sparse or generative priors.
problem Recovering signals from quadratic systems with full-rank matrices.
method Thresholded Wirtinger flow (TWF) and projected gradient descent (PGD) algorithms.
result The proposed methods significantly outperform existing algorithms in signal recovery.
Efficiently infers sparse networks from count data with reduced memory usage.
problem Sparse network inference for count data with high dimensions and dependencies.
method Improved Bigraphical Lasso using eigenvalue decomposition of Cartesian product graph.
result Reduced computational complexity from O(n2p2) to O(n2+p2). Random projections help in representing sparse graphs efficiently.
problem Efficiently representing sparse graphs of varying sizes and vertex sets.
method Random projection of adjacency matrices to retain graph functionality and properties.
result Random projections can accurately represent graphs of different sizes and vertex sets in the same space.
Efficiently represents large geodesic distance matrices for MDS analysis.
problem Quadratic growth of geodesic distance matrices for large point sets.
method Sparse biharmonic interpolation to learn a subset of points for efficient approximation.
result 2x faster and 20x less memory usage than current methods, enabling analyses of large point sets.
Method estimates M-matrices in graphical models with improved accuracy.
problem Estimating M-matrices as precision matrices in Gaussian graphical models.
method Adaptive multiple-stage estimation method solving weighted ℓ1-regularized problems.
result Method outperforms state-of-the-art methods in precision matrix estimation and graph edge identification.
Paper predicts travel costs across regions using neural networks.
problem Predicting travel costs in sparse, stochastic OD matrices.
method Recurrent Multi-Graph Neural Networks (R-MGNN) for sparse, stochastic OD matrix forecasting.
result Framework effectively predicts future OD matrices without empty elements.
New method speeds up sparse graph neural networks training on dense hardware.
problem Training sparse graph neural networks is slow on custom hardware.
method Inspired by sparse matrix optimization, developed techniques for dense hardware.
result Sparse graph neural networks trained in 13 minutes on 512-core TPUv2 Pod.
New algorithm recovers matrices that are both low rank and sparse in rows and columns.
problem Recovering matrices that are simultaneously low rank and row/column sparse.
method Gradient Descent with hard Thresholding (GDT) algorithm to minimize a bi-convex function over a nonconvex set of constraints.
result GDT achieves linear convergence to near optimal solutions with statistical error.
This paper considers the problem of networks reconstruction from heterogeneous data using a Gaussian Graphical Mixture Model (GGMM). It is well known that parameter estimation in this context is challenging due to large numbers of variables coupled with the degeneracy of the likelihood. We propose as a solution a penal…
This work compresses heavy-tailed weight matrices for tighter generalization bounds.
problem Empirical evidence linking heavy-tailed weight matrices to test set accuracy but lack of formal relationship with generalization bounds.
method Utilized the compression framework to show that heavy-tailed matrices can be compressed, resulting in sparse weight matrices.
result Demonstrated a non-vacuous generalization bound for compressed networks with heavy-tailed weight matrices.
Improved perturbation reduces matrix condition number to O(n) with minimal storage.
problem Reducing the condition number of deterministic matrices for efficient algorithmic use.
method Introduced pattern matrices and sparse perturbations with dependent entries.
result Condition number reduced to O(n) with O(n) random numbers in O(log n) precision.
The paper improves support recovery in high-dimensional precision matrix estimation using meta learning.
problem Support recovery in high-dimensional precision matrix estimation with reduced sample complexity.
method Pooling samples from different tasks and using an improper ℓ1-regularized log-determinant Bregman divergence to estimate a single precision matrix. result The support of the improperly estimated single precision matrix is equal to the true support union with high probability.
Paper tackles non-convex optimization and statistical inference for tensor graphical models.
problem Estimating and inferring dependency structure in tensor-valued data.
method Alternating minimization algorithm for non-convex optimization and de-biased statistical inference.
result Proves alternating minimization attains optimal statistical rate of convergence and proposes FDR control for testing hypotheses.
rags2ridges simplifies graphical modeling of high-dimensional data.
problem Graphical modeling of high-dimensional precision matrices.
method Modular framework for extraction, visualization, and analysis of Gaussian graphical models.
result Provides a one-stop-shop for graphical modeling of high-dimensional precision matrices.
Proposes a new graph representation method using tensor products.
problem Dynamic graph representation and theoretical properties.
method Bind-and-sum approach in hyperdimensional computing (HDC), tensor product as binding operation.
result Memory vs. size analysis of graph representation size scaling.