Paper proposes a new method for sparse covariance Cholesky factor estimation.
problem Estimating sparse covariance matrices for ordered data.
method Matrix loss penalization approach for sparse Cholesky factor estimation.
result The proposed method outperforms existing regression-based approaches in simulations and real data.
This paper develops a fast algorithm for solving nonlinear PDEs using sparse Cholesky factorization.
problem Efficiently solving nonlinear PDEs with Gaussian processes and kernel methods.
method Sparse Cholesky factorization for near-linear complexity.
result Near-linear complexity algorithm for working with kernel matrices of nonlinear PDEs.
A new method learns DAGs from Gaussian data without verifying acyclicity.
problem Learning DAGs from Gaussian data without verifying acyclicity.
method Relaxation technique for permutation matrix estimation and cyclic coordinatewise descent for sparse Cholesky factor estimation.
result The method recovers DAGs without verifying acyclicity constraints.
A new method for efficient causal structure learning at scale.
problem Causal structure learning is computationally challenging at scale.
method Relaxed sparsest-permutation formulation with support-level relaxation and masked zero-fill incomplete Cholesky factorization.
result The method enables scalable comparison of candidate orderings and matches the accuracy of slower baselines.
Method regularizes Cholesky factors to detect nonstationarity in longitudinal data.
problem Detecting nonstationarity in large covariance matrices of longitudinal data.
method Fused-Lasso regularization on Cholesky factors.
result Regularization leads to smooth subdiagonals, indicating nonstationarity.
The modified Cholesky decomposition is commonly used for precision matrix estimation given a specified order of random variables. However, the order of variables is often not available or cannot be pre-determined. In this work, we propose to address the variable order issue in the modified Cholesky decomposition for sp…
We consider multi-task regression models where observations are assumed to be a linear combination of several latent node and weight functions, all drawn from Gaussian process (GP) priors that allow nonzero covariance between grouped latent functions. We show that when these grouped functions are conditionally independ…
A new method for efficient Gaussian process inference using sparse approximations.
problem Scalable and accurate inference for latent Gaussian processes.
method Variational approximation with sparse inverse Cholesky factors and double Kullback-Leibler minimization.
result The proposed method can achieve highly accurate approximations with polylogarithmic time complexity.
Kernel clustering algorithm improved for large datasets using incomplete Cholesky factorization.
problem Large memory usage in kernel-based clustering for large-scale datasets.
method Approximate the kernel matrix using incomplete Cholesky factorization and apply linear k-means clustering. result The proposed method achieves similar performance to kernel k-means clustering but handles large-scale datasets efficiently. Paper proposes a fast algorithm to recover causal DAGs with latent variables.
problem Discovering causal relationships in the presence of latent variables.
method Cholesky factorization of covariance matrix with optimization for latent variables.
result The algorithm significantly outperforms previous methods in synthetic and real-world datasets.
Improved aircraft structure prediction using derivative-enhanced sparse Cholesky GP method.
problem Accurate real-time prediction of aircraft structure performance.
method Combining derivative data with a modified dynamic sparse Cholesky linear system solver.
result Improved prediction accuracy of aircraft structure performance.
New SPD metrics improve stability and efficiency in neural networks.
problem Designing stable and efficient Riemannian metrics on SPD manifolds.
method Cholesky decomposition to derive SPD metrics.
result Proposed metrics provide closed-form operators, computational efficiency, and improved numerical stability.
Bayesian networks are a class of popular graphical models that encode causal and conditional independence relations among variables by directed acyclic graphs (DAGs). We propose a novel structure learning method, annealing on regularized Cholesky score (ARCS), to search over topological sorts, or permutations of nodes,…
The paper explores Cholesky decompositions for symmetric matrices and their geometric properties.
problem Understanding the structure and properties of symmetric matrices through Cholesky decompositions.
method Introducing cones of symmetric matrices, proving Cholesky-type factorizations, and showing geometric properties.
result Each symmetric matrix admits an uncountable family of Cholesky-type factorizations, and these cones are isometric Riemannian manifolds.
RPCholesky approximates kernel matrices with few evaluations.
problem Approximating kernel matrices efficiently.
method Randomly pivoted partial Cholesky factorization.
result RPCholesky provides nearly optimal low-rank approximations.
In this paper, we present a general, multistage framework for graphical model approximation using a cascade of models such as trees. In particular, we look at the problem of covariance matrix approximation for Gaussian distributions as linear transformations of tree models. This is a new way to decompose the covariance…
The sparse inverse covariance estimation problem is commonly solved using an ℓ1-regularized Gaussian maximum likelihood estimator known as "graphical lasso", but its computational cost becomes prohibitive for large data sets. A recent line of results showed--under mild assumptions--that the graphical lasso esti…
This paper proposes the recursive and square-root BLS algorithms to improve the original BLS for new added inputs, which utilize the inverse and inverse Cholesky factor of the Hermitian matrix in the ridge inverse, respectively, to update the ridge solution. The recursive BLS updates the inverse by the matrix inversion…
We propose an algorithmic framework for convex minimization problems of a composite function with two terms: a self-concordant function and a possibly nonsmooth regularization term. Our method is a new proximal Newton algorithm that features a local quadratic convergence rate. As a specific instance of our framework, w…
We consider the problem of learning a Gaussian variational approximation to the posterior distribution for a high-dimensional parameter, where we impose sparsity in the precision matrix to reflect appropriate conditional independence structure in the model. Incorporating sparsity in the precision matrix allows the Gaus…
New algorithm extends Greville's method for partitioned matrices efficiently and stably.
problem Efficiently compute pseudoinverse of partitioned matrices without retraining.
method Incorporates inverse Cholesky factorization to reduce computational complexity and improve stability.
result 1 iteration to compute pseudoinverse of whole matrix from first part, addressing all cases.
Efficiently discovers causal DAG permutations without additional assumptions.
problem Learning a DAG up to Markov equivalence.
method Utilizing DAG-specific problem structure, introduces an efficient algorithm for sparse permutations.
result Significant improvement in permutation discovery compared to existing algorithms.
We demonstrate an equivalence between reproducing kernel Hilbert space (RKHS) embeddings of conditional distributions and vector-valued regressors. This connection introduces a natural regularized loss function which the RKHS embeddings minimise, providing an intuitive understanding of the embeddings and a justificatio…
Conjugate gradient methods improve efficiency for high-dimensional GLMMs.
problem Efficiency bottleneck in computing high-dimensional GLMM precision matrices.
method Combining spectral analysis and random graph theory with conjugate gradient methods.
result CG-based methods achieve linear scaling in cost with model parameters and observations.
We present a new Riemannian metric, termed Log-Cholesky metric, on the manifold of symmetric positive definite (SPD) matrices via Cholesky decomposition. We first construct a Lie group structure and a bi-invariant metric on Cholesky space, the collection of lower triangular matrices whose diagonal elements are all posi…
New heuristic selects fewer assets for efficient portfolios, reducing costs.
problem High transaction costs and fees from including many assets in portfolios.
method Surrogate formulation to select assets, re-optimizes portfolio with fewer assets.
result Effective in constructing portfolios with fewer assets, reducing costs.
New quadrature method using randomly pivoted Cholesky outperforms existing techniques.
problem Efficiently approximating integrals of functions in reproducing kernel Hilbert spaces.
method Nodes drawn by randomly pivoted Cholesky algorithm.
result Randomly pivoted Cholesky quadrature is fast and achieves comparable accuracy to more computationally intensive methods.
The original Broad Learning System (BLS) on new added nodes and its existing efficient implementation both assume the ridge parameter lambda -> 0 in the ridge inverse to approximate the generalized inverse, and compute the generalized inverse solution for the output weights. In this paper, we propose two ridge solution…
New method differentiates square-root Kalman filters robustly.
problem Gradient calculation issues in square-root Kalman filters.
method Closed-form chain rule derived from Gramian identity, resolves non-orthogonal and rank-deficient issues.
result Robust automatic differentiation for Kalman filters, resolving numerical stability and gradient issues.
Scalable algorithm for sampling Gaussian processes using sparse grids and preconditioners.
problem Generating high-dimensional Gaussian random vectors for GP sampling is computationally challenging.
method Proposes a scalable algorithm using inducing points approximation with sparse grids and additive Schwarz preconditioners.
result Demonstrates the efficacy and accuracy of the proposed method through experiments and comparisons.
New Krylov subspace methods speed up mixed-effects models with crossed random effects.
problem Slow computations for high-dimensional crossed random effects in mixed-effects models.
method Krylov subspace-based methods for generalized mixed-effects models with cross effects.
result Speedups by factors of up to 10,000 in computations for mixed-effects models.
A scalable algorithm for GP regression selects relevant covariates efficiently.
problem Scalable variable selection in large GP regression models.
method VGPR algorithm using Vecchia approximation for sparse precision matrix, mini-batch subsampling.
result Improved scalability and accuracy in selecting relevant covariates.
New method for geodesics of multivariate normals, derived from a Toda lattice.
problem Computing geodesics of multivariate normal distributions.
method Using block Cholesky decomposition and a natural Riemannian submersion, a new Toda lattice type Lax pair is derived.
result A new Toda lattice type Lax pair derived from geodesics and block Cholesky decomposition.
Accelerated RPCholesky speeds up kernel matrix approximations.
problem Efficiently approximating large kernel matrices.
method Accelerated randomly pivoted Cholesky (RPCholesky) with block matrix computations and rejection sampling.
result Approximates kernel matrices up to 40 times faster.
New method trains sparse Gaussian processes without matrix inversion.
problem Costly training of Gaussian processes at scale.
method Inverse-free approach using matmul-only natural-gradient updates.
result Significantly improved stability and convergence in training.
Develops a fast algorithm for fitting multilevel factor models.
problem Fitting multilevel factor models with covariance structure.
method Novel expectation-maximization algorithm tailored for multilevel factor models.
result Shows efficient computation of inverse of positive definite MLR matrix.
Spike and Slab priors have been of much recent interest in signal processing as a means of inducing sparsity in Bayesian inference. Applications domains that benefit from the use of these priors include sparse recovery, regression and classification. It is well-known that solving for the sparse coefficient vector to ma…
New method for hyperparameter tuning in sparse matrix factorization.
problem Hyperparameter tuning in sparse matrix factorization.
method Numerical method based on evaluating the zero point of normalization factor in sparse matrix prior.
result Our method outperforms existing algorithms in ground-truth sparse matrix reconstruction.
Solving systems of linear equations is a problem occuring frequently in water engineering applications. Usually the size of the problem is too large to be solved via direct factorization. One can resort to iterative approaches, in particular the conjugate gradients method if the matrix is symmetric positive definite. P…
In many applications, data come with a natural ordering. This ordering can often induce local dependence among nearby variables. However, in complex data, the width of this dependence may vary, making simple assumptions such as a constant neighborhood size unrealistic. We propose a framework for learning this local dep…
Sparse APCA identifies sparse factors in financial returns over time.
problem Analyzing co-movements of high-dimensional panel data over time.
method Sparse asymptotic PCA with truncated power method for sparse factors and sequential deflation for multi-factor cases.
result Identification of nine risk factors influencing the S&P 500 stock market.
Proposes FARM model combining latent factor and sparse regression.
problem Testing adequacy of latent factor and sparse regression models.
method Factor Augmented sparse linear Regression Model (FARM) with FabTest and ANOVA type tests.
result Model robustness and effectiveness validated through experiments.
We observe that gradients computed via the reparameterization trick are in direct correspondence with solutions of the transport equation in the formalism of optimal transport. We use this perspective to compute (approximate) pathwise gradients for probability distributions not directly amenable to the reparameterizati…
A new method speeds up Bayesian Optimization for hyperparameter tuning.
problem Efficient hyperparameter tuning for machine learning models.
method Lazy Gaussian Processes approximation to reduce cubic complexity to quadratic.
result Significant speedup in Bayesian Optimization, up to 162x in single node.
Deep weight factorization improves neural network training through smooth optimization of sparse penalties.
problem Challenges in applying sparse regularization in neural networks due to non-differentiability of penalties.
method Introduces deep weight factorization, decomposing weights into multiple factors for smooth optimization of L1-penalized networks. result Deep weight factorization outperforms shallow factorization and pruning methods consistently across various architectures and datasets.
Sparse VAE learns latent factors from high-dimensional data.
problem Unsupervised representation learning on high-dimensional data.
method Sparse VAE model that learns latent factors summarizing data associations.
result Sparse VAE can recover true model parameters with infinite data.
Bayesian model infers factor dimensionality and sparse loading matrix adaptively.
problem Inference of high-dimensional sparse factor model with varying sparsity and factor dimensions.
method Adaptive Bayesian sparse factor model with posterior concentration.
result Posterior distribution asymptotically concentrates on true factor dimensionality and sparsity.
Sparse GFA identifies disease factors in FTD subgroups.
problem Heterogeneity in neurological disorders hinders understanding and treatment.
method Sparse Group Factor Analysis (GFA) with regularised horseshoe priors.
result Identified latent disease factors differentially expressed in FTD subgroups.