The paper analyzes Karcher means on restricted PSD matrices with statistical guarantees.
problem Statistical analysis of non-linear manifolds in machine learning.
method Intrinsic mean model on restricted PSD matrices, Karcher mean analysis, extrinsic signal-plus-noise model.
result Non-asymptotic statistical analysis of Karcher means with deterministic error bounds.
Rank-one measurements limit feasible sets for low-rank PSD matrices.
problem Feasibility of PSD matrices under rank-one measurements.
method Characterization of feasible sets for PSD matrices given rank-one projections.
result Radius of feasible sets determines singleton solution sets for low-rank matrices.
New methods for sketching non-PSD matrices improve regression and optimization tasks.
problem Efficiently handling non-PSD matrices in computations.
method Developed novel matrix sketching techniques for non-PSD and complex matrices.
result Improved performance in convex and non-convex optimization, regression, and vector-matrix-vector queries.
We consider whether algorithmic choices in over-parameterized linear matrix factorization introduce implicit regularization. We focus on noiseless matrix sensing over rank-r positive semi-definite (PSD) matrices in Rn×n, with a sensing mechanism that satisfies restricted isometry properties (RIP)…
An algorithm for computing positive semidefinite factorizations of matrices.
problem Computing positive semidefinite factorizations of matrices.
method Non-commutative extension of Lee-Seung's algorithm (Matrix Multiplicative Update, MMU).
result The MMU algorithm ensures PSD updates and achieves critical points.
Improved covariance matrix estimation for portfolio optimization with guaranteed PSD and controlled conditioning.
problem Guaranteeing positive semidefinite ness and controlling spectral conditioning in IQ estimators.
method Introducing squeezing identity and atomic-IQ parameterization to construct structured channel matrices with PSD guarantees and analytic eigen floor for conditioning control.
result Atomic-IQ improves Sharpe ratios and delivers a more stable risk profile compared to standard estimators.
Paper introduces a new kernel model for PSD-valued functions with theoretical guarantees and applications.
problem Enforcing positive semi-definiteness (PSD) in function models with good performance and theoretical guarantees.
method Kernel sum-of-squares model for PSD-valued functions, extending previous models for non-negative scalar functions.
result The model constitutes a universal approximator of PSD functions and can represent any smooth and strongly convex function.
The paper presents two schemes for sampling matrices from specific distributions on a manifold.
problem Sampling matrices from Gibbs distributions on the manifold of positive semi-definite matrices with fixed rank.
method Two explicit schemes based on Euler-Maruyama discretization of the Riemannian Langevin equation with Brownian motion on the manifold.
result Numerical validation of the schemes using specific energy functions and metrics.
This paper describes a versatile method that accelerates multichannel source separation methods based on full-rank spatial modeling. A popular approach to multichannel source separation is to integrate a spatial model with a source model for estimating the spatial covariance matrices (SCMs) and power spectral densities…
A new algorithm improves sampling for graph learning models.
problem Euclidean proposals struggle near the boundary of PSD matrices.
method ConeMALA, a geometry-aware Langevin algorithm.
result ConeMALA achieves higher ESS/sec and stable diagnostics.
Unified approach to Bayesian inference with guarantees on covariance matrices.
problem Approximate Bayesian inference with PSD guarantees.
method Bayes-Newton methods extending Newton's method for optimisation.
result Novel algorithms with PSD covariance matrices.
ITSPACE improves covariance alignment faster than other methods.
problem Optimizing covariance matrices for machine learning tasks.
method Proximal majorization-minimization method that directly optimizes the Bures-Wasserstein objective.
result ITSPACE achieves lower BW gap solutions faster than other methods.
PSD models simplify probability density estimation.
problem Effective modeling of probability densities for inference.
method Positive semi-definite (PSD) models for non-negative functions.
result PSD models efficiently support product and sum rules.
A new imputation method estimates missing values by matching observed marginals from masked data.
problem Missing values in data undermine statistical and machine learning analysis.
method Estimates a distribution from masked observations using positive semi-definite kernel density estimation.
result The method yields both single and multiple imputations from the same fitted density, with statistical consistency and fast adaptive excess risk.
Despite the success of the popular kernelized support vector machines, they have two major limitations: they are restricted to Positive Semi-Definite (PSD) kernels, and their training complexity scales at least quadratically with the size of the data. Many natural measures of similarity between pairs of samples are not…
Efficient approximations reduce computation of matrix-based Renyi's entropy.
problem High computational complexity of matrix-based Renyi's entropy.
method Taylor, Chebyshev, and Lanczos approximations to reduce complexity.
result Reduced complexity to significantly less than O(n2) with negligible accuracy loss. Several important applications, such as streaming PCA and semidefinite programming, involve a large-scale positive-semidefinite (psd) matrix that is presented as a sequence of linear updates. Because of storage limitations, it may only be possible to retain a sketch of the psd matrix. This paper develops a new algorith…
Improving scalability and stability of Stein discrepancies for scalable goodness-of-fit testing
problem Improving scalability and stability of Stein discrepancies for scalable goodness-of-fit testing
method Reformulating Stein discrepancy construction as an explicit SNR^2 maximisation problem
result Avoiding exponential SNR^2 collapse and achieving stable SNR^2
We consider the non-square matrix sensing problem, under restricted isometry property (RIP) assumptions. We focus on the non-convex formulation, where any rank-r matrix X∈Rm×n is represented as UV⊤, where U∈Rm×r and V∈Rn×r. In this paper…
This paper studies geodesics between covariance matrices of different ranks using the Bures-Wasserstein metric.
problem Geodesics between covariance matrices of varying ranks.
method Analyzes the Bures-Wasserstein distance on covariance matrices, completing previous work on geodesics and providing explicit formulas.
result The set of all minimizing geodesics between two covariance matrices is parametrized by a closed unit ball in R(k−r)imes(l−r). New PSDMF algorithms derived from PR and ARM methods.
problem Positive semidefinite matrix factorization (PSDMF) challenges.
method Design PSDMF algorithms based on phase retrieval (PR) and affine rank minimization (ARM) methods.
result New PSDMF algorithms inherit numerical properties from PR and ARM methods.
New method for inferring time series graph from sparse-group log-sum penalty.
problem Inferring conditional independence graph from high-dimensional stationary multivariate Gaussian time series.
method Sparse-group log-sum penalty (LSP) and alternating direction method of multipliers (ADMM) for iterative optimization.
result Local convergence of inverse PSD estimators to the true value with rate of convergence.
Machine learning uncovers hidden correlations in granular material behavior.
problem Predicting mechanical behavior of granular materials from particle size distributions.
method Used Discrete Element Method to generate packings, trained artificial Neural Network.
result Artificial Neural Network revealed hidden correlations between particle size distributions and mechanical behavior.
Develops nonstationary MOGP kernels for better performance.
problem Limited applicability of existing MOGP kernels for nonstationary data.
method Harmonizable spectral mixture kernels for nonstationary MOGP.
result Automatic identification of nonstationary behavior in data.
Bayesian optimization gains efficiency by leveraging symmetries through a modified max kernel.
problem Improving Bayesian optimization efficiency for functions with group symmetries.
method Developed a PSD projection of the max kernel to exploit symmetries without violating kernel properties.
result The modified max kernel achieves lower regret compared to existing invariant and non-invariant kernels.
Scaled gradient descent improves matrix recovery for ill-conditioned matrices with optimal sampling complexity.
problem Recovering low-rank matrices from limited measurements efficiently and accurately.
method Scaled gradient descent (ScaledGD) with optimal sample complexity and improved iteration complexity.
result ScaledGD achieves optimal sample complexity and improved iteration complexity for ill-conditioned matrices.
New filters for non-linear systems achieve closed-form solutions.
problem Intractability of Bayesian filtering for non-linear systems.
method Gaussian PSD Models for efficient closed-form filtering.
result Closed-form filtering with strong theoretical guarantees and adaptive error.
Recurrent neural networks are powerful models for processing sequential data, but they are generally plagued by vanishing and exploding gradient problems. Unitary recurrent neural networks (uRNNs), which use unitary recurrence matrices, have recently been proposed as a means to avoid these issues. However, in previous …
Optimal sample complexity for learning DDAGs from noisy data.
problem Learning interactions in linear dynamical systems over DAGs.
method Proposed a metric and algorithm based on PSD matrix for reconstruction.
result Optimal sample complexity n=Θ(qlog(p/q)) for learning DDAGs. In this paper, we study the prediction of a circularly symmetric zero-mean stationary Gaussian process from a window of observations consisting of finitely many samples. This is a prevalent problem in a wide range of applications in communication theory and signal processing. Due to stationarity, when the autocorrelati…
We introduce Clique Matrices as an alternative representation of undirected graphs, being a generalisation of the incidence matrix representation. Here we use clique matrices to decompose a graph into a set of possibly overlapping clusters, de ned as well-connected subsets of vertices. The decomposition is based on a s…
The paper detects amateur drones using acoustic signals, overcoming interference.
problem Detection of unmonitored UAVs (amateur drones) for security and safety.
method Machine learning approach using Independent Component Analysis (ICA) for acoustic signal detection.
result The RMS values of PSD with KNN outperform other methods in detecting AmDr.
We define homological matrices, construct examples of one-dimension restricted homological quantum field theories, and show a relationship between the two theories.
It is natural to ask: what kinds of matrices satisfy the Restricted Eigenvalue (RE) condition? In this paper, we associate the RE condition (Bickel-Ritov-Tsybakov 09) with the complexity of a subset of the sphere in Rp, where p is the dimensionality of the data, and show that a class of random matrices with indep…
New geometric framework for positive semidefinite matrices of fixed rank.
problem Statistical analysis of positive semidefinite matrices of fixed rank.
method Introducing a manifold S(n,p)∗ with Riemannian geometry and Lie group structure. result Analytical closed forms for geodesics and Fréchet means.
We prove ultradifferentiable Chevelley restriction theorems for a wide range of ultradifferentiable classes. As a special case we find that isotropic functions, i.e., functions defined on the vector space of real symmetric matrices invariant under the action of the special orthogonal group by conjugation, possess some …
Paper studies asymmetric matrix sensing, proving gradient descent converges to low-rank solutions.
problem Reconstructing asymmetric low-rank matrices from linear measurements.
method Factorized gradient descent with coupling and regularization properties.
result Gradient descent from small random initialization converges to globally optimal and generalizing solutions.
We consider the problem of clustering noisy finite-length observations of stationary ergodic random processes according to their nonparametric generative models without prior knowledge of the model statistics and the number of generative models. Two algorithms, both using the L1-distance between estimated power spectra…
This paper proposes an efficient method for sampling from stochastic differential equations using PSD models.
problem Efficient sampling from stochastic differential equations with positive semi-definite models.
method The approach leverages a PSD model to sample from the Fokker-Planck equation or its fractional variant, with a complexity of m2dlog(1/ε). result The method produces i.i.d. samples with error ε in Wasserstein-1 distance, with a cost of O(dε−2(d+1)/β−2log(1/ε)2d+3) per sample. A new method for efficiently computing derivatives of skew-symmetric matrix exponentials.
problem Efficient computation of derivatives for skew-symmetric matrices.
method Characterization of invertibility, construction of nearby logarithm, and efficient implementation.
result Explicit formulae for differentiation and its inverse of skew-symmetric matrix exponentials.
RPCholesky approximates kernel matrices with few evaluations.
problem Approximating kernel matrices efficiently.
method Randomly pivoted partial Cholesky factorization.
result RPCholesky provides nearly optimal low-rank approximations.
We consider the problem of clustering noisy finite-length observations of stationary ergodic random processes according to their generative models without prior knowledge of the model statistics and the number of generative models. Two algorithms, both using the L1-distance between estimated power spectral densities…
New kernel handles irregularly-spaced multivariate time series.
problem No kernel exists for irregularly-spaced multivariate time series.
method Built a series kernel from vector kernels, ensuring it's PSD.
result Validated the series kernel on multiple datasets and time series classification.
New algorithm helps escape saddle points in optimization problems.
problem Optimizing smooth non-convex functions to avoid saddle points.
method Perturbed Saddle-escape Descent (PSD) algorithm with explicit constants.
result PSD finds approximate second-order stationary points efficiently.
Simplified analysis of SGD for linear regression with weight averaging.
problem Understanding SGD optimization in linear regression models.
method Simplified analysis using linear algebra tools, bypassing complex operator manipulations.
result Recovery of bias and variance bounds for SGD in linear regression.
A new method for assessing Bayesian sampling quality, PSD, is proposed and shown to be more powerful and efficient.
problem Scalability and convergence assessment of Bayesian sampling algorithms, especially for high-dimensional problems.
method Polynomial Stein Discrepancy (PSD) for measuring discrepancy between samples and posterior distributions.
result PSD detects differences in the first r moments for Gaussian targets and is more powerful and efficient than competitors.
Matrices satisfying the Restricted Isometry Property (RIP) play an important role in the areas of compressed sensing and statistical learning. RIP matrices with optimal parameters are mainly obtained via probabilistic arguments, as explicit constructions seem hard. It is therefore interesting to ask whether a fixed mat…
Power spectral density (PSD) maps providing the distribution of RF power across space and frequency are constructed using power measurements collected by a network of low-cost sensors. By introducing linear compression and quantization to a small number of bits, sensor measurements can be communicated to the fusion cen…