arXiv research

We propose a novel estimation approach for the covariance matrix based on the $l_1$-regularized approximate factor model. Our sparse approximate factor (SAF) covariance estimator allows for the existence of weak factors and hence relaxes the pervasiveness assumption generally adopted for the standard approximate factor…

Covariance pooling is a feature pooling method with good classification accuracy. Because covariance features consist of second-order statistics, the scale of the feature elements are varied. Therefore, normalizing covariance features using a matrix square root affects the performance improvement. When pooling methods …

Bayesian model averaging fails under covariate shift, affecting neural networks' performance.

We conduct a study of the aliased spectral densities of Matérn covariance functions on a regular grid of points, providing clarity on the properties of a popular approximation based on stochastic partial differential equations; while others have shown that it can approximate the covariance function well, we find that i…

A scalable algorithm for GP regression selects relevant covariates efficiently.

Paper develops an online covariance estimator for nonsmooth stochastic approximation problems.

This study approximates distances between Gaussian processes and covariance operators using RKHS.

Standard sparse pseudo-input approximations to the Gaussian process (GP) cannot handle complex functions well. Sparse spectrum alternatives attempt to answer this but are known to over-fit. We suggest the use of variational inference for the sparse spectrum approximation to avoid both issues. We model the covariance fu…

Much recent work has concerned sparse approximations to speed up the Gaussian process regression from the unfavorable O(n3) scaling in computational time to O(nm2). Thus far, work has concentrated on models with one covariance function. However, in many practical situations additive models with multiple covariance func…

The covariance matrix of a $p$-dimensional random variable is a fundamental quantity in data analysis. Given $n$ i.i.d. observations, it is typically estimated by the sample covariance matrix, at a computational cost of $O(np^{2})$ operations. When $n,p$ are large, this computation may be prohibitively slow. Moreover, …

Unified error analysis for low-rank approximation improves data assimilation performance.

High precision analytical approximation is proposed for variance-covariance based risk allocation in a portfolio of risky assets. A general case of a single-period multi-factor Merton-type model with stochastic recovery is considered. The accuracy of the approximation as well as its speed are compared to and shown to b…

Geometric families of low-rank covariances improve flexibility and tractability in high dimensions.

Novel neural GP kernels learn stable, flexible covariance structures.

Improves BBVI for high-dimensional Gaussian approximations by using low-rank approximations.

A latent force model is a Gaussian process with a covariance function inspired by a differential operator. Such covariance function is obtained by performing convolution integrals between Green's functions associated to the differential operators, and covariance functions associated to latent functions. In the classica…

This paper presents a new method for estimating high dimensional covariance matrices. The method, permuted rank-penalized least-squares (PRLS), is based on a Kronecker product series expansion of the true covariance matrix. Assuming an i.i.d. Gaussian random sample, we establish high dimensional rates of convergence to…

New methods estimate covariance for matrix data without assuming fixed size or specific distributions.

New method stabilizes private LASSO for high-dimensional data with diverse covariate scales.

Linear cost method approximates Gaussian Matérn processes with exponentially convergent accuracy.

Paper solves a key problem in learning from high-dimensional covariance matrices.

A new method reduces high-dimensional filtering to quadratic complexity.

Study integrates machine learning with SAA for optimizing decisions based on uncertain parameters and covariates.

Our article considers a Gaussian variational approximation of the posterior density in a high-dimensional state space model. The variational parameters to be optimized are the mean vector and the covariance matrix of the approximation. The number of parameters in the covariance matrix grows as the square of the number …

New distribution simplifies covariance matrix inference.

Paper estimates Hurst parameter from implied volatilities.

New method prevents posterior collapse in iVAE models.

New algorithm reduces sample complexity for sparse linear regression.

The paper analyzes how Gaussian kernel parameters affect posterior covariance in Gaussian processes.

Covariance shrinkage via stochastic interpolation

We propose methods for estimating correspondence between two point sets under the presence of outliers in both the source and target sets. The proposed algorithms expand upon the theory of the regression without correspondence problem to estimate transformation coefficients using unordered multisets of covariates and r…

Gaussian and bootstrap methods improve ATE estimator accuracy.

Denise learns a function to quickly decompose covariance matrices robustly.

Optimization algorithms that leverage gradient covariance information, such as variants of natural gradient descent (Amari, 1998), offer the prospect of yielding more effective descent directions. For models with many parameters, the covariance matrix they are based on becomes gigantic, making them inapplicable in thei…

The paper develops time-uniform inference methods for stochastic approximation parameters.

EiGLasso speeds up sparse Kronecker-sum covariance estimation.

Paper analyzes holdout cross-validation for large non-Gaussian covariance estimation.

Novel Fréchet regression method handles errors-in-variables with low-rank covariates.

Efficiently selects important variables in high-dimensional logistic regression.

Mean Field Variational Bayes (MFVB) is a popular posterior approximation method due to its fast runtime on large-scale data sets. However, it is well known that a major failing of MFVB is its (sometimes severe) underestimates of the uncertainty of model variables and lack of information about model variable covariance.…

This work studies applications and generalizations of a simple estimation technique that provides exponential concentration under heavy-tailed distributions, assuming only bounded low-order moments. We show that the technique can be used for approximate minimization of smooth and strongly convex losses, and specificall…

This paper proposes a novel scheme for reduced-rank Gaussian process regression. The method is based on an approximate series expansion of the covariance function in terms of an eigenfunction expansion of the Laplace operator in a compact subset of $\mathbb{R}^d$. On this approximate eigenbasis the eigenvalues of the c…

Improved stability for large-scale Bayesian sampling.

PACE-GGM uses Gaussian mechanism for private covariance estimation.

Adaptive learning rate algorithms such as RMSProp are widely used for training deep neural networks. RMSProp offers efficient training since it uses first order gradients to approximate Hessian-based preconditioning. However, since the first order gradients include noise caused by stochastic optimization, the approxima…

Gaussian process (GP) models form a core part of probabilistic machine learning. Considerable research effort has been made into attacking three issues with GP models: how to compute efficiently when the number of data is large; how to approximate the posterior when the likelihood is not Gaussian and how to estimate co…

Study on friction forces for nonholonomic systems using affine connections.

Diagonal transformations preserve independence structures in non-Gaussian distributions.