Study predictive performance of linear regression with random functional covariates.
problem Theoretical predictive performance of linear regression with random functional covariates.
method Theoretical analysis of ridge and ridge-less least-squares regression with random functional covariates.
result Probabilistic bounds on predictive excess risk for random functional covariates.
Efficiently estimates covariance for sparse functional data.
problem Sparse data in functional analysis.
method Random-knots and B-spline estimators for covariance function.
result Asymptotic pointwise covariance estimates for sparsified data.
Study on neural network initialization with shaped infinite depth-and-width networks.
problem Understanding the distribution of random covariance matrices in shaped infinite-depth-and-width networks.
method Introduced the Neural Covariance SDE to model the distribution of the random covariance matrix.
result Identified the precise scaling of the activation function necessary for a non-trivial limit.
Method uses random forest with distance covariance for transfer learning in healthcare.
problem Transfer learning in random forests with sparse differences between source and target.
method Distance covariance-based feature weights in residual random forest.
result Upper bound on mean square error rate for transfer learning in RF.
Standard sparse pseudo-input approximations to the Gaussian process (GP) cannot handle complex functions well. Sparse spectrum alternatives attempt to answer this but are known to over-fit. We suggest the use of variational inference for the sparse spectrum approximation to avoid both issues. We model the covariance fu…
Bayesian methods estimate regression functions on submanifolds using graph Laplacian eigenbasis.
problem Estimating regression functions on unknown smooth submanifolds.
method Random geometric graph structure, Bayesian priors based on random basis expansion in graph Laplacian eigenbasis.
result Posterior contraction rates are minimax optimal for any positive smoothness index.
New methods estimate survival functions with time-varying covariates.
problem Estimating survival functions with time-varying covariates.
method Generalized conditional inference and relative risk forests, adapted transformation forest.
result Proposed methods outperform traditional models in estimating survival functions.
Covariance is shown as a commutator in random variable calculus.
problem Expressing covariance as a commutator of operators.
method Demonstrated through commutator identities involving expectations and products of functions.
result Revealed the underlying Lie algebraic structure in efficient influence curve calculus.
Generalizes randomized SVD for better matrix approximations using Gaussian vectors.
problem Computing accurate rank-k approximations of matrices with limited data.
method Extends randomized SVD to multivariate Gaussian vectors, incorporating prior knowledge and using Gaussian processes.
result Demonstrates improved accuracy in approximating matrices and Hilbert-Schmidt operators.
A latent force model is a Gaussian process with a covariance function inspired by a differential operator. Such covariance function is obtained by performing convolution integrals between Green's functions associated to the differential operators, and covariance functions associated to latent functions. In the classica…
GBMixed boosts mixed models for clustered data, estimating mean and variance flexibly.
problem Flexible estimation of mean and variance components in clustered data.
method Gradient Boosting framework for linear mixed models with likelihood-based gradients.
result GBMixed accurately recovers complex nonlinear fixed effects and covariances.
CovRegRF estimates covariance matrix from covariates using random forests.
problem Estimating conditional covariances or correlations among multivariate responses.
method Random forest trees with a custom splitting rule to maximize covariance difference.
result Accurate covariance matrix estimates and controlled Type-1 error.
The Matérn covariance function is a popular choice for prediction in spatial statistics and uncertainty quantification literature. A key benefit of the Matérn class is that it is possible to get precise control over the degree of mean-square differentiability of the random process. However, the Matérn class possesses e…
The thesis presents a new perspective on high-dimensional optimization.
problem The failure point of classical optimization methods in high dimensions.
method A distributional view of optimization, focusing on random objective functions and Bayesian Optimization.
result The distributional view explains predictable progress in high-dimensional optimization and provides insights into optimal step size control.
Paper introduces a novel method for dynamic covariance estimation with random forests.
problem Estimating high-dimensional dynamic covariance matrices with multiple covariates.
method Nonparametric approach using random forests.
result Uniform consistency theory and error rates established for high-dimensional scenarios.
Develops RF-GLS for binary geospatial data.
problem Challenges in extending RF to binary geospatial data.
method Proposes RF-GLS for binary data, embedding it in generalized mixed effects models.
result Establishes consistency of RF-GP for mean function and covariate effect estimation.
Random forest performance depends on SNR and covariate characteristics.
problem Understanding when random forests perform well.
method Systematic analysis of out-of-sample MSE for different SNR scenarios.
result Randomization effectiveness depends on SNR and covariate characteristics.
Study reveals three limiting regimes for neural network functionals.
problem Understanding the behavior of functionals of random neural networks.
method Central and non-central limit theorems, Hermite expansions, Diagram Formula, Stein-Malliavin techniques.
result Three distinct limiting regimes based on fixed points of covariance function.
Extends Gaussian process theory to Banach spaces.
problem Extending Gaussian process theory to Banach spaces.
method Investigates the connection between Gaussian processes and Gaussian random elements in reproducing kernel Banach spaces.
result Characterizes positive definite functions that arise from covariance operators in Banach space setting.
Selective inference for group lasso estimators across various distributions and covariates.
problem Developing selective inference methods for group lasso estimators.
method Randomized group-regularized optimization problem with post-selection likelihood.
result Selective point estimator and Wald-type confidence regions for regression parameters.
Iterative methods for fitting a Gaussian Random Field (GRF) model via maximum likelihood (ML) estimation requires solving a nonconvex optimization problem. The problem is aggravated for anisotropic GRFs where the number of covariance function parameters increases with the dimension. Even evaluation of the likelihood fu…
Develops a forward variable selection method for interpretable random forest models.
problem Interpreting high-dimensional non-parametric models like random forests.
method Forward variable selection using CRPS as loss function, with hypothesis testing at each step.
result Method selects a smaller set of variables that optimizes predictive performance.
Paper develops online statistical inference methods for stochastic optimization using Kiefer-Wolfowitz algorithms.
problem Online statistical inference of model parameters in stochastic optimization problems.
method Kiefer-Wolfowitz algorithm with random search directions, asymptotic distribution analysis.
result Developed valid confidence intervals for online statistical inference.
Study on random matrices in deep neural networks with IID entries.
problem Distribution of singular values in product of random matrices for deep neural networks.
method Random matrix theory with a streamlined approach for non-Gaussian data.
result Generalization of macroscopic universality property to non-Gaussian data.
Develops robust methods for infinite-dimensional stochastic processes.
problem Measuring covariations in stochastic evolution equations in infinite dimensions.
method Asymptotic theory for jump robust measurement of covariations.
result Identifies scaling limits for realized covariations.
Study improves error bounds for sparse regression with heavy-tailed covariates.
problem Estimating sparse coefficients in linear regression with heavy-tailed covariates.
method Employed an ℓ1-penalized Huber regression method. result Error bound identical to Gaussian case for L-subexponential covariates. An algorithm finds optimal covariates for blocking in randomized experiments.
problem Minimizing variance in causal effect estimates from heterogeneous data.
method Using causal graphs, an algorithm identifies optimal covariates for blocking.
result An efficient algorithm reduces variance in causal effect estimates.
We introduce a new method to explain Gaussian processes using Shapley values.
problem Explaining the uncertainty in Gaussian process models.
method Extending Shapley values to stochastic cooperative games for Gaussian processes.
result Our method generates explanations that are random variables and satisfy favorable axioms.
Develops a new random forest method for clustered data with improved prediction and inference.
problem Improving prediction and inference accuracy for clustered data with within-cluster dependence.
method Clustered Random Forests, using weighted least squares estimators for leaf predictions.
result Optimal prediction and inference weights vary under covariate shift, necessitating user-chosen weights.
The paper proposes a method to estimate treatment effects using CAR designs with additional covariates.
problem Estimating distributional treatment effects in CAR designs with additional covariates.
method Flexible distribution regression framework that incorporates additional covariates using machine learning methods.
result The proposed estimator attains the semiparametric efficiency bound for distributional treatment effects under CAR.
Introduces intrinsic Riemannian cross-covariance for manifold-valued random objects.
problem Covariance estimation for random objects on Riemannian manifolds.
method Defines covariance and correlation via parallel transport.
result Proposed covariance is independent of coordinate choices.
LoCoV reduces portfolio optimization errors from sample covariance matrices.
problem Large errors in sample covariance matrix for optimal portfolio weights.
method LoCoV (low dimension covariance voting) algorithm to reduce these errors.
result LoCoV outperforms classical methods in portfolio optimization experiments.
Deep Gaussian Processes are reinterpreted as deep trigonometric networks for tractable inference.
problem Challenging inference in DGPs due to intractable marginalization in latent function space.
method Viewing DGPs as deep trigonometric networks with Bochner's theorem, and using the wide limit with a bottleneck to translate DGPs into deep trigonometric networks.
result The weight space view yields the same effective covariance functions as obtained in function space, and varying prior distributions over network parameters is equivalent to employing different kernels.
The random matrix theory method of planar Gaussian diagrammatic expansion is applied to find the mean spectral density of the Hermitian equal-time and non-Hermitian time-lagged cross-covariance estimators, firstly in the form of master equations for the most general multivariate Gaussian system, secondly for seven part…
Although there is a rich literature on methods for allowing the variance in a univariate regression model to vary with predictors, time and other factors, relatively little has been done in the multivariate case. Our focus is on developing a class of nonparametric covariance regression models, which allow an unknown p …
The paper deals with distribution of singular values of product of random matrices arising in the analysis of deep neural networks. The matrices resemble the product analogs of the sample covariance matrices, however, an important difference is that the population covariance matrices, which are assumed to be non-random…
Study on estimating distances between covariance operators and Gaussian processes.
problem Estimating distances between covariance operators and Gaussian processes.
method Riemannian distances, concentration results for Hilbert space-valued random variables, RKHS covariance and cross-covariance operators.
result Both distances converge in the Hilbert-Schmidt norm and can be consistently and efficiently estimated.
This study approximates distances between Gaussian processes and covariance operators using RKHS.
problem Approximating distances between Gaussian processes and covariance operators from finite samples.
method Using reproducing kernel Hilbert space (RKHS) covariance and cross-covariance operators, the study shows how to consistently and efficiently estimate Sinkhorn divergence from finite samples.
result Convergence rates are dimension-independent and of the same order as Hilbert-Schmidt distance.
We study covariance matrix estimation for the case of partially observed random vectors, where different samples contain different subsets of vector coordinates. Each observation is the product of the variable of interest with a 0−1 Bernoulli random variable. We analyze an unbiased covariance estimator under this mod…
New tree and forest methods use oblique splits for better risk bounds.
problem Improving risk bounds for regression algorithms.
method Randomized decision trees and forests with oblique splits.
result Oblique splits lead to better risk bounds for multi-index models.
New method for inference on covariates in NMF with random effects.
problem Formal inference for covariate effects in NMF with non-negativity constraints.
method NMF-RE model with random effects, ridge updates, df-based cap, asymptotic linearization, wild bootstrap.
result Valid inference on covariates with non-negativity constraint, avoiding degeneracy.
This paper improves computational efficiency in kernel ridge regression under covariate shift.
problem Covariate shift in nonparametric regression.
method Random projections in RKHS to reduce computational demands.
result Significant computational savings can be achieved without compromising learning performance under covariate shift.
This study improves estimation of the first principal component in multivariate functional data.
problem Estimating the first principal component of multivariate random processes.
method Defined covariance functions and operators, introduced LASSO optimization, and established minimax lower bounds.
result The method provides an optimal variance in the minimax sense for estimating eigenelements.
Two new covariance estimators for ROOT-SGD improve statistical inference.
problem Uncertainty measurement for ROOT-SGD's normal distribution estimator.
method Developed two covariance estimators: plug-in and Hessian-free.
result Hessian-free estimator is asymptotically consistent and Hessian-free.
Variable selection in sparse regression models is an important task as applications ranging from biomedical research to econometrics have shown. Especially for higher dimensional regression problems, for which the link function between response and covariates cannot be directly detected, the selection of informative va…
DARTS optimizes covariate selection in trials with limited data.
problem Limited budget for high-dimensional pretreatment data.
method Dynamic Adaptive Rerandomization via Thompson Sampling (DARTS).
result DARTS efficiently concentrates budget on informative features.
Relying on recent advances in statistical estimation of covariance distances based on random matrix theory, this article proposes an improved covariance and precision matrix estimation for a wide family of metrics. The method is shown to largely outperform the sample covariance matrix estimate and to compete with state…
New insights into how high-dimensional models handle covariate shifts.
problem Covariate shift in high-dimensional random feature regression.
method Exact high-dimensional asymptotics of random feature regression under covariate shift.
result Overparameterized models exhibit enhanced robustness to covariate shift.