Proposes a new method for PDF estimation using RBIG transforms.
problem Challenging task of multidimensional PDF estimation.
method Rotation-based Iterative Gaussianization (RBIG) transforms.
result RBIG transforms can be used to estimate unknown PDFs efficiently.
Unified DNN-based precoder for MIMO networks with multiple objectives.
problem Optimizing data transmission, energy harvesting, and security in MIMO networks.
method Rotation-based precoding and DNN for multi-objective optimization.
result DNN-based precoder reduces computational complexity and achieves near-optimal performance.
Securely share encrypted data for machine learning training.
problem Securely sharing encrypted data for machine learning training without revealing the data.
method Rotation based method using flow model.
result Theoretical justification and demonstration of effectiveness in different scenarios.
This dissertation advances scalable Gaussian processes using iterative methods and pathwise conditioning.
problem The classical Gaussian process formulation is not scalable for large datasets and modern hardware.
method Combining iterative methods and pathwise conditioning to improve scalability.
result Significantly reduced memory requirements and facilitated application to larger datasets.
New recommendations improve Gaussian process accuracy and stability.
problem Numerical instabilities and poor test likelihoods in iterative Gaussian process learning.
method Investigated CG tolerance, preconditioner rank, and Lanczos decomposition rank. Recommended small CG tolerance and large root decomposition size.
result L-BFGS-B optimizer achieves convergence with fewer gradient updates, improving Gaussian process accuracy.
Gaussianization flows transform any random vector into a Gaussian, enabling efficient computation and sample generation.
problem Transforming any random vector into a Gaussian for efficient computation and sample generation.
method Iterative Gaussianization and normalizing flow model.
result Gaussianization flows are universal approximators and achieve better performance on tabular datasets.
Warm starts improve Gaussian process regression by up to 16x.
problem Optimizing hyperparameters for Gaussian processes.
method Iterative Gaussian processes with warm start optimization.
result Warm starts achieve the same results as conventional methods but significantly speed up computations.
This paper approximates SA iterates using Gaussian distributions for tail bounds.
problem Characterizing the distribution of stochastic approximation iterates in finite time.
method Approximating pre-limit distributions of SA iterates by Gaussian sequences with recursively defined covariances.
result Explicit bounds on the Wasserstein-1 distance between rescaled iterates and Gaussians.
We propose a novel method for maximum likelihood-based parameter inference in nonlinear and/or non-Gaussian state space models. The method is an iterative procedure with three steps. At each iteration a particle filter is used to estimate the value of the log-likelihood function at the current parameter iterate. Using …
New iterative solvers speed up Gaussian process regression with derivatives.
problem Scaling Gaussian process regression with derivatives for high-dimensional problems and large budgets.
method Iterative solvers using fast matrix-vector multiplications and pivoted Cholesky preconditioning.
result Bayesian optimization with derivatives can now scale to high-dimensional problems and large evaluation budgets.
This paper speeds up iterative GP inference with warm starting.
problem Improving scalability of Gaussian process inference.
method Warm starting sequential posteriors using known solutions.
result Significant speed-ups and improved Bayesian optimisation performance.
Kernel Multigrid accelerates Back-fitting for additive Gaussian Processes.
problem Slow convergence of Back-fitting in training additive Gaussian Processes.
method Kernel Packets (KP) and Sparse Gaussian Process Regression (GPR) to enhance Back-fitting.
result Kernel Multigrid reduces the required iterations to O ( log n ) \mathcal{O}(\log n) O ( log n ) . Unified bounds for iterative algorithms with Gaussian data matrices.
problem Establishing non-asymptotic bounds for iterative algorithms with Gaussian data.
method Explicit coupling between iterates and Gaussian process with deterministic covariance.
result Tight, dimension-free bounds for generalized first-order methods.
This work uses QPGPs to improve ILC performance in repetitive tasks.
problem Performance degradation in repetitive motion tasks due to environmental changes and robot wear.
method Incorporates Quasi-Periodic Gaussian Processes into a predictive ILC framework.
result The proposed approach achieves faster convergence and robustness under disturbances.
Polynomial time algorithm matches correlated Gaussian matrices without vanishing correlation.
problem Matching vertices in two correlated Erdős-Rényi graphs.
method Iterative matching algorithm for correlated Gaussian Wigner matrices.
result First polynomial time algorithm for graph matching with arbitrarily small constant correlation.
The paper learns dependency graphs from non-Gaussian data.
problem Capturing dependency graphs from real data that may not be Gaussian.
method Additive over-parametrization with shrinkage to incorporate variable dependencies; iterative Gaussian graph learning algorithm.
result The estimators achieve satisfactory accuracy in measuring dependency structures.
New iterative methods improve scalability of Gaussian process approximations for large data.
problem Scalability issues in Gaussian process approximations for large spatial data.
method Iterative methods combined with preconditioners to reduce computational costs.
result Preconditioners accelerate convergence and improve predictive variances.
A new 1-iteration GMM learning algorithm improves robustness and accuracy.
problem Improving robustness and accuracy in Gaussian Mixture Model learning.
method GMM expansion idea, 1-iteration learning algorithm, theoretical proof of convergence.
result Guaranteed convergence of the new algorithm regardless initial parameters.
The paper uses Gaussian mixture models for Bayesian networks and proposes an optimization algorithm.
problem Modeling nodes in Bayesian networks with complex distributions.
method Gaussian mixture models combined with double iteration algorithm.
result The double iteration algorithm optimizes Gaussian mixture models effectively.
This paper refines the Gaussian Sinkhorn algorithm for general multivariate models.
problem Finite-dimensional solutions for general Gaussian multivariate models.
method Recursive formulation of the Sinkhorn algorithm for Gaussian models, including closed form expressions of entropic transport maps and Schrödinger bridges.
result Refined convergence analysis of Gaussian Sinkhorn algorithms.
Paper analyzes iterates in high-dimensional linear models and proposes estimators for their generalization error.
problem Analyzing iterates in high-dimensional linear models with comparable feature and sample sizes.
method Novel estimators for generalization error, debiasing corrections, and valid confidence intervals.
result Estimators are n \sqrt{n} n -consistent and can be used for early stopping. Paper refutes conjecture on tensor power iteration convergence in overcomplete models.
problem Understanding convergence of tensor power iteration in overcomplete random tensors.
method Analysis of tensor power iteration dynamics from random initialization.
result Polynomially many steps are necessary for convergence, refutes logarithmic conjecture.
Novel NGCA algorithm finds non-Gaussian subspace without iterative steps.
problem Identifying a linear subspace with non-Gaussian projected data.
method Log-density gradient estimation for eigenvalue decomposition.
result Identified subspace converges to true subspace at optimal rate.
An incremental SVDD algorithm for online data using Gaussian kernel.
problem Efficiently handling online or large data for SVDD.
method Incremental learning algorithm using Gaussian kernel, focusing on existing support vectors and new data points.
result Significant gains in efficiency with almost no loss in outlier detection accuracy or objective function value.
This work provides safety guarantees for iterative GP predictions.
problem Analytical intractability of uncertainty tracking in iterative GP predictions.
method Deriving formal probability error bounds for iterative GP predictions.
result Formal bounds ensure that GP trajectories lie within specified regions with high probability.
Improved generalization with iterate averaging and adaptive algorithms.
problem Enhancing model generalization performance in high-dimensional settings.
method Gaussian process perturbation model, combine IA with large learning rates and regularisation, less frequent averaging, adaptive gradient methods.
result Iterate averaging with adaptive algorithms significantly outperforms SGD on various datasets.
New method speeds up Gaussian process training and inference for large datasets.
problem Training and inference in Gaussian processes are computationally expensive for large datasets.
method Iterative alternating projection method that accesses subblocks of the kernel matrix, reducing time and space complexity.
result Empirically, the method accelerates GP training and inference by up to 72x compared to conjugate gradients.
Efficiently differentiate functions of large matrices using new adjoint systems.
problem Differentiating functions of large matrices in scientific and probabilistic machine learning models.
method Deriving and implementing new adjoint systems for Lanczos and Arnoldi iterations in JAX.
result Efficient differentiation of PDEs, Gaussian process models, and Bayesian neural networks.
FPI methods compute barycenters of Gaussian sets for various dissimilarity measures.
problem Efficiently compute barycenters of Gaussian sets for multiple dissimilarity measures.
method Fixed-Point Iterations (FPI) for several dissimilarity measures.
result FPI provides a useful toolbox for fusion/reduction of Gaussian sets.
Proposes momentum methods for Lie groups, improving on classical algorithms.
problem Optimization on nonlinear spaces, especially Lie groups.
method Generalizes Nesterov's Accelerated Gradient method to Lie groups.
result Demonstrates faster convergence for NAG-like methods on Lie groups.
The paper speeds up hyperparameter optimisation in Gaussian processes.
problem Scaling hyperparameter optimisation to large datasets.
method Improvements to linear system solvers (pathwise gradient, warm starting, early stopping).
result Speed-ups of up to 72x and residual norm decreases of up to 7x.
SWAG improves deep learning uncertainty with a simple, scalable method.
problem Improving uncertainty estimation in deep learning models.
method SWAG uses stochastic weight averaging to fit a Gaussian distribution over neural network weights.
result SWAG approximates the true posterior and performs well on various tasks.
Robust PCA reduces to power iterations for outlier-resilient feature extraction.
problem Sensitivity of PCA to non-Gaussian samples and outliers.
method Robust formulation of PCA based on maximum correntropy criterion.
result MCPI reduces to power iterations, making PCA more robust to outliers.
Warm-start strategies speed up GP inference by 19x.
problem Efficient sequential inference in Gaussian processes.
method Three warm-start strategies exploiting smaller linear systems.
result Warm-starting achieves up to 19x speed-up in convergence.
New method improves Gaussian Mixture Model fitting speed.
problem Slow convergence of EM algorithm for complex data.
method Riemannian Newton Trust-Region method for Gaussian Mixture Models.
result Outperforms existing methods in runtime and iterations.
New iterative methods improve Vecchia-Laplace approximations for large data sets.
problem Inaccurate and slow Vecchia-Laplace approximations for large data sets.
method Iterative methods to improve Vecchia-Laplace approximations, including preconditioners and novel methods for predictive variances.
result Order of magnitude speed-up and threefold increase in prediction accuracy compared to state-of-the-art methods.
Scalable Gaussian processes with latent Kronecker structure for large datasets.
problem Limited scalability of Gaussian processes for large datasets.
method Leveraging latent Kronecker structure, projecting kernel matrix onto latent Kronecker product, using iterative linear system solvers and pathwise conditioning.
result Outperforms state-of-the-art sparse and variational GPs on real-world datasets with up to five million examples.
Improved privacy-preserving linear regression via iterative Hessian mixing.
problem Differentially private linear regression with improved accuracy and efficiency.
method Iterative Hessian Mixing (IHM) for differentially private ordinary least squares (DP-OLS).
result IHM provides better utility guarantees and outperforms AdaSSP in empirical evaluations.
The paper improves boundary detection and density estimation on noisy data.
problem Detecting boundary points and estimating density on noisy data from compact manifolds.
method Doubly stochastic scaling of the Gaussian heat kernel via Sinkhorn iterations.
result The new estimates of boundary points and density outperform standard methods, especially under noise.
Study efficient iterative method for distribution matching using sliced optimal transport.
problem Efficiently match distributions using sliced optimal transport.
method Slice-matching scheme based on sliced optimal transport, with quantitative non-asymptotic rates derived.
result Derive quantitative non-asymptotic rates for convergence to target distribution.
EM algorithm quickly finds near-optimal parameters in Gaussian mixtures.
problem Parameter estimation in symmetric two-component Gaussian mixtures.
method Randomly initialized EM algorithm, analyzing convergence rate and optimality.
result EM algorithm converges in O ( n ) O(\sqrt{n}) O ( n ) iterations with high probability. A new robust GP regression algorithm that trims outliers improves model accuracy.
problem Severe bias in GP regression due to data contamination by outliers.
method Iterative trimming of extreme data points.
result Significantly outperforms standard and robust GP variants in most test cases.
Paper develops Gaussian approximations and bootstrap for federated LSA with trade-off bounds.
problem Analyzing convergence rates and trade-offs in federated linear stochastic approximation.
method Established Berry-Esseen-type bounds for federated LSA, developed multiplier bootstrap for inference.
result First federated Gaussian approximations with explicit trade-off terms and non-asymptotic validity guarantees.
A fast Modal EM algorithm for Gaussian mixtures.
problem Clustering with Gaussian mixtures.
method Modal EM algorithm for Gaussian mixtures.
result High flexibility in various clustering contexts.
New CAGP-GS framework improves Gaussian process uncertainty quantification.
problem Scaling issue in Gaussian processes for large datasets.
method Calibrated probabilistic linear solvers for reduced complexity.
result CAGP-GS framework provides more realistic uncertainty quantification.
Rotates MFVI for better Gaussian approximations.
problem Improving variational approximations for complex distributions.
method Rotated coordinate system, PCA-based rotation, iterative Gaussianization.
result Significantly more accurate approximations with lower computational cost.
We present a numerical model for the dynamics of thin viscous threads based on a discrete, Lagrangian formulation of the smooth equations. The model makes use of a condensed set of coordinates, called the centerline/spin representation: the kinematical constraints linking the centerline's tangent to the orientation of …
ELU algorithm improves on EM for over-specified Gaussian mixtures.
problem Slow convergence of EM in over-specified Gaussian mixtures.
method Developed ELU algorithm for two-component mixtures, combining exponential location update and gradient descent.
result ELU converges to final statistical radius after logarithmic iterations, resolving open question.