A new BO method adapts hyperparameters online and uses a novel kernel for global and local optimization.
problem Expensive black-box optimization problems.
method Online length-scale adaption, mixed-global-local kernel, and adaptive hyperparameters.
result The proposed method outperforms state-of-the-art BO methods on global optimization benchmarks.
The Efficient Global Optimization (EGO) algorithm uses a conditional Gaus-sian Process (GP) to approximate an objective function known at a finite number of observation points and sequentially adds new points which maximize the Expected Improvement criterion according to the GP. The important factor that controls the e…
New algorithm reduces regret bounds for Bayesian optimization with unknown hyperparameters.
problem Optimizing black-box functions with unknown hyperparameters, especially length scale.
method Length Scale Balancing (LB) - aggregating multiple surrogate models with varying length scales.
result LB achieves a regret bound only logaritically away from the oracle algorithm.
Much recent work has concerned sparse approximations to speed up the Gaussian process regression from the unfavorable O(n3) scaling in computational time to O(nm2). Thus far, work has concentrated on models with one covariance function. However, in many practical situations additive models with multiple covariance func…
ESS improves MCMC efficiency for correlated & multimodal distributions.
problem Slice Sampling's sensitivity to initial length scale and difficulty with correlated distributions.
method Adaptive tuning and parallel walkers for efficient sampling.
result ESS improves efficiency by more than an order of magnitude on correlated distributions.
Wrinkles form on a thin sheet bonded to a sphere, revealing energy and length scale behaviors.
problem Understanding the wrinkle formation on a thin sheet bonded to a sphere.
method Analyzing the energy of the system with the sheet's thickness as a small parameter, determining leading and next-order behaviors.
result The wrinkling pattern varies with radius, with the number of wrinkles being approximately integer multiples of the sheet thickness.
The question of how best to estimate a continuous probability density from finite data is an intriguing open problem at the interface of statistics and physics. Previous work has argued that this problem can be addressed in a natural way using methods from statistical field theory. Here I describe new results that allo…
The study analyzes convergence of adaptive optimizers under low-precision training.
problem Understanding why low-precision training remains effective for large models.
method Developed a theoretical framework for analyzing convergence of adaptive optimizers under floating-point quantization.
result Adaptive optimizers retain convergence rates close to full-precision methods under logarithmic mantissa scaling.
DGCP uses deep neural networks to optimize hyperparameters for Gaussian processes.
problem Optimizing hyperparameters for Gaussian processes in non-uniform input spaces.
method DGCP uses a deep neural network to learn hyperparameters of non-stationary covariance functions.
result DGCP improves Gaussian process predictions in locally adapted or sparse input spaces.
Study on geodesics on high genus expander surfaces, proving filling and non-simple properties.
problem Properties of geodesics on expander surfaces of high genus.
method Adapting Margulis' counting strategy to low length scales.
result Almost every geodesic of certain lengths is filling or non-simple.
Designing a covariance function that represents the underlying correlation is a crucial step in modeling complex natural systems, such as climate models. Geospatial datasets at a global scale usually suffer from non-stationarity and non-uniformly smooth spatial boundaries. A Gaussian process regression using a non-stat…
New kernel interprets 3D anisotropic data with rotations and improved predictions.
problem Capturing rotated anisotropy in 3D spatial fields.
method Introduces a Lie-algebraic kernel with three principal length-scales and an explicit rotation.
result Posterior recovers rotated anisotropy and improves prediction over axis-aligned kernels.
Accurate approximations to density functionals have recently been obtained via machine learning (ML). By applying ML to a simple function of one variable without any random sampling, we extract the qualitative dependence of errors on hyperparameters. We find universal features of the behavior in extreme limits, includi…
The paper presents a method to infer unknown forcing functions in differential equations using Gaussian processes and adjoints.
problem Inferring unknown forcing functions in differential equations from noisy observations.
method Using adjoint methods to efficiently infer Gaussian process (GP) driven differential equations, with truncated basis expansions of the GP kernel.
result Efficient Bayesian inference of forcing functions modeled as GPs using adjoints, with lower computation than MCMC methods.
This work addresses two main issues of the standard Kernel Entropy Component Analysis (KECA) algorithm: the optimization of the kernel decomposition and the optimization of the Gaussian kernel parameter. KECA roughly reduces to a sorting of the importance of kernel eigenvectors by entropy instead of by variance as in K…
Learning the distribution of natural images is one of the hardest and most important problems in machine learning. The problem remains open, because the enormous complexity of the structures in natural images spans all length scales. We break down the complexity of the problem and show that the hierarchy of structures …
We introduce a new regression framework, Gaussian process regression networks (GPRN), which combines the structural properties of Bayesian neural networks with the non-parametric flexibility of Gaussian processes. This model accommodates input dependent signal and noise correlations between multiple response variables,…
MKA improves Gaussian process regression for large datasets.
problem Gaussian process regression struggles with large datasets.
method MKA is a memory-efficient, direct kernel approximation method.
result MKA achieves better performance with small kernel length scales.
The theory of quantum computation can be constructed from the abstract study of anyonic systems. In mathematical terms, these are unitary topological modular functors. They underlie the Jones polynomial and arise in Witten-Chern-Simons theory. The braiding and fusion of anyonic excitations in quantum Hall electron liqu…
Temporal Normalizing Flows enhance density estimation of time-dependent data.
problem Accurate and robust density estimation of time-dependent stochastic data.
method Leveraging normalizing flows for temporal data, tNFs estimate multi-scale distributions without prior scale knowledge.
result Temporal Normalizing Flows improve density estimation of time-dependent data, including multi-scale distributions.
Proposes a new model for EHR data using time-dependent Gaussian processes.
problem Joint modeling of multiple clinical variables over time.
method Multivariate nonstationary Gaussian processes with time-varying parameters and posterior inference via HMC.
result The proposed model outperforms stationary models and reveals latent correlations predictive of patient risk.
A new method for accurately reconstructing signals without knowing the kernel or signal regularity.
problem Recovering signals from noisy measurements without prior knowledge of the convolution kernel or signal regularity.
method Parametrizing the convolution kernel and prior length-scales, jointly estimated in the inversion procedure.
result Accurate reconstructions of signals with varying regularity and unknown kernel size.
The shape equation and linking conditions for a vesicle with two-phase domains are derived. We refine the conjecture on the general neck condition for the limit shape of a budding vesicle proposed by Jülicher and Lipowsky [Phys. Rev. Lett. \textbf{70}, 2964 (1993); Phys. Rev. E \textbf{53}, 2670 (1996)], and then we us…
New methods rank variables for Gaussian processes better than automatic relevance determination.
problem Variable selection for Gaussian process models using inverse length-scale parameters has limitations.
method Two novel methods rank variables based on their predictive relevance using posterior predictive distribution predictions.
result Improved variable selection compared to automatic relevance determination in terms of variability and predictive performance.
The paper extends NSGPs with L1-regularization for sparsity and solves the resulting R-NSGP regression problem.
problem Sparsity in non-stationary temporal data.
method Developed an ADMM-based method for solving the regularized NSGP regression problem.
result The proposed methods induce sparsity in the parameters of NSGPs.
Paper proposes NOSTILL-GP for accurate space-time modeling in environmental monitoring.
problem Accurate modeling of space-time dynamics in environmental phenomena.
method NOSTILL-GP - a non-stationary, spatio-temporal Gaussian Process model with efficient training strategies.
result Demonstrates the effectiveness and general applicability of NOSTILL-GP for environmental monitoring.
We present techniques for effective Gaussian process (GP) modelling of multiple short time series. These problems are common when applying GP models independently to each gene in a gene expression time series data set. Such sets typically contain very few time points. Naive application of common GP modelling techniques…
Revises Gauss's Lemma using metrical distortion and differential slip.
problem Revising Gauss's Lemma in Riemannian geometry.
method Defining metrical distortion and differential slip, showing their geometric implications.
result Geodesically radial volume and length preservation properties.
The Gaussian process latent variable model (GP-LVM) is a popular approach to non-linear probabilistic dimensionality reduction. One design choice for the model is the number of latent variables. We present a spike and slab prior for the GP-LVM and propose an efficient variational inference procedure that gives a lower …
Machine learning improves polymer design accuracy.
problem Designing polymers with desired phase behavior in disordered systems.
method Inverse design via machine learning, including gradient boosting with decision trees and particle-swarm optimization.
result High-accuracy tuning of poly(2-oxazoline) cloud point with RMSE of 4 °C.
Paper introduces a new, tractable measure of model complexity.
problem Need for a reliable measure of model complexity.
method Mathematically rigorous measure based on gradient similarities.
result Generalizes to various model types and insights into double descent.
Physics-informed GANs model subsurface flow at the Hanford Site.
problem Uncertainty quantification for subsurface flow modeling at the Hanford Site.
method Physics-informed GANs, hierarchical domain parallelism, multiple GPUs, efficient communication.
result Highly scalable physics-informed GANs model subsurface flow on Summit supercomputer.
Model shows advantageous position in trade networks leads to success.
problem Characterizing the importance of nodes in trade networks during globalization.
method Mapped trade network evolution to percolation problem and analyzed topological features.
result Advantageous position at different scales determines node success.
Prior distributions of binarized natural images are learned by using a Boltzmann machine. According the results of this study, there emerges a structure with two sublattices in the interactions, and the nearest-neighbor and next-nearest-neighbor interactions correspondingly take two discriminative values, which reflect…
Paper analyzes the phase retrieval problem in X-ray imaging.
problem Phase retrieval problem in X-ray imaging of amorphous samples.
method Analysis of well-posedness and development of experimental protocols.
result The phase retrieval problem is generally ill-posed.
Diffusion models reveal latent hierarchical structure in data.
problem Quantitative measurement of data's hierarchical structure.
method Forward-backward experiments in diffusion models.
result Changes in latent variables manifest as correlated chunks in data.
Bayesian optimization guided by experimenter intuition and beliefs.
problem Finding optimal functions with experimenter's beliefs incorporated.
method Sequential Subspace Search using Gaussian Process.
result Algorithm converges in sub-linear time with finite effective dimension.
Study the geometric structure of graph Laplacian embeddings for manifold data.
problem Identifying coarse structure in manifold data sampled from a mixture model.
method Analyze spectral clustering procedure for data sampled from a manifold, focusing on graph Laplacian embeddings.
result Embedded data concentrates on cones centered around orthogonal vectors when the mixture model is well-separated.
New algorithm improves Gaussian process hyperparameter tuning for large datasets.
problem Scalable hyperparameter tuning for Gaussian processes on large datasets.
method Estimates smoothness and length-scale parameters in Matern kernel using novel loss functions.
result Improved uncertainty quantification over traditional methods.
Delayed rejection HMC improves sampling efficiency for multiscale distributions.
problem Hamiltonian Monte Carlo struggles with wide-ranging distributions, especially in high-curvature areas.
method Introduces a delayed rejection variant of HMC, using geometrically smaller step sizes for retries.
result Up to five-fold performance gains in effective sample size per gradient evaluation.
Deep learning improves model discovery from sparse sensor data.
problem Improving physical understanding and predictions from coarse, non-grid sampled data.
method Physics-informed neural networks and automatic differentiation.
result Deep learning can recover underlying equations from sparse, non-grid data.
Improved Gaussian Process regression using TQFF over RFF and Gaussian QFF.
problem Limited performance of Quadrature Fourier Features (QFF) in approximating highly oscillatory functions.
method Developed Trigonometric Quadrature Fourier Features (TQFF) using a novel non-Gaussian quadrature rule.
result TQFF provides better approximation accuracy and fewer features compared to RFF and Gaussian QFF.
Deep learning is a broad set of techniques that uses multiple layers of representation to automatically learn relevant features directly from structured data. Recently, such techniques have yielded record-breaking results on a diverse set of difficult machine learning tasks in computer vision, speech recognition, and n…
Bayesian optimization has proven to be a highly effective methodology for the global optimization of unknown, expensive and multimodal functions. The ability to accurately model distributions over functions is critical to the effectiveness of Bayesian optimization. Although Gaussian processes provide a flexible prior o…
The paper develops a state-space approach to deep Gaussian processes for efficient state estimation.
problem Efficient regression and state estimation for deep Gaussian processes.
method Hierarchical transformed Gaussian process priors, state-space representation, linear stochastic differential equations, sequential methods.
result The state-space approach enables efficient state estimation and regression for deep Gaussian processes.
A new method for faster bandwidth selection in Gaussian kernel ridge regression.
problem Efficiently selecting the bandwidth in Gaussian kernel ridge regression.
method Formulated an approximate Jacobian expression for bandwidth selection, proposing a closed-form heuristic.
result Our method is as accurate as cross-validation and marginal likelihood maximization but up to six orders of magnitude faster.
A hybrid method combines model-based and data-driven approaches for multiscale constitutive responses.
problem High computational costs and inaccuracies in nonlinear multiscale methods.
method Hybrid methodology combining model-based constitutive laws, data-driven corrections, and computational multiscale approaches.
result Model-data-driven approach improves macroscale simulations with similar accuracy and computational cost.
Deep learning solves wave-based inverse problems, including super-resolution imaging.
problem Solving inverse wave scattering problems across all length scales.
method Wide-band butterfly network coupled with dynamic noise injection.
result Framework successfully solves super-resolution imaging problems.