Model predicts composite structures assembly quality with input uncertainty.
problem Accurate prediction of dimensional deviations and residual stress in composite structures assembly.
method Neural Network Gaussian Process considering input uncertainty.
result NNGPIU model outperforms other methods for nonsmooth, nonlinear responses.
We consider reinforcement learning in input-driven environments, where an exogenous, stochastic input process affects the dynamics of the system. Input processes arise in many applications, including queuing systems, robotics control with disturbances, and object tracking. Since the state dynamics and rewards depend on…
Quantum Process Tomography (QPT) methods aim at identifying, i.e. estimating, a given quantum process. QPT is a major quantum information processing tool, since it especially allows one to characterize the actual behavior of quantum gates, which are the building blocks of quantum computers. However, usual QPT procedure…
Simplified DGPs training by fixing inducing inputs to subset of data.
problem Challenging training of deep Gaussian processes.
method Fixed subset of data for inducing inputs, variational sampling.
result Significant reduction in trainable parameters and computation cost without performance degradation.
Safe active learning for time-series models with Gaussian processes.
problem Learning time-series models while respecting safety constraints.
method Employing Gaussian processes with a nonlinear exogenous input structure, the approach dynamically explores the input space to generate data for model learning.
result The approach effectively learns time-series models under safety constraints, as demonstrated in a technical application.
ETC improves Transformer models for long and structured inputs.
problem Scaling input length and encoding structured inputs in Transformers.
method Introduces global-local attention, relative position encodings, and CPC pre-training.
result Achieves state-of-the-art results on four natural language datasets.
Bayesian Gaussian process models handle uncertain data locations in PDE approximations.
problem Handling uncertainties in data locations for PDE approximations.
method Bayesian inference of uncertain inputs integrated into Gaussian process predictions.
result Substantial reduction in predictive uncertainties achieved through Bayesian inference.
Natural Language Processing models help encode categorical process inputs.
problem Encoding categorical variables in industrial process modeling.
method Using NLP models for categorical variable encoding, combined with dimensionality reduction.
result Meaningful embeddings of categorical variables improve feature importance.
This study analyzes how RNNs process context in sentiment analysis.
problem Understanding how recurrent neural networks process context in sentiment analysis.
method Developed methods to reverse engineer RNNs, identifying contextual effects and quantifying their strength and timescale.
result Identified inputs that induce contextual effects and quantified their properties.
Following great success in the image processing field, the idea of adversarial training has been applied to tasks in the natural language processing (NLP) field. One promising approach directly applies adversarial training developed in the image processing field to the input word embedding space instead of the discrete…
Random projections improve GP regression performance, reducing high-dimensional inputs to 1D.
problem Gaussian processes struggle with high-dimensional inputs, leading to overfitting and high computational cost.
method Use additive sums of kernels operating on random projections of inputs.
result Predictive performance converges to full-dimensional kernel performance with increasing projections, even in 1D.
A dynamical system can be regarded as an information processing apparatus that encodes input streams from the external environment to its state and processes them through state transitions. The information processing capacity (IPC) is an excellent tool that comprehensively evaluates these processed inputs, providing de…
Framework optimizes multiple objectives considering input uncertainty.
problem Efficiently optimizing multiple objectives with input uncertainty.
method Robust Gaussian Process model and two-stage Bayesian optimization process.
result Found a robust Pareto frontier considering input uncertainty.
WE constructs GP kernels for mixed inputs using weighted EDMs.
problem Limitation of standard GP models in handling categorical variables.
method WEGP constructs kernel function using weighted EDMs for categorical inputs.
result WEGP improves GP model accuracy in both synthetic and real-world optimization problems.
A new method for Gaussian Processes handles mixed continuous and categorical inputs.
problem Modeling cross-correlations between continuous and categorical data.
method Low-Rank Correlation (LRC) method for Gaussian Processes with flexible rank approximation.
result LRC outperforms existing methods in estimating cross-correlations and predicting response surfaces.
Kernel improves Gaussian process scalability for wide datasets.
problem Under-performance of Gaussian processes on wide data.
method Introduces Bézier Gaussian Process kernel with exponential summarising variables growth and linear cost.
result Empirically demonstrates scalability to both tall and wide datasets.
Efficiently processes dynamic inputs in AI writing assistants with incremental computation.
problem Efficiently updating AI models in real-time with dynamic inputs.
method Incremental computing using vector quantization to filter and reuse intermediate values in neural networks.
result Comparable accuracy with 12.1X fewer operations for processing dynamic inputs.
Paper introduces a new model to handle multi-task learning across different input domains.
problem Learning correlated tasks across varying input domains.
method Develops a novel heterogeneous stochastic variational linear model of coregionalization (HSVLMC) for multi-task learning.
result The proposed model outperforms existing models in diverse multi-task scenarios.
Proposes GPHMEs using Gaussian processes for hierarchical expert models.
problem Hierarchical mixtures of experts with complex gating functions.
method Gaussian process-gated hierarchical mixtures of experts (GPHMEs) with non-linear gating and expert functions.
result Outperforms tree-based HMEs and achieves good performance with reduced complexity.
We propose a novel deep learning paradigm of differential flows that learn a stochastic differential equation transformations of inputs prior to a standard classification or regression function. The key property of differential Gaussian processes is the warping of inputs through infinitely deep, but infinitesimal, diff…
AIRBO optimizes robustly under uncertain inputs.
problem Challenges in Bayesian Optimization due to input uncertainty.
method AIRBO models arbitrary input uncertainty with MMD and accelerates posterior inference.
result AIRBO achieves state-of-the-art performance under various input uncertainties.
Gaussian processes with differential privacy protect both inputs and outputs.
problem Previous DP methods only protected model outputs, not inputs.
method Sparse GP with private variational approximation, adjusting covariance for DP noise.
result Accurate models can be produced under strong privacy protection with sufficient data.
Improves active learning efficiency by warping input space based on observed outputs.
problem Insensitivity of Gaussian process uncertainty to actual observations.
method Input warping with learned monotone reparameterization to adjust acquisition function behavior.
result Significantly improved sample efficiency across various benchmarks, especially in non-stationary conditions.
Gaussian processes are improved to account for input noise in earth observation.
problem Accurate error assessment in earth observation models.
method Propose a GP model that propagates input noise through the pipeline.
result Improved error representation in temperature predictions from infrared data.
We introduce Graph Neural Processes (GNP), inspired by the recent work in conditional and latent neural processes. A Graph Neural Process is defined as a Conditional Neural Process that operates on arbitrary graph data. It takes features of sparsely observed context points as input, and outputs a distribution over targ…
The class of Gaussian Process (GP) methods for Temporal Difference learning has shown promise for data-efficient model-free Reinforcement Learning. In this paper, we consider a recent variant of the GP-SARSA algorithm, called Sparse Pseudo-input Gaussian Process SARSA (SPGP-SARSA), and derive recursive formulas for its…
We introduce a novel kernel that models input-dependent couplings across multiple latent processes. The pairwise joint kernel measures covariance along inputs and across different latent signals in a mutually-dependent fashion. A latent correlation Gaussian process (LCGP) model combines these non-stationary latent comp…
NES improves robust optimization with noisy inputs.
problem Finding robust optima in problems with input and measurement noise.
method Noisy-Input Entropy Search (NES) acquisition function based on Gaussian process modeling.
result NES reliably finds robust optima, outperforming existing methods.
Optimal kernel learning improves GP regression for high-dimensional inputs.
problem High computational costs and low prediction accuracy in GP models with many inputs.
method Approximates GP covariance with a convex combination of kernel functions, identifying active variables.
result Improves prediction accuracy and correctly identifies active input variables.
The problem of combined state and input estimation of linear structural systems based on measured responses and a priori knowledge of structural model is considered. A novel methodology using Gaussian process latent force models is proposed to tackle the problem in a stochastic setting. Gaussian process latent force mo…
Transformationally invariant processors constructed by transformed input vectors or operators have been suggested and applied to many applications. In this study, transformationally identical processing based on combining results of all sub-processes with corresponding transformations at one of the processing steps or …
MIMONets speed up neural network inference by processing multiple inputs in parallel.
problem Reducing computational cost in neural network inference for large datasets.
method Proposes MIMONets, which augment neural network architectures with variable binding mechanisms to handle multiple inputs in superposition.
result Achieves significant speedups (2-4x) with minimal accuracy loss, demonstrating adaptability across different architectures.
Centroid Transformers reduce memory and computation by summarizing inputs into centroids.
problem Efficiently summarize inputs with reduced memory and computation.
method Generalizes self-attention to map N inputs to M centroids (M ≤ N), reducing complexity.
result Centroid Transformers reduce memory and computation while preserving key information.
In this paper we consider the problem of finding stable maxima of expensive (to evaluate) functions. We are motivated by the optimisation of physical and industrial processes where, for some input ranges, small and unavoidable variations in inputs lead to unacceptably large variation in outputs. Our approach uses multi…
We study the Gaussian Process regression model in the context of training data with noise in both input and output. The presence of two sources of noise makes the task of learning accurate predictive models extremely challenging. However, in some instances additional constraints may be available that can reduce the unc…
Enhances multi-fidelity modeling with DGPs for different input domains.
problem Improving prediction accuracy with multi-fidelity models using different input domains.
method Extends Deep Gaussian Processes (DGPs) to handle different input domains for high and low-fidelity models.
result Demonstrates improved performance on real-world physical problems.
We introduce fully scalable Gaussian processes, an implementation scheme that tackles the problem of treating a high number of training instances together with high dimensional input data. Our key idea is a representation trick over the inducing variables called subspace inducing inputs. This is combined with certain m…
Active learning selects inputs for GPSSM to learn latent states.
problem Optimally learn latent states of a GPSSM through active selection of inputs.
method Use mutual information to select informative inputs; approximate mutual information for GPSSM.
result Effective active learning of GPSSM dynamics in physical systems.
LMGPs extend GPs to handle mixed data, offering better accuracy and interpretability.
problem Handling mixed data types (quantitative and qualitative) in metamodeling.
method Introduce LMGPs that learn a latent manifold for qualitative inputs, using a low-rank linear map.
result LMGPs outperform existing methods in accuracy and versatility.
This paper introduces Non-Autonomous Input-Output Stable Network(NAIS-Net), a very deep architecture where each stacked processing block is derived from a time-invariant non-autonomous dynamical system. Non-autonomy is implemented by skip connections from the block input to each of the unrolled processing stages and al…
A new method guides neural networks to focus on specific features of input data.
problem Training neural networks to consider specific features of input data.
method Focus-and-Expand ( ax) method: Gradual manipulation of input features.
result Achieves state-of-the-art results in bias removal and image classification tasks.
TVS-FNNs can approximate any continuous function on expanded input spaces.
problem Processing a broader range of inputs like sequences and matrices.
method Proving a universal approximation theorem for TVS-FNNs.
result TVS-FNNs can approximate any continuous function on expanded input spaces.
State-space systems generate probabilistic dependencies between inputs and outputs.
problem Understanding probabilistic dependencies in state-space systems.
method Introducing a probabilistic framework and proving sufficient conditions for output existence and uniqueness.
result State-space systems can generate probabilistic dependencies, even without functional relations.
Proposes a framework to fuse heterogeneous data sources for better modeling.
problem Heterogeneous data sources with different input parameter spaces.
method Input mapping calibration (IMC) and latent variable Gaussian process (LVGP).
result Improved predictive accuracy over single source models.
Biological and artificial neural systems are composed of many local processors, and their capabilities depend upon the transfer function that relates each local processor's outputs to its inputs. This paper uses a recent advance in the foundations of information theory to study the properties of local processors that u…
Gaussian processes are used in machine learning to learn input-output mappings from observed data. Gaussian process regression is based on imposing a Gaussian process prior on the unknown regressor function and statistically conditioning it on the observed data. In system identification, Gaussian processes are used to …
The paper develops GP classifiers for noisy inputs in multi-class classification.
problem Input noise in real-world data affects supervised learning performance.
method Developed multi-class Gaussian Process classifiers using variational inference.
result The proposed methods outperform noise-ignoring GP classifiers in predictive distribution.
A new method warps inputs to learn nonstationary kernels efficiently.
problem Learning nonstationary patterns in data with varying smoothness.
method Sparse spectrum Gaussian processes with input warping as conditional Gaussian measures.
result Efficient learning of nonstationary patterns with fewer parameters.