New method for non-stationary GPR using HMC for gradient-based inference.
problem Modeling input-dependent dynamics in non-stationary GPR.
method Gradient-based inference with Hamiltonian Monte Carlo (HMC).
result Non-stationary GPR outperforms stationary models in gene expression modeling.
We reduce variance in RL with input-dependent baselines.
problem High variance in RL with standard baselines in input-driven environments.
method Derive and use a bias-free, input-dependent baseline; propose a meta-learning approach.
result Input-dependent baselines improve training stability and policy quality.
Input-dependent smoothing mitigates classical issues but suffers from the curse of dimensionality.
problem Certifiably robust classifiers with input-dependent smoothing suffer from the curse of dimensionality.
method Proposed a theoretical and practical framework for input-dependent smoothing under strict restrictions.
result Input-dependent smoothing mitigates some classical issues but is limited by the curse of dimensionality.
The ACCRU framework improves probabilistic forecasts by capturing input-dependent uncertainty.
problem Uncertainty in deterministic predictions, especially for skewed and non-Gaussian errors.
method Neural network trained with a loss function balancing accuracy and reliability to learn input-dependent, non-Gaussian uncertainty distributions.
result Improves probabilistic forecasts relative to existing methods, capturing skewed and non-Gaussian errors.
This paper proposes a fast method for estimating input-dependent prediction intervals in Extreme Learning Machines.
problem Estimating reliable prediction intervals for Extreme Learning Machines with heteroscedastic outputs.
method A separate Extreme Learning Machine model estimates input-dependent prediction intervals using a weighted Jackknife method to correct for model uncertainty.
result The proposed method is fast, robust to heteroscedastic outputs, and handles large datasets and insufficient training data.
This paper presents a novel approach for approximate integration over the uncertainty of noise and signal variances in Gaussian process (GP) regression. Our efficient and straightforward approach can also be applied to integration over input dependent noise variance (heteroscedasticity) and input dependent signal varia…
New method corrects deep neural network variance for outlier data.
problem Outliers in deep neural network outputs.
method Gradient Conjugate Priors (GCP) network.
result Corrected variance is in a small neighborhood of ground truth variance.
Simple technique turns any adversarial attack into a universal one using few test examples.
problem Creating universal adversarial attacks with minimal data.
method Universalization technique using few adversarial test examples and spectral properties.
result Simple universalization technique achieves comparable fooling rates to state-of-the-art methods.
We introduce a new regression framework, Gaussian process regression networks (GPRN), which combines the structural properties of Bayesian neural networks with the non-parametric flexibility of Gaussian processes. This model accommodates input dependent signal and noise correlations between multiple response variables,…
Novel Bayesian prior for neural networks encodes amplitude and lengthscale.
problem Lack of user-friendly priors for specifying basic properties in Bayesian neural networks.
method Introduced Poisson Process Radial Basis Function Networks (PP-RBFN) as a novel prior.
result PP-RBFN allows decoupled specification of amplitude and lengthscale, and estimated function is consistent.
Study identifies latent variables and models from spacecraft data.
problem Learning reliable models from spacecraft data with complex relationships.
method Inductive bias inspired by controllable canonical forms for sparse, input-dependent latent variables.
result Identifies latent variables up to scaling and determines dynamic models up to transformations for linear and affine systems.
New kernels model non-stationary data efficiently.
problem Efficiently modeling non-stationary data with Gaussian processes.
method Model spectral density as a mixture of frequency surfaces, solve generalised Fourier transform.
result Derives efficient inference methods for non-stationary kernels.
CoDA Nets improve interpretability in neural networks.
problem Improving interpretability in neural networks.
method Dynamic Alignment Units (DAUs) for input-dependent linear transformations.
result CoDA Nets achieve on par results with ResNet and VGG models on complex datasets.
NAIS-Net stabilizes deep networks using non-autonomous dynamical systems.
problem Stabilizing deep neural networks to prevent vanishing/exploding gradients.
method NAIS-Net uses non-autonomous dynamical systems with skip connections to enforce stability.
result NAIS-Net proves to be globally asymptotically stable and reduces generalization gap.
A novel kernel models latent variable couplings across multiple processes.
problem Modeling latent variable couplings across multiple processes.
method Mutually-dependent Hadamard kernel and latent correlation Gaussian process (LCGP) model.
result The LCGP model recovers latent signal correlations and achieves state-of-the-art performance.
Gaussian Process (GP) regression models typically assume that residuals are Gaussian and have the same variance for all observations. However, applications with input-dependent noise (heteroscedastic residuals) frequently arise in practice, as do applications in which the residuals do not have a Gaussian distribution. …
AF improves classification models by adaptively weighting trees.
problem Improving classification model performance.
method AF combines OP2T for input-dependent weights and MIO for dynamic refinement.
result AF consistently outperforms RF, XGBoost, and other weighted RF.
Simple method improves deep classifier accuracy under noisy labels.
problem Training deep classifiers with noisy labels.
method Probabilistic approach using temperature parameterized softmax.
result Improves accuracy, log-likelihood and calibration on noisy datasets.
GA-Net selectively attends to parts of a sequence for text classification.
problem Inefficient global attention mechanisms on long sequences.
method Gated Attention Network (GA-Net) using an auxiliary network to dynamically select and attend to important parts of the sequence.
result GA-Net achieves better performance with less computation and interpretability.
ETGPSSM efficiently models high-dimensional, non-stationary systems with reduced complexity.
problem Prohibitive computational and parametric complexity in high-dimensional, non-stationary dynamical systems.
method ETGPSSM integrates a single shared GP with input-dependent normalizing flows for scalable and flexible modeling.
result ETGPSSM outperforms existing models in computational efficiency and accuracy.
A new method scales CCA parameters by input to learn more correlated representations.
problem Limitation of conventional CCA models in learning highly correlated representations.
method Introduces a dynamic scaling method for training input-dependent canonical correlation models.
result Learned representations are more correlated and retrieval results are preferable.
This paper introduces a multi-output Gaussian process for censored data.
problem Modeling bias in censored data using correlations between multiple outputs.
method Heteroscedastic multi-output Gaussian process with input-dependent noise and variational inference.
result The model better estimates the true process under complex censoring dynamics.
Flexible GP model improves wind power prediction accuracy.
problem Accurate probabilistic prediction of wind power for grid stability.
method Heteroscedastic non-stationary Gaussian process with generalised spectral mixture kernel.
result The proposed model outperforms conventional GP models in wind power prediction.
Improved Mamba model for long-range sequence tasks.
problem Mamba's poor performance on long-range sequential tasks.
method Proposed B2S6, combining block-wise selective dynamics and channel-specific bias.
result Empirically, B2S6 outperforms S4 and S4D on LRA tasks.
The sparse pseudo-input Gaussian process (SPGP) is a new approximation method for speeding up GP regression in the case of a large number of data points N. The approximation is controlled by the gradient optimization of a small set of M `pseudo-inputs', thereby reducing complexity from N^3 to NM^2. One limitation of th…
Improved image classification accuracy with a probabilistic model of label noise.
problem Noisy labels in large-scale image classification datasets.
method A probabilistic model using a multivariate Normal distribution on the final hidden layer of a neural network, capturing input-dependent label noise.
result Significantly improved accuracy on various datasets compared to standard methods.
DFS dynamically decides bitwidths for layers to balance accuracy and efficiency.
problem Balancing model accuracy and inference speed for deep networks.
method Dynamic Fractional Skipping (DFS) framework that assigns bitwidths to layers for input-adaptive inference.
result DFS achieves superior tradeoff between computational cost and model accuracy.
Non-bilinear observations make optimal control harder, showing non-convex costs and non-affine optimal controllers.
problem Optimal control from bilinear observations in linear systems is challenging.
method Analytical and numerical methods to study the non-convex cost-to-go and non-affine optimal controllers.
result The Separation Principle does not hold for bilinear observations, leading to non-convex costs and non-affine optimal controllers.
Bayesian optimisation tackles stochastic MPC hyper-parameter tuning.
problem Fine-tuning hyper-parameters in stochastic MPC models.
method Heteroscedastic Bayesian optimisation framework.
result Framework effectively tunes hyper-parameters in control problems.
GraN-GAN normalizes gradients for better GAN performance.
problem Improving image generation in GANs with piecewise linear discriminators.
method Piecewise Gradient Normalization (GraN) for input-dependent normalization.
result Significant performance gains in image generation across various datasets.
Token-adaptive FFN design improves LLM expressivity.
problem Fixed activation functions limit FFN expressivity in LLMs.
method Mixture of Activations (MoA) and learnable activations (LA).
result MoA achieves lower terminal loss and better scaling than baselines.
NAMEx merges experts using Nash bargaining for improved performance.
problem Sparse Mixture of Experts merging strategies lack a principled weighting mechanism.
method Reinterpreting expert merging through game theory, introducing Nash Merging and complex momentum.
result NAMEx consistently outperforms competing methods across various tasks and system sizes.
New definitions of ESP for quantum reservoir computing handle non-stationary systems.
problem Traditional ESP does not apply to non-stationary systems.
method Introduce two new categories of ESP: non-stationary ESP and subset/subspace ESP.
result Demonstrates correspondence between non-stationary ESP and QRC with NARMA tasks.
Researchers use Gaussian processes with non-stationary kernels to model precipitation patterns in the Upper Indus Basin.
problem Uncertainty in precipitation patterns in the Upper Indus Basin, Himalayas.
method Proposes Gaussian processes with structured non-stationary kernels to model precipitation patterns, accounting for spatial variation with a latent Gaussian process.
result The proposed model adapts to varying precipitation patterns across distinct topography and outperforms stationary models in ablation experiments.
A new machine learning model uses matrix exponentials for universal approximation.
problem Developing a robust and efficient machine learning model.
method Introduces a novel architecture using matrix exponentials as the only nonlinearity.
result The model achieves universal approximation properties and outperforms other models on benchmark tasks.
This paper uses SPNs with GPs to efficiently model complex data.
problem Inference cost and memory issues in Gaussian processes.
method Integrating Gaussian processes into sum-product networks.
result The model efficiently learns input-dependent parameters and hyper-parameters.
FDN improves probabilistic regressors' adaptability to distribution shifts.
problem Overconfidence in modern probabilistic regressors under distribution shift.
method FDN uses input-conditioned distributions over network weights, trained with a Monte Carlo beta-ELBO objective.
result FDN produces predictive mixtures whose dispersion adapts to the input, providing shift-aware uncertainty.
Sparse Gaussian Processes improve scalability by learning inducing points from data.
problem Scaling issues in Gaussian Processes due to cubic computational cost.
method Amortized learning of inducing points and variational posterior parameters using neural networks.
result Significant reduction in the number of inducing points, improving scalability.
HS-MoE selects sparse experts using adaptive priors and data-adaptive gating.
problem Sparse expert selection in mixture-of-experts architectures.
method Combines horseshoe prior with input-dependent gating for data-adaptive sparsity.
result Data-adaptive sparsity in expert usage.
Study quantization effects on high-dimensional linear regression learning.
problem Understanding quantization's impact on learning high-dimensional linear regression models.
method Analyzes stochastic gradient descent for high-dimensional linear regression under various quantization targets.
result Establishes precise bounds on excess risk for different quantization schemes.
Paper models planar pushing with probabilistic data-driven methods.
problem Predicting the outcomes and variability of planar pushing interactions.
method Variational Heteroscedastic Gaussian processes (VHGP) to capture mean and variance of stochastic function.
result Learned models outperform analytical models with fewer than 1000 samples.
Proposes regularization for robust image models using Wasserstein geometry.
problem Robustness to in-class variations in input data.
method Wasserstein-2 geometry, Tikhonov-type regularizer, data augmentation.
result Improves generalization under adversarial perturbations and large variations.
Paper improves Gumbel-Softmax estimator variance reduction.
problem Challenges in gradient estimation for models with discrete latent variables.
method Rao-Blackwellization applied to straight-through Gumbel-Softmax estimator.
result Reduces mean squared error and variance of Gumbel-Softmax estimator.
CLAPS improves conformal regression by adaptively scaling interval widths based on last-layer Laplace uncertainty.
problem Lack of adaptive interval width scaling in conformal regression for heterogeneous inputs.
method CLAPS uses heteroscedastic last-layer Laplace uncertainty to adaptively scale interval widths, combining aleatoric and epistemic uncertainties.
result CLAPS provides competitive interval efficiency with nominal-level coverage, reducing to aleatoric scaling as epistemic uncertainty decreases.
Paper proposes a method to predict MOBA game winners with calibrated confidence.
problem Predicting MOBA game winners with noisy data and uncertain noise.
method A novel confidence-calibration method considering data uncertainty.
result Achieves outstanding expected calibration error (ECE) of 0.57%.
New method reparameterizes discrete variables to reduce gradient variance.
problem Low variance gradient estimation for discrete variables in neural networks.
method Marginalizing out the variable of interest to bypass discontinuity, resulting in a new reparameterization trick.
result The new reparameterization reduces gradient variance significantly, theoretically not larger than likelihood-ratio method.
New method learns PDE solutions from low-fidelity data.
problem Challenges in learning PDE surrogates with scarce data.
method Flow matching in infinite-dimensional space with conditional neural operators.
result Accurately learns PDE solutions across different resolutions and fidelities.
Bayesian optimization improves with nonstationary covariance functions.
problem Stationary covariance functions fail to capture prior information in high dimensions.
method Proposes nonstationary covariance functions to encode prior information and adaptively promote local exploration.
result Nonstationary covariance functions increase sample efficiency in high dimensions.