Study shows limitations and possibilities of learning quantum circuit output distributions.
problem Learnability of output distributions of local quantum circuits.
method Investigated within two oracle models: statistical query model and direct sample access model.
result Output distributions of super-logarithmic depth Clifford circuits are not efficiently learnable in the statistical query model.
Reverse sensitivity analysis for risk models under various stresses.
problem Understanding model changes under output stress.
method Deriving the closest stressed distribution and model parameters.
result Numerically efficient method for calculating stressed model.
Efficiently approximates uncertainty in classification models using Dirichlet distributions.
problem Inefficient computation of uncertainty estimates in Bayesian deep learning.
method Revised Laplace Bridge method to construct a Dirichlet approximation of softmax output distributions.
result The Dirichlet approximation leads to more efficient computation and better uncertainty estimates.
Enhances fairness in multi-output models using optimal transport.
problem Improving fairness in multi-output models like multi-task/multi-class classification and representation learning.
method Post-processing method using optimal transport mappings to move model outputs towards empirical Wasserstein barycenter.
result Demonstrates effectiveness of the proposed approach on multi-task/multi-class classification and representation learning tasks.
In the context of machine learning, disparate impact refers to a form of systematic discrimination whereby the output distribution of a model depends on the value of a sensitive attribute (e.g., race or gender). In this paper, we propose an information-theoretic framework to analyze the disparate impact of a binary cla…
TDistNNs improve prediction intervals for neural networks by using t-distributions.
problem Traditional neural networks provide only point estimates, lacking predictive uncertainty.
method TDistNNs generate t-distributed outputs with adjustable degrees of freedom, enhancing robustness to non-Gaussian data.
result TDistNNs produce narrower prediction intervals with proper coverage compared to Gaussian-based PNNs.
New clustering method uses Wasserstein distance to analyze simulation outputs.
problem Analyzing stochastic simulation outputs to uncover relationships and patterns.
method Agglomerative clustering using regularized Wasserstein distance.
result Identifies staffing plans yielding similar performance outcomes.
MixDiff detects OOD samples in constrained access environments by comparing perturbed samples.
problem Detecting out-of-distribution samples in models with restricted access.
method Apply identical perturbation to target and similar ID sample, compare model outputs.
result MixDiff enhances OOD detection performance consistently across various datasets.
E2M predicts metric space outputs using deep learning.
problem Predicting non-Euclidean outputs like distributions and matrices.
method Weighted Fréchet means over learned weights.
result E2M achieves state-of-the-art performance across various outputs.
We are concerned with obtaining well-calibrated output distributions from regression models. Such distributions allow us to quantify the uncertainty that the model has regarding the predicted target value. We introduce the novel concept of distribution calibration, and demonstrate its advantages over the existing defin…
We present a novel approach for learning an HMM whose outputs are distributed according to a parametric family. This is done by {\em decoupling} the learning task into two steps: first estimating the output parameters, and then estimating the hidden states transition probabilities. The first step is accomplished by fit…
New method predicts model output distributions to improve data attribution.
problem Traditional training data attribution ignores randomness in model training.
method Distributional Training Data Attribution (d-TDA) using influence functions.
result Influence functions are effective and emerge naturally from d-TDA.
A new variable importance measure for DRFs detects broader impacts on output distributions.
problem Estimating full conditional distributions of multivariate outputs given inputs.
method Based on the drop and relearn principle and MMD distance.
result Consistent and high-performing variable importance measure for DRFs.
Bayesian kernel regression improves functional output prediction.
problem Functional output regression in supervised learning.
method Kernel methods, leveraging covariance structure within function values.
result Enhanced prediction accuracy and handling of high-dimensional nonlinearity.
We propose the Insertion-Deletion Transformer, a novel transformer-based neural architecture and training method for sequence generation. The model consists of two phases that are executed iteratively, 1) an insertion phase and 2) a deletion phase. The insertion phase parameterizes a distribution of insertions on the c…
Proposes a deep ordinal regression framework using optimal transport loss and unimodal output probabilities.
problem Lack of unimodal output probabilities in recent ordinal regression models.
method Introduces a deep learning framework based on optimal transport loss and unimodal output distribution, inspired by the Proportional Odds model.
result Demonstrates improved performance and unimodal output probabilities on real-world datasets compared to existing methods.
Study improves estimation of rare language model outputs.
problem Estimating probabilities of rare outputs in language models.
method Importance sampling vs. activation extrapolation for low probability estimation.
result Importance sampling outperforms activation extrapolation.
New method certifies risks of LLM outputs, improving accuracy and reliability.
problem Uncertain and incorrect outputs from large language models.
method Information-lift certificates using PAC-Bayes bounds and skeleton design.
result Achieves 77.0% coverage at 2% risk, outperforming baselines.
In-N-Out improves model robustness to out-of-distribution data.
problem Learning robust models with few in-distribution labeled examples.
method Pre-training with auxiliary information and self-training with pseudolabels.
result In-N-Out outperforms auxiliary inputs or outputs alone on both in-distribution and OOD error.
Multi-output regression models must exploit dependencies between outputs to maximise predictive performance. The application of Gaussian processes (GPs) to this setting typically yields models that are computationally demanding and have limited representational power. We present the Gaussian Process Autoregressive Regr…
Proposes BATer for improved adversarial example detection.
problem Detecting adversarial examples in neural networks.
method Introduces a Bayesian adversarial example detector (BATer) using random components in a Bayesian neural network.
result BATer outperforms state-of-the-art detectors in adversarial example detection.
Unified framework for output analysis using Monte Carlo sampling.
problem Accurately assess the quality of estimated values in predictive models.
method Unified output analysis framework through Monte Carlo sampling, leveraging fast iterative bootstrap sampling and higher-order influence functions.
result Clear advantage in building more robust confidence intervals with higher coverage probability.
Sig-PCA integrates model outputs and observations to correct model biases.
problem Improving model accuracy and reliability by correcting biases and numerical approximations.
method Sig-PCA framework that combines summary statistics from model outputs with localized observations via a neural network.
result Corrects model outputs to align closely with observational data, preserving essential statistical information.
Soft labeling impacts OOD detection in neural networks.
problem Impact of soft labeling on OOD detection in deep neural networks.
method Empirical analysis of how soft labeling affects OOD detection performance.
result Soft labeling can either improve or deteriorate OOD detection performance.
Neural Processes (NPs) (Garnelo et al 2018a;b) approach regression by learning to map a context set of observed input-output pairs to a distribution over regression functions. Each function models the distribution of the output given an input, conditioned on the context. NPs have the benefit of fitting observed data ef…
Differential privacy, a notion of algorithmic stability, is a gold standard for measuring the additional risk an algorithm's output poses to the privacy of a single record in the dataset. Differential privacy is defined as the distance between the output distribution of an algorithm on neighboring datasets that differ …
Gaussian processes (GPs), or distributions over arbitrary functions in a continuous domain, can be generalized to the multi-output case: a linear model of coregionalization (LMC) is one approach. LMCs estimate and exploit correlations across the multiple outputs. While model estimation can be performed efficiently for …
Improved selection of best outputs from LLMs for better accuracy.
problem BoN fails to reliably find the correct answer with imperfect rewards.
method Majority-of-the-Bests (MoB) selects the mode of a bootstrapped output distribution of BoN.
result MoB consistently improves over BoN in 25 out of 30 setups.
Multi-output Gaussian processes (MOGP) are probability distributions over vector-valued functions, and have been previously used for multi-output regression and for multi-class classification. A less explored facet of the multi-output Gaussian process is that it can be used as a generative model for vector-valued rando…
Proposes FOAGP for efficient orthogonal effect decomposition of black-box computer experiments.
problem Challenges in sensitivity analysis of black-box computer experiments with complex, nonlinear functional outputs.
method Functional-output orthogonal additive Gaussian process (FOAGP) with conditional orthogonality constraint.
result Demonstrates effectiveness in orthogonal effect decomposition and variance decomposition through simulations and real-world application.
Gaussian process model for vector-valued function has been shown to be useful for multi-output prediction. The existing method for this model is to re-formulate the matrix-variate Gaussian distribution as a multivariate normal distribution. Although it is effective in many cases, re-formulation is not always workable a…
Learning, taking into account full distribution of the data, referred to as generative, is not feasible with deep neural networks (DNNs) because they model only the conditional distribution of the outputs given the inputs. Current solutions are either based on joint probability models facing difficult estimation proble…
Bounds neural network output distribution to Gaussian for random initialization.
problem Quantifying the distribution of randomly initialized deep neural networks.
method Quantitative Gaussian approximation using quadratic Wasserstein distance.
result Explicit inequalities show how network sizes affect Gaussian behavior.
This paper introduces a multi-output Gaussian process for censored data.
problem Modeling bias in censored data using correlations between multiple outputs.
method Heteroscedastic multi-output Gaussian process with input-dependent noise and variational inference.
result The model better estimates the true process under complex censoring dynamics.
New method improves calibration in multi-output probabilistic models.
problem Challenges in achieving multivariate calibration in multi-output regression.
method General regularization framework to enforce multivariate calibration during training for arbitrary pre-rank functions.
result Significant improvement in calibration across all pre-rank functions without sacrificing predictive accuracy.
Exact bounds derived for neural network outputs with noisy inputs.
problem Bounding the output distribution of neural networks with random inputs.
method Applying ReLU NNs to derive bounds for general NNs, then using these to find exact error guarantees.
result Exact upper and lower bounds for the output distribution of neural networks with random inputs.
The paper improves recommendation systems by ensuring their outputs are reliable.
problem Recommendation systems often lack reliability guarantees for their outputs.
method The method uses a pre-trained ranking model to create a set of items with rigorous FDR control.
result The approach provides a way to guarantee the reliability of recommendation outputs.
Transformer with denoising diffusion improves probabilistic density estimation.
problem Estimating non-Gaussian and multimodal probability distributions for regression problems.
method Training a denoising diffusion head on top of a Transformer model.
result The model provides reasonable probability density estimation for high-dimensional inputs.
Study on Bayesian deep linear networks with multiple outputs and convolutional layers.
problem Characterize feature learning in finite-width Bayesian deep linear networks.
method Exact and analytical formulas for joint and posterior distributions, using large deviation theory.
result Quantitative description of feature learning in infinite-width regime.
Conditional restricted Boltzmann machines are undirected stochastic neural networks with a layer of input and output units connected bipartitely to a layer of hidden units. These networks define models of conditional probability distributions on the states of the output units given the states of the input units, parame…
PCENet reduces uncertainty in high-dimensional data efficiently.
problem Uncertainty quantification in high-dimensional data is computationally expensive.
method Two-stage learning process: variational autoencoder for low-dimensional representation, polynomial chaos expansion for mapping.
result Model captures system dynamics, learns under uncertainty, estimates high-dimensional data uncertainty, matches output distribution moments.
Hidden Markov Models (HMM) have been used for several years in many time series analysis or pattern recognitions tasks. HMM are often trained by means of the Baum-Welch algorithm which can be seen as a special variant of an expectation maximization (EM) algorithm. Second-order training techniques such as Variational Ba…
Introduces foundation priors for using model-generated data in empirical research.
problem Using model-generated data as real observations in empirical research.
method Introduces foundation priors as an exponential-tilted, generalized Bayesian update of the user's primitive prior.
result Synthetic data reflects both model patterns and user's priors, enabling principled use in empirical work.
DBPA assesses LLM perturbations using frequentist hypothesis testing.
problem Quantifying input perturbation impacts on LLM outputs.
method DBPA reformulates perturbation analysis as frequentist hypothesis testing, using Monte Carlo sampling for empirical null and alternative distributions.
result DBPA provides interpretable p-values and scalar effect sizes for LLM perturbations.
WassFFed addresses fairness in Federated Learning by ensuring consistency between local and global models.
problem Achieving fairness in Federated Learning where data is distributed among diverse user groups.
method WassFFed employs a Wasserstein barycenter calculation to aggregate local models' outputs, ensuring consistency and fairness.
result WassFFed outperforms existing approaches in balancing accuracy and fairness.
We introduce an adaptive output-sensitive Metropolis-Hastings algorithm for probabilistic models expressed as programs, Adaptive Lightweight Metropolis-Hastings (AdLMH). The algorithm extends Lightweight Metropolis-Hastings (LMH) by adjusting the probabilities of proposing random variables for modification to improve c…
Obtaining reliable uncertainty estimates of neural network predictions is a long standing challenge. Bayesian neural networks have been proposed as a solution, but it remains open how to specify their prior. In particular, the common practice of an independent normal prior in weight space imposes relatively weak constr…
Conformal Alignment ensures trustworthy outputs from foundation models.
problem Ensuring outputs from foundation models align with human values in high-stakes tasks.
method A framework that trains an alignment predictor using reference data to select trustworthy outputs.
result Conformal Alignment accurately identifies trustworthy outputs via lightweight training over moderate reference data.