This paper studies the geometry of minimum-volume confidence sets for multinomial parameters.
problem Determining if minimum-volume confidence sets for multinomial outcomes are disjoint.
method Enumerating and covering the continuous regions of the exact p-value function to study the geometry of minimum-volume confidence sets.
result The geometry of minimum-volume confidence sets for multinomial parameters is studied, providing insights into their structure and properties.
Constructs non-asymptotic confidence regions for unknown functions in RKHS.
problem Global probabilistic confidence regions for unknown functions in RKHS.
method Reduces confidence region construction to estimating RKHS norm.
result Valid confidence regions can be constructed non-asymptotically.
New method improves auto-labeling accuracy by optimizing confidence functions.
problem Overconfident model scores lead to poor TBAL performance.
method Developed a new post-hoc method, Colander, to optimize TBAL confidence functions.
result Achieves up to 60% improvement in coverage over baseline methods.
CoinDICE estimates confidence intervals for unknown behavior policies in reinforcement learning.
problem Estimating value of a target policy using only behavior policy data.
method Function space embedding, generalized empirical likelihood method, Lagrangian optimization.
result Valid confidence intervals with tighter and more accurate estimates than existing methods.
A new concept of confidence in learning is defined and analyzed.
problem Understanding and quantifying trust in learning processes.
method Formal axioms, continuum measures, vector fields, loss functions.
result Confidence can be represented and optimized in learning.
Method constructs confidence regions for linear models with arbitrary predictors.
problem Constructing confidence regions for linear models with non-linear predictors.
method Mixed Integer Linear Programming for constraints.
result Empty confidence regions for hypothesis testing.
Paper proposes a method to estimate confidence bands for survival random forests.
problem No statistically valid and computationally feasible approach for estimating confidence bands for survival random forests.
method Extending recent developments in infinite-order incomplete U-statistics, the paper proposes an unbiased confidence band estimation.
result The proposed method accurately estimates the confidence band and achieves desired coverage rate.
WS-KDE provides robust confidence bounds for stochastic functions.
problem Optimizing time-consuming black-box functions with stochastic outputs.
method Wilson Score Kernel Density Estimation (WS-KDE) for Bayesian optimization.
result WS-KDE provides reliable confidence bounds for any stochastic function.
The paper improves nonparametric confidence bands for band-limited functions.
problem Constructing nonparametric simultaneous confidence bands with nonasymptotic and distribition-free guarantees.
method Based on Paley-Wiener reproducing kernel Hilbert spaces, the paper relaxes assumptions, improves noise estimation, and tightens constraints.
result Enhanced confidence bands with improved efficiency and tighter constraints.
The paper improves confidence regions for band-limited functions using tighter norm bounds and majority voting.
problem Constructing reliable confidence regions for band-limited functions from noisy data.
method Improved norm bounds using Hoeffding's inequality and empirical Bernstein bound, majority voting to aggregate intervals.
result Confidence intervals retain their simultaneous coverage guarantee even when aggregated from random subsamples.
The paper shows over-confidence in models isn't just due to over-parametrization.
problem Over-confidence in machine learning models, especially in binary classification.
method Theoretical analysis of logistic regression and other binary classification problems.
result Logistic regression is inherently over-confident in certain settings, but over-confidence is not always the case.
Note improves confidence bounds for random variables.
problem Improving confidence bounds for random variables with unbounded ranges and different distributions.
method PAC-Bayes-ification of a derived confidence bound.
result Streamlined proofs for existing results.
This paper improves GP-UCB by using a shifted exponential distribution for confidence parameters.
problem Theoretical confidence parameter in GP-UCB increases with iterations, leading to large values.
method Introduced IRGP-UCB, a randomized variant of GP-UCB using a shifted exponential distribution for confidence parameters.
result IRGP-UCB achieves sub-linear regret without increasing the confidence parameter.
The paper creates nonparametric confidence bands for band-limited functions.
problem Estimating confidence bands for band-limited functions with finite samples and unknown noise.
method Uses Paley-Wiener reproducing kernel Hilbert spaces and gradient-perturbation methods.
result Non-asymptotic guarantees for confidence regions without assuming a parametric model.
A method identifies abrupt changes in functions with fixed confidence under noisy feedback.
problem Identifying abrupt changes in piecewise constant functions quickly and with certainty.
method Fixed-confidence piecewise constant bandit problem, focusing sampling efforts near change points.
result Asymptotically optimal method proven computationally efficient and effective in experiments.
The paper develops approximations for Pearson's chi-square statistic and applies them to confidence intervals.
problem Finding confidence intervals for strictly convex functions of discrete distribution weights.
method Non-asymptotic local normal approximation for multinomial probabilities, deriving bounds and coupling inequalities.
result Developed methods to find confidence intervals for negative entropy of discrete distributions.
We provide a pointwise confidence bound for non-linear least-squares with fixed design.
problem Confidence estimation in non-linear ℓ 2 \ell^2 ℓ 2 -regularized least squares. method Pointwise confidence bound for local minimizers, using weighted norm involving inverse-Hessian.
result The proposed confidence bound scales with the test input's similarity to the training data.
The paper develops optimal confidence regions for categorical data.
problem Constructing tight confidence regions for categorical data.
method Develops new theory for minimum average volume confidence regions.
result Shows optimality of the regions for categorical data and its implications for machine learning.
Improved algorithms for stochastic linear bandits using tighter confidence sequences.
problem Stochastic linear bandits with improved worst-case regret guarantees.
method Novel tail bound for adaptive martingale mixtures to construct tighter confidence sequences.
result Linear bandit algorithm achieves competitive worst-case regret.
The paper develops methods for constructing confidence regions for regression functions in binary classification.
problem Building distribution-free confidence regions for regression functions in binary classification.
method Resampling test and empirical risk minimization approach for model classes with finite pseudo-dimensions and inverse Lipschitz parameterizations.
result Strong uniform consistency and exponential probably approximately correct bounds on the L 2 L_2 L 2 sizes of the regions. Exploits class similarity for better machine learning models with confidence labels and projective loss functions.
problem Poor model performance due to confusing similar classes.
method Exploits class similarity with confidence labels and projective loss functions.
result Improved model performance on noisy labels.
This paper proposes a decorrelation-based approach to test hypotheses and construct confidence intervals for the low dimensional component of high dimensional proportional hazards models. Motivated by the geometric projection principle, we propose new decorrelated score, Wald and partial likelihood ratio statistics. Wi…
Estimating the value function for a fixed policy is a fundamental problem in reinforcement learning. Policy evaluation algorithms---to estimate value functions---continue to be developed, to improve convergence rates, improve stability and handle variability, particularly for off-policy learning. To understand the prop…
Learn low-degree functions with few random queries.
problem Learning low-degree functions from limited random queries.
method Learn bounded functions f : { − 1 , 1 } n o [ − 1 , 1 ] f:\{-1,1\}^n o[-1,1] f : { − 1 , 1 } n o [ − 1 , 1 ] of degree at most d d d with L 2 L_2 L 2 -accuracy ε \varepsilon ε and confidence 1 − δ 1-δ 1 − δ from log ( f r a c n δ ) ε − d − 1 C d 3 / 2 log d \log( frac{n}δ)\,\varepsilon^{-d-1} C^{d^{3/2}\sqrt{\log d}} log ( f r a c n δ ) ε − d − 1 C d 3/2 l o g d random queries. result Learn low-degree functions efficiently with logarithmic number of random queries.
We propose a generic framework to calibrate accuracy and confidence of a prediction in deep neural networks through stochastic inferences. We interpret stochastic regularization using a Bayesian model, and analyze the relation between predictive uncertainty of networks and variance of the prediction scores obtained by …
Optimal learning via moderate deviations theory improves statistical accuracy.
problem Statistical estimation of expected loss in various models.
method Develops confidence intervals using moderate deviation principle.
result Proposed confidence intervals are statistically optimal.
New method constructs confidence bands for ODE models with unknown regulatory effects.
problem Building confidence bands for ODE models with unknown regulatory relations is challenging.
method Localized kernel learning approach combined with de-biasing method.
result The constructed confidence band has the desired asymptotic coverage probability and accurately recovers the regulatory network.
ACORE improves hypothesis testing and confidence sets in likelihood-free inference.
problem Constructing hypothesis tests and confidence sets in likelihood-free inference settings.
method Formulates classical LRT as a classification problem, uses machine learning to improve estimates.
result Demonstrates improved accuracy in hypothesis testing and confidence sets.
Paper introduces methods for more reliable probabilistic predictions with confidence intervals.
problem Inaccurate labeling of datasets due to unreliable probabilistic predictions from weak labeling functions.
method Proposes a methodology to provide confidence intervals for label probabilities using uncertainty sets of distributions.
result Improves reliability of probabilistic predictions and provides confidence intervals for label probabilities.
Classifiers used in the wild, in particular for safety-critical systems, should not only have good generalization properties but also should know when they don't know, in particular make low confidence predictions far away from the training data. We show that ReLU type neural networks which yield a piecewise linear cla…
This paper presents approximate confidence intervals for each function of parameters in a Banach space based on a bootstrap algorithm. We apply kernel density approach to estimate the persistence landscape. In addition, we evaluate the quality distribution function estimator of random variables using integrated mean sq…
A new method for semi-supervised learning with missing data using GMM and margin confidence.
problem Handling missing data in semi-supervised learning with classification uncertainty.
method Explicitly models missingness mechanism, uses margin confidence and Aranda Ordaz function, develops ECM algorithm.
result Effective reduction of bias and robustness in semi-supervised learning with substantial missing labels.
Improved Bayesian optimisation method using randomised Gaussian process UCB.
problem Improving performance in Bayesian optimisation.
method Developed a modified Gaussian process upper confidence bound (GP-UCB) acquisition function.
result The method achieves better performance than GP-UCB in various problems.
Confidence bands for tuning curves improve hyperparameter comparison in NLP.
problem Ambiguity in comparing hyperparameter tuning methods.
method Constructs exact, simultaneous, and distribution-free confidence bands for tuning curves.
result Confidence bands provide a robust basis for comparing methods rigorously.
New method for constructing confidence intervals for time series data.
problem Constructing confidence intervals for statistical functionals from time series data.
method Proposes a general purpose confidence interval procedure based on overlapping batches of time series data.
result Large overlapping batches yield confidence intervals of higher quality than generic methods.
Paper proposes a method to improve deep neural networks' confidence estimates.
problem Overconfident predictions limit practical use of deep neural networks in safety-critical applications.
method Proposes a novel loss function, Correctness Ranking Loss, to regularize class probabilities.
result The method produces well-ranked confidence estimates and is effective for out-of-distribution detection and active learning.
New algorithm for precise changepoint localization without assumptions.
problem Offline changepoint localization in arbitrary distributions.
method Distribution-free algorithm CONformal CHangepoint localization (CONCH) using exchangeability arguments.
result Derives principled score functions for informative and small confidence sets with normalized length shrinking to zero.
In the field of reinforcement learning there has been recent progress towards safety and high-confidence bounds on policy performance. However, to our knowledge, no practical methods exist for determining high-confidence policy performance bounds in the inverse reinforcement learning setting---where the true reward fun…
A universal framework for constructing confidence sets using sequential likelihood mixing.
problem Constructing reliable confidence sets for realizable likelihood functions.
method Sequential likelihood mixing, integrating Bayesian inference and regret inequalities.
result Establishes fundamental connections and provable coverage guarantees for various inference techniques.
Paper improves confidence intervals and variance estimation for deep learning models.
problem Improving confidence intervals and variance estimation in deep learning models.
method Residual-based framework for conditional variance estimation; robust bootstrap procedure for confidence intervals.
result First non-asymptotic bounds for variance estimation using ReLU networks.
The paper addresses the difficulty of decision makers trusting AI-assisted predictions and proposes a method to improve confidence values.
problem Decision makers struggle to trust AI-assisted predictions based on confidence values.
method The paper investigates why decision makers have difficulties and proposes a method to construct more useful confidence values.
result Multicalibration with respect to the decision maker's confidence on her own predictions is a sufficient condition for alignment, leading to better decisions.
The paper honors Lai's contributions to multi-armed bandits and establishes new regret bounds.
problem Improving regret bounds in multi-armed bandit problems.
method Establishes non-asymptotic regret bounds for upper confidence bound indices.
result New regret bounds match Lai-Robbins lower bound.
A theorem for debiasing machine learning with finite sample guarantees.
problem Calculating confidence intervals for machine learning functionals.
method Debiased machine learning based on bias correction and sample splitting.
result Nonasymptotic debiased machine learning theorem with finite sample guarantees.
This paper improves confidence measurement in deep metric learning models.
problem Measuring confidence in deep metric learning models is challenging.
method Approximates class distributions using Gaussian kernel smoothing and calibrates the confidence metric.
result Improves generalization and robustness of deep metric learning models.
New principle optimizes bandit decisions with context.
problem Dealing with general function classes and large context spaces in bandits.
method Upper Counterfactual Confidence Bounds (UCCB) principle.
result Proves optimality and efficiency in complex settings.
We study the tradeoff between computational effort and classification accuracy in a cascade of deep neural networks. During inference, the user sets the acceptable accuracy degradation which then automatically determines confidence thresholds for the intermediate classifiers. As soon as the confidence threshold is met,…
The paper provides a method to find optimal machine learning model parameters with confidence.
problem Finding optimal machine learning model parameters that generalize well to the entire population.
method Constructs valid confidence sets for the optimal parameter using only training data.
result Valid confidence sets for optimal machine learning model parameters can be generated using bootstrapping techniques.
The paper improves confidence set construction for statistical inference.
problem Constructing reliable confidence sets in statistical inference.
method Establishes a finite-sample bound using effective dimension and generalized self-concordance.
result Developed a confidence set adapted to optimization landscapes.