Paper introduces a novel method for estimating model confidence in deep neural classifiers.
problem Reliable confidence estimation for deep neural classifiers in safety-critical applications.
method Proposes a novel target criterion (true class probability) and learns it from data with an auxiliary model.
result The proposed method outperforms strong baselines in various tasks and network architectures.
The law of total probability may be deployed in binary classification exercises to estimate the unconditional class probabilities if the class proportions in the training set are not representative of the population class proportions. We argue that this is not a conceptually sound approach and suggest an alternative ba…
Unified view of KL-divergence and IPMs via DRE, with new DRM metrics.
problem Unified understanding of KL-divergence and IPMs.
method Unified representation via maximum likelihood density-ratio estimation (DRE).
result Unified form of IPMs and novel DRM metrics.
Study on optimal rates for sequential probability assignment using smoothed analysis.
problem Optimal rates for sequential probability assignment under smoothed adversaries.
method General-purpose reduction from minimax rates to transductive learning, development of an efficient algorithm using MLE oracle.
result Optimal (logarithmic) fast rates for parametric and finite VC dimension classes, sublinear regret for general classes.
The paper shows how MMD metrizes weak convergence for certain kernels.
problem Characterizing MMD metrizing weak convergence for a wide class of kernels.
method Proving MMD metrizes weak convergence for specific kernels on a locally compact space.
result Corrected prior results and identified new kernels metrizing weak convergence.
A new method for adapting to label shifts using class probability matching.
problem Adapting to label shifts where class probabilities differ between source and target domains.
method Class Probability Matching using Kernel Methods (CPMKM) framework.
result CPMKM outperforms existing methods on real datasets.
Reduces quantifier variance with accuracy optimization of base classifier.
problem Minimizing quantifier variance under prior probability shift.
method Optimizes the Brier score of a base classifier for training data.
result Optimizing Brier score on training data reduces quantifier variance on test data.
New method for sequential probability assignment reduces regret using contextual Shtarkov sums.
problem Minimizing regret in sequential probability assignment with arbitrary hypothesis classes.
method Introducing contextual Shtarkov sum and contextual Normalized Maximum Likelihood (cNML) algorithm.
result The contextual Shtarkov sum characterizes minimax regret and provides a minimax optimal strategy.
Label shift refers to the phenomenon where the prior class probability p(y) changes between the training and test distributions, while the conditional probability p(x|y) stays fixed. Label shift arises in settings like medical diagnosis, where a classifier trained to predict disease given symptoms must be adapted to sc…
We characterize the asymptotic performance of nonparametric goodness of fit testing. The exponential decay rate of the type-II error probability is used as the asymptotic performance metric, and a test is optimal if it achieves the maximum rate subject to a constant level constraint on the type-I error probability. We …
Paper derives convergence rates for NPMLE in Hellinger distance using deep neural networks.
problem Difficulty in proving convergence of excess risk in nonparametric logistic regression.
method Unified approach for analyzing NPMLE, deriving convergence rates in Hellinger distance.
result Derives nearly optimal convergence rates for NPMLE with deep neural networks.
MPT improves CNN and energy-based models' OOD detection and generalization.
problem Challenging out-of-distribution detection in computer vision.
method Applying Maximum Probability Theorem as a regularization scheme in CNN and energy-based models.
result MPT-based regularization strategy stabilizes and improves generalization and robustness of base models.
Maximum mean discrepancy (MMD) has been widely adopted in domain adaptation to measure the discrepancy between the source and target domain distributions. Many existing domain adaptation approaches are based on the joint MMD, which is computed as the (weighted) sum of the marginal distribution discrepancy and the condi…
We assume that an individual invests in a financial market with one riskless and one risky asset, with the latter's price following geometric Brownian motion as in the Black-Scholes model. Under a constant rate of consumption, we find the optimal investment strategy for the individual who wishes to minimize the probabi…
We present asymptotic and finite-sample results on the use of stochastic blockmodels for the analysis of network data. We show that the fraction of misclassified network nodes converges in probability to zero under maximum likelihood fitting when the number of classes is allowed to grow as the root of the network size …
New method for estimating class proportions in open-set label shift data.
problem Estimating class proportions and distributions when test data includes novel classes.
method Semiparametric density ratio model framework with maximum empirical likelihood estimators and confidence intervals.
result Improved estimation accuracy and classification performance compared to existing methods.
Deep neural networks can approximate any target probability distribution given certain conditions.
problem Approximating complex probability distributions with deep neural networks.
method Proving the existence of a deep neural network mapping that approximates a target distribution under various integral probability metrics.
result Upper bounds on the size of the neural network in terms of dimension and approximation error for different metrics.
Quantum ML predicts data with improved speed and accuracy.
problem Predicting data using maximum likelihood in a quantum setting.
method Quantum states embedding and minimization of quantum relative entropy.
result Unified framework for classical and quantum LLMs with performance guarantees.
This paper proposes a method for multi-class classification problems, where the number of classes K is large. The method, referred to as Candidates vs. Noises Estimation (CANE), selects a small subset of candidate classes and samples the remaining classes. We show that CANE is always consistent and computationally effi…
Proposes a guaranteed regularization method for maximum likelihood estimation using gauge symmetry in Kullback-Leibler divergence.
problem Overfitting in maximum likelihood estimation.
method Introduces a regularization approach based on gauge symmetry in Kullback-Leibler divergence.
result The method provides a theoretically guaranteed optimal model without frequent hyperparameter tuning.
The paper reinterprets Bayesian priors and posteriors using Riemannian manifolds.
problem The dependence of maximum a posteriori estimates on parametrization.
method Assuming a Riemannian manifold with Fisher metric, the paper reinterprets priors and posteriors as distributions over probability distributions, making estimates independent of parametrization.
result A maximum a posteriori estimate independent of parametrization is defined.
One of the earliest conjectures in computational learning theory-the Sample Compression conjecture-asserts that concept classes (equivalently set systems) admit compression schemes of size linear in their VC dimension. To-date this statement is known to be true for maximum classes---those that possess maximum cardinali…
The paper proposes a method to estimate joint probability from unpaired data using entropic transport kernels.
problem Estimating joint probability from unpaired data with unknown internal ordering.
method Maximum-likelihood inference, entropic optimal transport kernels, EMML algorithm.
result The method can recover true density from empirical approximations as the number of blocks increases.
We present a novel synthesis of Fisher information and asset pricing theory that yields a practical method for reconstructing the probability density implicit in security prices. The Fisher information approach to these inverse problems transforms the search for a probability density into the solution of a differential…
MAXENT method outperforms ML in sparse data with specific prior correlations.
problem Evaluating MAXENT method's validity limits and comparing it with ML.
method Bayesian decision theory, Dirichlet density, KL distance, regularized maximum likelihood.
result MAXENT can outperform ML in sparse data with specific prior correlations.
New method improves ensemble inference for high-class tasks.
problem High inference costs for ensemble models.
method Proxy-Dirichlet target to minimize reverse KL-divergence.
result Resolves gradient issues for large-scale classification tasks.
MEP-Net uses MEP to generate solutions from limited data.
problem Generating solutions to scientific problems with incomplete information.
method Combines MEP with neural networks to learn complex distributions from moment constraints.
result Demonstrates MEP-Net's effectiveness in modeling biochemical reaction networks and generating complex distributions.
The paper compares one-hot encoding to Naïve Bayes for categorical variables.
problem Incorrect one-hot encoding affects Naïve Bayes performance.
method Mathematical and experimental analysis of PoB vs. categorical Naïve Bayes.
result Posterior probabilities are usually greater in the PoB case, but agree on the maximum a posteriori class label.
This work introduces a new metric for comparing imprecise probability models.
problem Quantifying differences between imprecise probability models.
method Integral imprecise probability metric framework based on Choquet integral.
result IIPM enables comparison across different imprecise probability models and quantifies epistemic uncertainty.
This work connects IRL methods from ML and economics.
problem Solving the inverse reinforcement learning problem.
method Shows connections and differences between various IRL methods.
result Identifies key computational and algorithmic differences.
New method optimizes non-linear functionals over probability measures.
problem Optimizing non-linear functionals defined over probability measures.
method N-particle underdamped Langevin algorithm with spacetime discretization.
result Converges globally in total variation distance.
A framework estimates categorical distributions under constraints, ensuring generality and uniqueness.
problem Estimating categorical distributions summarizing sample data under marginal constraints.
method Theoretical framework + Iterative Proportional Fitting (IPF) to estimate the distribution.
result A unique categorical distribution of Maximum Entropy under marginal constraints exists and is estimated.
Gradient flows on distributions of distributions for machine learning tasks.
problem Designing gradient flows for datasets of probability distributions.
method Representing classes as conditional distributions, modeling datasets as mixture distributions, using Wasserstein over Wasserstein (WoW) distance and gradients.
result Demonstrated gradient flows for dataset transfer and distillation tasks.
New algorithms minimize MMD to approximate probability measures efficiently.
problem Approximating probability measures by representative point sets.
method Sequential greedy minimization of maximum mean discrepancy (MMD) over candidate sets, with mini-batch variants.
result Consistency of proposed algorithms and mini-batch variants established.
The need to estimate smooth probability distributions (a.k.a. probability densities) from finite sampled data is ubiquitous in science. Many approaches to this problem have been described, but none is yet regarded as providing a definitive solution. Maximum entropy estimation and Bayesian field theory are two such appr…
We investigate activities that have different periods of duration. We define the profit intensity as a measure of this economic category. The profit intensity in a repeated trading has a unique property of attaining its maximum at a fixed point regardless of the shape of demand curves for a wide class of probability di…
New estimators improve efficiency in two-phase designs with coarsened data.
problem Efficient estimation in two-phase designs with incomplete data.
method Developed new estimators within the TMLE framework.
result New estimators are asymptotically equivalent and more efficient.
Assessing reliably the confidence of a deep neural network and predicting its failures is of primary importance for the practical deployment of these models. In this paper, we propose a new target criterion for model confidence, corresponding to the True Class Probability (TCP). We show how using the TCP is more suited…
A new method calculates optimal decisions from classifier outputs, improving predictions in drug discovery.
problem Finding optimal decisions from classifier outputs in fields like medicine.
method Develops a transducer that calculates probabilities from classifier outputs, enabling expected-utility maximization.
result Improves prediction accuracy in drug discovery problems, sometimes close to theoretical maximum.
Efficient algorithms for large Maxent models improve wildfire probability predictions.
problem Training large-scale, non-smooth Maxent models efficiently for big data.
method First-order optimization algorithms using Kullback-Leibler divergence.
result Our algorithms outperform state-of-the-art methods by one order of magnitude.
Researchers develop a method to learn CTBN structure using penalized maximum likelihood.
problem Learning the structure of continuous-time Bayesian networks (CTBNs) when the dependence structure is unknown.
method Penalized maximum likelihood method.
result The algorithm can accurately identify the network structure with high probability under certain conditions.
We give polynomial-time algorithms for the exact computation of lowest-energy (ground) states, worst margin violators, log partition functions, and marginal edge probabilities in certain binary undirected graphical models. Our approach provides an interesting alternative to the well-known graph cut paradigm in that it …
Maximizing withdrawal success in a pooled annuity fund with multiple annuitants.
problem Optimizing withdrawal success in a pooled annuity fund with homogeneous annuitants.
method Maximizing the probability of completing withdrawals until death over portfolio weight functions.
result Increasing the number of annuitants can significantly increase the maximum probability of withdrawal success.
As part of autonomous car driving systems, semantic segmentation is an essential component to obtain a full understanding of the car's environment. One difficulty, that occurs while training neural networks for this purpose, is class imbalance of training data. Consequently, a neural network trained on unbalanced data …
We formulate a new class of conditional generative models based on probability flows. Trained with maximum likelihood, it provides efficient inference and sampling from class-conditionals or the joint distribution, and does not require a priori knowledge of the number of classes or the relationships between classes. Th…
Transformers can simulate MLE for Bayesian network sequences.
problem Understanding transformers' capabilities in Bayesian network sequence generation.
method In-context maximum likelihood estimation (MLE) for autoregressive sequence generation.
result A simple transformer model can estimate Bayesian network probabilities and generate new samples.
Proposes a deep ordinal regression framework using optimal transport loss and unimodal output probabilities.
problem Lack of unimodal output probabilities in recent ordinal regression models.
method Introduces a deep learning framework based on optimal transport loss and unimodal output distribution, inspired by the Proportional Odds model.
result Demonstrates improved performance and unimodal output probabilities on real-world datasets compared to existing methods.
New method estimates Schrödinger bridges using ML techniques.
problem Finding most likely stochastic evolution between two distributions.
method Equivalence with maximum likelihood estimation, numerical Gaussian process approach.
result Direct application of ML techniques for SBP estimation.