Study calculates Bayes risk for semi-supervised learning with uncertain labels.
problem Uncertain labeling in semi-supervised classification.
method Gaussian mixture model, Bayes risk computation, comparison with algorithm performance.
result New insights into semi-supervised learning algorithm performance.
Lower bounds on Bayes risk for realizable models derived using information theory.
problem Deriving lower bounds on Bayes risk for realizable machine learning models.
method Information-theoretic analysis using rate-distortion theory and mutual information.
result Lower bounds on Bayes risk for realizable models, matching known bounds up to logarithmic factors.
Labels define the effective timescale for learning from short observations.
problem Learning from short observations with aggregated labels.
method Analytical and Monte Carlo methods to study label variance and effective timescales.
result Labels define the effective timescale for learning, distinguishing architectural from protocol limits.
The study calculates the risk of semi-supervised multitask learning on Gaussian mixtures.
problem Understanding the risk in semi-supervised multitask learning on Gaussian mixtures.
method Statistical physics methods applied to Gaussian mixture models.
result The study evaluates the performance gain of learning tasks together versus separately.
Gradient descent converges to minimum Bayes risk for two-layer ReLU networks in mean field regime.
problem Training two-layer ReLU networks using gradient descent in the mean field regime.
method Describes a condition for convergence to minimum Bayes risk, extending previous results to ReLU-activated networks.
result The condition for convergence does not depend on initialization and concerns weak convergence of network realization.
New estimator reduces risk in slate bandits by leveraging Bayes risk criterion.
problem Evaluating slate policies using logged data when policies factorize over slots.
method Developed a new estimator using a control variate approach, showing risk improvement over existing methods.
result The new estimator has lower risk than the pseudoinverse estimator in slate bandit problems.
The study compares clustering risk in Hidden Markov and i.i.d. models, showing the Bayes classifier is nearly optimal.
problem Comparing clustering risk in Hidden Markov and i.i.d. models.
method Analysis of Bayes risk, theoretical bounds, and simulations.
result The Bayes classifier is nearly optimal for clustering in both Hidden Markov and i.i.d. models.
A new variational method with statistical guarantees for Bayesian inference.
problem Improving variational inference with provable statistical guarantees.
method Introducing α-Variational Inference (α-VB) with statistical guarantees. result The α-VB method provides optimal convergence rates for parameter estimates. Distance weighted discrimination (DWD) is a margin-based classifier with an interesting geometric motivation. DWD was originally proposed as a superior alternative to the support vector machine (SVM), however DWD is yet to be popular compared with the SVM. The main reasons are twofold. First, the state-of-the-art algor…
New algorithm reduces bias in trained models, near-optimal performance proven.
problem Reduction of bias in trained machine learning models.
method Scalable post-processing algorithm for debiasing trained models, including deep neural networks (DNNs).
result Proven to be near-optimal by bounding its excess Bayes risk.
Deep learning detects novel changes in time series data.
problem Detecting novel changes in time series with unknown probability structures.
method Causally extracts an innovations sequence for novelty detection.
result Minimax optimality established for the novelty detection method.
Improved model accuracy can reduce overall user accuracy in competitive markets.
problem The impact of model competition on overall user accuracy.
method Defined a model of competition for classification tasks and used data representations to study the effect of scale.
result Improving data representation quality can decrease overall predictive accuracy across users (social welfare) in a competitive market.
Paper improves risk estimation for extreme events.
problem Estimating extreme risks accurately.
method Modified Bayes risk for expectiles, asymptotic expansions, efficient estimators.
result Asymptotic normality of estimators proved.
New method certifies deep graph classifiers with tighter risk bounds.
problem Certifying the reliability of deep graph classifiers.
method Linearized deep assignment flows with random initial conditions, using PAC-Bayes risk certification.
result Computes tighter out-of-sample risk certificates efficiently.
Optimal search for change point anomaly in multiple processes.
problem Detecting a change point in an anomalous process among multiple normal processes.
method Deterministic search algorithm balancing sample complexity and detection accuracy.
result Asymptotically optimal in minimizing Bayes risk.
MARLA uses deep reinforcement learning for multi-agent AHT, reducing Bayes risk.
problem Decentralized active hypothesis testing among multiple agents.
method MARLA algorithm using deep multi-agent reinforcement learning.
result MARLA outperforms single-agent learning in AHT problems.
The NL score optimizes speaker recognition tasks.
problem Improving speaker recognition accuracy.
method Established the theory of optimal scores based on normalized likelihood.
result NL score is equivalent to PLDA likelihood ratio under certain conditions.
A statistical decision problem is hidden in the core of option pricing. A simple form for the price C of a European call option is obtained via the minimum Bayes risk, R_B, of a 2-parameter estimation problem, thus justifying calling C Bayes (B-)price. The result provides new insight in option pricing, among others obt…
Paper proposes a method to estimate total variation distance for synthetic data fidelity.
problem Assessing the fidelity of synthetic data generated by AI.
method Discriminative approach to estimate total variation distance between two distributions.
result Estimation of total variation distance reduces to quantifying Bayes risk in classification.
Study generalizes property elicitation to imprecise probabilities.
problem Minimizing risk over imprecise probability distributions.
method Maximin risk minimization over a set of imprecise probabilities.
result Conditions for elicitability of IP-properties.
The study provides statistical theory for WGANs in time series forecasting.
problem Statistical analysis of WGANs for time series forecasting.
method Statistical theory and upper bounds for excess Bayes risk, weak convergence, and confidence intervals.
result Developed confidence intervals for time series forecasting using WGANs.
Generalizes underlap coefficient for multivariate group separation.
problem Quantifying distributional separation across groups in statistical learning.
method Generalizes underlap coefficient (UNL) to multivariate settings, studies its relationship with Bayes risk and mutual information, proposes an efficient importance sampling estimator.
result UNL as a measure of dependence between group labels and variables of interest, interpretable measure of partition-covariate dependence in clustering.
New criterion assesses cluster separability for validation.
problem Validating cluster analysis results and determining the number of clusters.
method Distinguishability criterion, combined loss function-based framework.
result Validated cluster configurations and determined the number of clusters.
Three LF training criteria improve neural network acoustic models without cross-entropy pre-training.
problem Improving purely sequence-trained neural network acoustic models.
method Comparison of three lattice-free discriminative training criteria (MMI, bMMI, sMBR) on LVCSR tasks.
result LF-bMMI models outperform plain LF-MMI models by 5% WER on Switchboard datasets.
A new procedure for learning cost-sensitive SVM(CS-SVM) classifiers is proposed. The SVM hinge loss is extended to the cost sensitive setting, and the CS-SVM is derived as the minimizer of the associated risk. The extension of the hinge loss draws on recent connections between risk minimization and probability elicitat…
Paper calculates the benefit of unlabeled data in semi-supervised learning for Gaussian mixtures.
problem Improving performance in semi-supervised learning with unlabeled data.
method Analytical computation of Bayes risk gap between supervised and semi-supervised approaches for Gaussian mixture models.
result Quantifies the accuracy increase due to unlabeled data in a Bayesian setting.
We consider composite loss functions for multiclass prediction comprising a proper (i.e., Fisher-consistent) loss over probability distributions and an inverse link function. We establish conditions for their (strong) convexity and explore the implications. We also show how the separation of concerns afforded by using …
Study shows MDA's effectiveness even when more components are assumed than in actual data.
problem Classification error in overspecified Mixture Discriminant Analysis.
method Two-component Gaussian mixture model, EM algorithm, theoretical analysis of convergence and error rates.
result EM algorithm converges exponentially fast to Bayes risk with suitable initialization.
Develops a new learning framework for dynamic data.
problem Poor performance of existing strategies in dynamic data and goals.
method Prospective Learning framework and Prospective ERM algorithm.
result Prospective ERM converges to Bayes risk under certain assumptions.
Develops a comprehensive theory of corruption in supervised learning.
problem Widespread corruption in data collection affects supervised learning problems.
method Introduces a general theory of corruption using Markov kernels, distinguishing and comparing corruption types.
result Establishes a unified framework for corruption types and develops mitigation strategies.
Paper examines M-SVM for multi-task learning, showing reliability and pre-convergence-rate factor improvements.
problem Whether MTL always provides reliable results and how MTL outperforms independent learning.
method Regularized multi-task learning (MTL) based on SVM models (M-SVM).
result M-SVM is Bayes risk consistent in large sample size, improving pre-convergence-rate factor (PCR) for small data.
New method reduces computational cost for estimating PAC-Bayes bounds.
problem High computational cost in estimating PAC-Bayes bounds.
method General alternative method that makes computational savings.
result Reduces computational cost on the order of the dataset size.
Study on decision-making cascades with agents having varying beliefs and noise levels.
problem Optimizing decision-making in a cascade of agents with heterogeneous beliefs and noise.
method Recursive belief update and analysis of optimal decision rules, predecessor selection problem characterization.
result Optimal decisions can deviate from true prior beliefs in certain conditions, highlighting the importance of social learning.
A new classifier uses linear programming to classify sets based on their covariance.
problem Classifying sets of observations as a whole, not individually.
method Proposes a new classifier, CLIPS, using linear programming for set classification.
result The CLIPS classifier performs better with multiple observations in a set.
Method minimizes total cost of classification by acquiring covariates efficiently.
problem Minimizing total cost of classification in applications with covariate acquisition costs.
method Formalizes optimization goal using Bayes risk, introduces assumptions for computable solution.
result Proposed method achieves lowest total costs compared to previous methods on medical datasets.
NeuMiss networks tackle supervised learning with missing values, offering efficient and robust predictions.
problem Challenges in supervised learning with missing values, especially when the response is a linear function of the complete data.
method Derive analytical form of optimal predictor under linearity assumption and various missing data mechanisms. Propose NeuMiss networks using multiplication by missingness indicator.
result Upper bound on Bayes risk and good predictive accuracy with independent complexity of missing data patterns.
Paper proposes C-STM for multimodal neuroimaging data classification.
problem Multimodal neuroimaging data fusion for better classification.
method Coupled Support Tensor Machine (C-STM) using latent factors from ACMTF.
result C-STM achieves better classification performance than single-mode classifiers.
Develops kernel machines for missing response data.
problem Missing responses in data.
method Proposes kernel machine families for handling missing responses, including doubly-robust estimators.
result Oracle inequalities and consistency proved for kernel machine estimators.
New insights into empirical Bayes and compound decision problems with improved regret bounds.
problem Estimating means of normally or Poisson distributed vectors under squared loss.
method Combines Bayesian and frequentist approaches using data-driven estimators.
result Optimal regret bounds for Poisson and normal mean models, resolving conjectures.
A new sequential method estimates Poisson means in streaming data, achieving optimality and efficiency.
problem Estimating Poisson means in a streaming, or online, framework.
method A quasi-Bayesian approach based on Newton's algorithm for a sequential estimate.
result Established frequentist guarantees including consistency and asymptotic optimality.
New measures generalize existing ones, linking information and risk.
problem Linking information measures and risk in statistical decision problems.
method Introducing new families of divergence measures and deriving an information processing equality.
result Extension of variational φ-divergence representation to multiple distributions. This paper discovers new identities linking geodesic and orthogeodesic lengths on hyperbolic surfaces.
problem Understanding relationships between geodesic and orthogeodesic lengths on hyperbolic surfaces.
method Investigates a broad family of identities involving lengths of all closed geodesics and orthogeodesics.
result Introduces new identities that include lengths of all closed geodesics, contrasting with previous identities.
New algorithm improves regression error bounds and accelerates performance for low noise.
problem Nonparametric least square regression in RKHS with optimal error bounds.
method Kernel Truncated Randomized Ridge Regression (KTRRR) with optimal generalization error bounds.
result Faster finite-time and asymptotic rates on low noise problems.
Establishes a correspondence between two mathematical identities.
problem None explicitly stated; focuses on identity correspondence.
method Establishes correspondence between Pestov and Weitzenböck identities.
result Established correspondence between Pestov and Weitzenböck identities.
A property, or statistical functional, is said to be elicitable if it minimizes expected loss for some loss function. The study of which properties are elicitable sheds light on the capabilities and limitations of point estimation and empirical risk minimization. While recent work asks which properties are elicitable, …
We consider the problem of transforming samples from one continuous source distribution into samples from another target distribution. We demonstrate with optimal transport theory that when the source distribution can be easily sampled from and the target distribution is log-concave, this can be tractably solved with c…
A Hilbert space embedding for probability measures has recently been proposed, wherein any probability measure is represented as a mean element in a reproducing kernel Hilbert space (RKHS). Such an embedding has found applications in homogeneity testing, independence testing, dimensionality reduction, etc., with the re…
Statistical machine learning plays an important role in modern statistics and computer science. One main goal of statistical machine learning is to provide universally consistent algorithms, i.e., the estimator converges in probability or in some stronger sense to the Bayes risk or to the Bayes decision function. Kerne…