This paper improves multi-class calibration methods using mutual information maximization-based binning.
problem Calibration of deep neural network predictions, especially for small prior classes.
method I-Max concept for binning, shared class-wise calibration strategy.
result Improves multi-class ranking and calibration performance using a small calibration set.
Two methods improve calibration of probabilistic classifiers, especially for multi-class problems.
problem Improving calibration of probabilistic classifiers, especially for multi-class problems.
method Two techniques: reduced calibration and class-wise calibration.
result Class-wise reduced calibration algorithms reduce prediction and per-class calibration errors.
New framework for evaluating multiclass classifier calibration.
problem Ensuring classifiers are well-calibrated for trustworthy predictions.
method Utility Calibration framework that measures calibration error relative to a utility function.
result Unified and robust interpretation of existing calibration metrics.
Proposes top-label calibration and M2B framework for multiclass to binary calibration.
problem Multiclass calibration and interpretation issues.
method Top-label calibration and M2B reduction framework.
result M2B + HB achieves lower calibration error than other methods.
This paper rethinks confidence calibration under covariate shifts.
problem Calibration methods struggle with covariate shifts and unstable importance weighting.
method Derives Expectation consistency condition and proposes Expectation consistency loss (ECL).
result ECL loss is compatible with various types of calibration and has the same sample complexity as ECE.
This paper tackles negative transfer in multi-task learning by introducing class-wise weights.
problem Negative transfer hampers function from achieving optimality in multi-task learning.
method Introduces class-wise weights to drive positive transfer and suppress negative transfer.
result Demonstrates improved performance in multi-task learning by reducing negative transfer.
New method regularizes deep networks by distilling self-knowledge.
problem Overfitting in deep neural networks.
method Self-knowledge distillation to regularize class-wise predictions.
result Significant improvement in generalization and calibration.
ART adapts class-wise resampling to improve imbalanced classification performance.
problem Class imbalance in classification tasks limits model performance.
method ART uses adaptive resampling based on class-wise performance metrics.
result ART consistently outperforms other methods on diverse benchmarks.
A new method for multiclass calibration using vector quantization.
problem Challenges in multiclass calibration, especially in high-stakes settings.
method Compositional approach via Vector Quantization (VQ) to learn region-specific calibration maps.
result Significant improvements in local calibration with competitive global calibration and predictive performance.
A mechanism to share risks and costs with guarantees against extreme outcomes.
problem Softening extreme individual burdens in risk sharing schemes.
method Formalizes Certified Allocation Problem; uses Conformal Risk Sharing with interpretable sharing policy and split conformal calibration.
result Reduces extreme obligations for high-risk agents while controlling harm to others.
PRISM-FCP improves federated prediction robustness against Byzantine attacks.
problem Byzantine attacks in federated learning.
method Partial model sharing and distance-based maliciousness scores.
result Maintains nominal coverage guarantees under Byzantine attacks.
Hidden cost: Smoothing shrinks decision boundaries, affecting class-wise accuracy.
problem The fragility of machine learning models and the need for robustness verification.
method Randomized smoothing approach to achieve statistical robustness.
result Smoothed classifiers' decision boundaries shrink, leading to class-wise accuracy disparity.
S-GAI initializes MLPs using spectral geometry from data, improving performance.
problem Lack of guidance on initial weights encoding data geometry.
method S-GAI uses SVD to estimate spectral class geometry, initializing MLPs from training data.
result S-GAI-initialized MLPs start from a more informative hidden state and achieve comparable accuracy.
New methods correct bias in LLM-as-a-Judge evaluations, but reliability depends on judge quality and model calibration.
problem Systematic bias in LLM-as-a-Judge evaluations using naive estimators.
method Analytical results, simulations, and real-data case study to diagnose reliability of corrected estimates.
result Corrected estimates, especially shared-calibration comparisons, can be unreliable under certain conditions.
Speech emotion recognition plays an important role in building more intelligent and human-like agents. Due to the difficulty of collecting speech emotional data, an increasingly popular solution is leveraging a related and rich source corpus to help address the target corpus. However, domain shift between the corpora p…
The paper calibrates shrinkage covariance estimators for spectral functionals in high dimensions.
problem Calibrating shrinkage covariance estimators for spectral functionals in high dimensions.
method Derives first-order null laws, distribution-free Davis-Kahan bands, and calibrated tests for spectral functionals under shrinkage.
result Calibrated tests and intervals for spectral functionals are provided, addressing the issue of estimation noise and shrinkage bias.
Statistical characteristics of deep network representations, such as sparsity and correlation, are known to be relevant to the performance and interpretability of deep learning. When a statistical characteristic is desired, often an adequate regularizer can be designed and applied during the training phase. Typically, …
FMP sampling improves model calibration without sharing data.
problem Difficulties in specifying priors for modern neural networks.
method FMP sampling replaces prior and likelihood with a predictive distribution, running the sampler centrally.
result FMP sampling significantly improves calibration over baseline methods.
Training deep neural networks is known to require a large number of training samples. However, in many applications only few training samples are available. In this work, we tackle the issue of training neural networks for classification task when few training samples are available. We attempt to solve this issue by pr…
New adversarial attack method based on deep feature distributions.
problem Adversarial attacks on CNN classifiers using output layer information.
method Modeling and exploiting class-wise and layer-wise deep feature distributions.
result Achieves state-of-the-art transfer-based attack results for undefended ImageNet models.
A new method for CT using graph-based regularization.
problem Transfer calibrations between instruments without suitable transfer standards.
method Employing manifold regularization of PLS objective to enforce invariant projections in latent variable space.
result Implicit removal of inter-device variation in predictive directions.
In this paper we study the probabilistic properties of the posteriors in a speech recognition system that uses a deep neural network (DNN) for acoustic modeling. We do this by reducing Kaldi's DNN shared pdf-id posteriors to phone likelihoods, and using test set forced alignments to evaluate these using a calibration s…
We build a multiassets heterogeneous agents model with fundamentalists and chartists, who make investment decisions by maximizing the constant relative risk aversion utility function. We verify that the model can reproduce the main stylized facts in real markets, such as fat-tailed return distribution and long-term mem…
Toy model study shows resampling/reweighting can improve feature learning in imbalanced classification.
problem Improving feature learning in imbalanced classification problems.
method High-dimensional toy model with replica method, class-wise resampling/reweighting, and simplified model.
result No resampling/reweighting can sometimes give best feature learning performance.
The paper improves classification accuracy by leveraging a shared signal across domains in high-dimensional classification.
problem Improving classification accuracy in high-dimensional data with shared signals across domains.
method Transfer learning for linear discriminant analysis, decomposing mean differences into common and domain-specific components.
result Deterministic limits for transfer performance, leading to optimal weights and corrections for bias.
Meta-learning improves Gaussian process uncertainty estimation.
problem Poor uncertainty estimation in Gaussian processes with deep kernels.
method Meta-learning to calibrate deep kernel GPs using task-specific uncalibrated and calibrated distributions.
result Improves uncertainty estimation performance with high regression performance.
New federated learning methods improve model performance on non-IID data.
problem Improving model performance on non-IID decentralized data.
method Proposed Federated AGMs using adaptive gradient methods with first-order and second-order momenta.
result The proposed Federated AGMs converge to a first-order stationary point under non-IID and unbalanced data settings for nonconvex optimization.
In this paper we propose the use of multiple local binary patterns(LBPs) to effectively classify land use images. We use the UC Merced 21 class land use image dataset. Task is challenging for classification as the dataset contains intra class variability and inter class similarities. Our proposed method of using multi-…
Model liquidity premia using a risk-sharing economy with quadratic costs.
problem Understanding the cross-section of liquidity premia earned by assets with different trading costs.
method Developed a risk-sharing economy model with quadratic transaction costs, leading to matrix-valued Riccati equations for equilibrium.
result Calibrated model to time series data, revealing liquidity premia across assets with varying trading costs.
Proposes and evaluates three diagnostic graphics for probabilistic classifiers.
problem Evaluating and comparing probabilistic classifiers.
method Triptych of diagnostic graphics: reliability diagram, ROC curve, Murphy diagram.
result Visual diagnostics reveal distinct aspects of forecast performance.
The paper tackles ride-hailing fleet repositioning with a calibrated demand approach.
problem Repositioning idle supply before future demand is observed in ride-hailing.
method A predict-then-optimize approach using calibrated demand regimes, a similarity gate, and spatial queue-regret decomposition.
result The spatial gate reduces mean wait time to 82.3s compared to 85.3s for a hand-tuned similarity gate and 85.8s for a distributional-only baseline.
FAML addresses biased evidence learning in multi-view learning, improving fairness and prediction reliability.
problem Biased evidence learning in multi-view learning, leading to unreliable uncertainty estimation.
method FAML introduces an adaptive prior and fairness constraint to balance evidence allocation across views.
result FAML enhances fairness and improves prediction reliability compared to state-of-the-art methods.
Develops a PIDE framework for option pricing with stochastic volatility and jumps.
problem Option pricing under stochastic volatility and jumps.
method PIDE framework derived from Lévy-type process, implemented via finite-difference discretization with FFT for nonlocal jump operator, calibrated using GMM.
result Stochastic volatility accounts for most pricing improvement, reducing implied-volatility RMSE by 39% compared to Black-Scholes.
MMDCP improves outlier detection and classification with adaptive prediction sets.
problem Label shift and distribution differences in multi-class settings.
method Combines distance measures with full conformal prediction for adaptive prediction sets.
result Valid coverage and effective control of class-wise false discovery rate (CW-FDR).
Packed-Ensembles improve uncertainty estimation in constrained hardware.
problem Hardware limitations restrict the size of ensembles and network capacity, degrading performance.
method Packed-Ensembles (PE) design and train lightweight structured ensembles by modulating encoding space and parallelizing into a single backbone.
result PE accurately preserves diversity and maintains performance on key metrics like accuracy, calibration, and out-of-distribution detection.
Bayesian hypergraph inference models disease pathways from EHR data.
problem Modeling rare diseases influenced by shared risk factors.
method Bayesian hypergraph inference framework reframing multi-disease modeling.
result Interpretable disease pathways and well-calibrated uncertainty quantification.
Federated learning calibrates insurance indices from renewable energy producers' data.
problem Calibrating parametric insurance indices under heterogeneous renewable energy production losses.
method Federated learning framework using Tweedie GLMs and distributed optimization.
result Federated learning recovers comparable index coefficients under moderate heterogeneity.
Paper explains learning property of logistic and softmax losses for balanced and imbalanced class data.
problem Understanding and optimizing loss functions for deep neural networks with class imbalances.
method Analyzing necessary conditions for convergence of logistic and softmax losses in CNNs.
result Proposes a novel reweighted logistic loss function that improves performance over softmax loss.
Paper proposes CCVR to improve classifier calibration in federated learning with non-IID data.
problem Improving classifier calibration in federated learning with non-IID data.
method Proposes CCVR, a simple algorithm that adjusts the classifier using virtual representations.
result CCVR achieves state-of-the-art performance on popular federated learning benchmarks.
Paper presents LSTMMDN for hourly bike flow estimation in Copenhagen.
problem Sparse or unavailable hourly bike flow data for safety analysis.
method Hybrid LSTM MDN model for hourly bike flow estimation.
result 66-77% more accurate bike flow estimates compared to calibration factors.
Unified study of out-of-distribution generalization across 172 datasets.
problem Measuring and improving robustness of transfer learning models.
method Collect and fine-tune 31k models on 172 dataset pairs, varying architectures and settings.
result In- and out-of-distribution accuracies increase jointly but their relation is dataset-dependent.
This paper proposes a new differential privacy definition using Rao distance.
problem Improving differential privacy definitions for better sequential composition.
method Using Rao distance instead of divergences of densities to define privacy.
result Proposed definition shares interpretation with previous definitions but improves sequential composition.
tsbootstrap handles time series uncertainty without assuming independence.
problem Time series data violate IID assumptions, leading to undercoverage in traditional methods.
method Provides various resampling and bootstrap methods, including classical and adaptive conformal calibration.
result Dependence-aware methods reduce coverage deficits, with sieve resampling performing best.
Conditional generative adversarial networks (cGANs) have gained a considerable attention in recent years due to its class-wise controllability and superior quality for complex generation tasks. We introduce a simple yet effective approach to improving cGANs by measuring the discrepancy between the data distribution and…
How to reconcile the classical Heston model with its rough counterpart? We introduce a lifted version of the Heston model with n multi-factors, sharing the same Brownian motion but mean reverting at different speeds. Our model nests as extreme cases the classical Heston model (when n = 1), and the rough Heston model (w…
We present an algorithm for computing class-specific universal adversarial perturbations for deep neural networks. Such perturbations can induce misclassification in a large fraction of images of a specific class. Unlike previous methods that use iterative optimization for computing a universal perturbation, the propos…
In Domain Adaptation (DA), the category-relevant losses usually occupy a dominant position, while they are usually built with hard or soft labels in existing models. We observed that hard labels are overconfident due to hard samples existed, and soft labels are ambiguous as too many small noisy probabilities involved, …
The paper develops a method to infer model parameters and shared dynamics from related physical systems using data.
problem Calibrating models to match data when detailed system properties and laws are unknown.
method Hierarchical Bayesian framework, adaptive surrogate models, bilevel optimization.
result Joint estimation of individual model parameters and shared dynamics using data from related systems.