Study shows label errors impact model disparity metrics, proposing mitigation methods.
problem Impact of label errors on model disparity metrics.
method Empirical study, characterizing label error effects; proposing estimation and relabeling methods.
result Label errors significantly affect model disparity metrics, particularly for minority groups.
Metric learning enhances combinatorial coverage metrics' ability to predict classification errors.
problem Dataset dependence of combinatorial coverage metrics in anticipating classification errors.
method Metric learning to improve latent space separation of data classes.
result Metric learning increases SDCCMs' ability to distinguish between correctly and incorrectly classified data.
Paper proposes a robust metric learning algorithm.
problem Robustness of metric learning against adversarial perturbations is insufficient.
method Proposes a novel Mahalanobis distance metric learning algorithm.
result Certifiable robustness improvement over Euclidean distance.
Unified NICEk metrics improve solar forecasting accuracy.
problem Lack of suitable error metrics for multidimensional solar irradiance forecasting.
method Introducing NICEk framework with Lk norms for evaluating forecasting models.
result NICESigma consistently outperforms traditional metrics in discriminative power and statistical significance.
A new metric optimizes forecasts for lumpy, intermittent demand.
problem Inaccurate demand forecasts lead to suboptimal logistics and production.
method Developed a novel metric that considers both statistical and business aspects.
result The new metric yields more accurate predictions for lumpy and intermittent demand.
Corrects errors in previous studies on metric measure spaces.
problem Identifies and corrects errors in previous research on metric measure spaces.
method Corrigendum to previous publications in Acta Math, JFA.
result Corrects errors in previous studies on metric measure spaces.
New decision-theoretic calibration error metric improves prediction reliability.
problem Improving the reliability of predictions for decision-making.
method Proposed Calibration Decision Loss (CDL) and an efficient algorithm to achieve near-optimal CDL.
result Near-optimal CDL guarantees vanishing payoff loss from miscalibration.
Paper proves polynomial equivalence of quantum complexity metrics.
problem Quantum complexity metrics equivalence.
method Study of right-invariant metrics on unitary group.
result All metrics in the equivalence class have polynomial slowdown in approximation.
A new KM clustering algorithm reduces error and scales to large datasets.
problem Traditional KM error analyses suffer from generalization gaps and lack true error bounds.
method Formalized true K-Medoids error, decomposed into ME and MME, provided convergence result, proposed MCPAM algorithm.
result MCPAM achieves true error bounds and scales to 1 billion points.
The most important aspect of any classifier is its error rate, because this quantifies its predictive capacity. Thus, the accuracy of error estimation is critical. Error estimation is problematic in small-sample classifier design because the error must be estimated using the same data from which the classifier has been…
We study Lipschitz models in reinforcement learning to bound prediction and value-function errors.
problem Bounding errors in reinforcement learning models with Lipschitz continuity constraints.
method We provide bounds on multi-step prediction error and value-function estimate using the Wasserstein metric for Lipschitz models.
result Lipschitz models lead to bounded errors in prediction and value-function estimates.
The paper explores learning metrics in low dimensions with bounds and complexities.
problem Learning metrics in low dimensions with bounds and complexities.
method Develops upper and lower bounds on generalization error, quantifies sample complexity, and bounds accuracy relative to the true metric.
result Novel mathematical approaches to metric learning and insights into ordinal embedding.
The paper clusters sequences from unknown distributions using k-medoids.
problem Clustering sequences from unknown composite distributions.
method k-medoids algorithm for sequences with composite distributions.
result Error probability decreases exponentially with increasing sample size.
A new framework for semi-supervised ordinal regression.
problem Lack of evaluation metrics and theoretical guarantees in existing semi-supervised ordinal regression.
method Empirical risk minimization principle, flexible model choices, and estimation error bound.
result Consistent risk estimator and improved performance across various metrics.
This paper tightens the generalization error bound for graph embedding in non-Euclidean spaces.
problem High generalization error in non-Euclidean graph embedding, preventing practical applications.
method Novel upper bound of graph embedding's generalization error using local Rademacher complexity.
result The new bound is tighter and faster, allowing better performance in non-Euclidean spaces.
A visualization aids in comparing regression models by highlighting errors and correlations.
problem Comparing regression models is difficult due to varying hyper-parameters and metrics.
method Introduces a novel visualization approach using 2D residual space, Mahalanobis distance, and colormaps.
result Enhanced understanding of regression model performance differences and error distributions.
Paper extends Bayesian Cramér-Rao bound with geometric considerations.
problem Estimation of covariance matrices with geometric structures.
method Intrinsic Bayesian Cramér-Rao bound with Riemannian geometry.
result Performance bounds for covariance matrix estimation.
Synthetic data augmentation can improve imbalanced classification metrics.
problem Improving imbalanced classification metrics
method Developing a framework for analyzing the effects of synthetic data augmentation on score-based classification
result Augmentation can improve AUROC, AUPRC, balanced accuracy, and F1 score
TCE measures calibration error with a test-based approach.
problem Measuring calibration error of probabilistic binary classifiers.
method TCE uses a novel loss function based on a statistical test.
result TCE offers clear interpretation, consistent scale, and enhanced visual representation.
New metrics improve scRNA-seq perturbation modeling by reducing mode collapse.
problem Outperformed by simple mean prediction in scRNA-seq perturbation modeling.
method Introduce DEG-aware metrics (WMSE, Rw2(Δ)) and negative/positive baselines. result WMSE loss function reduces mode collapse and improves model performance.
Study classifies and typologizes performance metrics for ML regression, forecasting, and prognostics.
problem Improving evaluation of machine learning models in regression, forecasting, and prognostics.
method Analysis of existing metrics and development of a typology based on structure and properties.
result Proposed framework of four categories of metrics and three key components determining their structure.
We develop coreset techniques for noisy clustering with provable guarantees.
problem Clustering with stochastic noise in datasets.
method Surrogate error metrics and coreset construction algorithm.
result Improved coreset size and better guarantees on true clustering cost.
Improves bit error tolerance in RRAM-based BNNs without overfitting.
problem Bit errors in RRAM-based BNNs reduce accuracy and overfit to training error rates.
method Proposes straight-through gradient approximation and a novel regularizer.
result Improves BNNs' robustness to bit errors without overfitting.
Paper studies SGD stability and optimization error in pairwise learning.
problem Stability and optimization error of SGD for pairwise learning.
method Established stability and optimization error trade-offs for SGD in convex, strongly convex, and non-convex settings.
result Lower bounds for SGD optimization error and excess expected risk.
Nonasymptotic error bounds and strong consistency rates for survival analysis methods.
problem Establishing reliable error bounds and consistency rates for survival analysis methods.
method Nonasymptotic error bounds for Kaplan-Meier-based nearest neighbor and kernel survival probability estimators in metric spaces.
result Rates of strong consistency match existing lower bounds for conditional CDF estimation.
The study categorizes reward errors in reinforcement learning, finding some can be beneficial.
problem Training language models with imperfect proxy rewards.
method Theoretical analysis of policy gradient optimization and categorization of reward errors.
result Reward errors can be benign or even beneficial, preventing policy from stalling.
Reference metrics are used to define the differential structure on multicube representations of manifolds, i.e., they provide a simple and practical way to define what it means globally for tensor fields and their derivatives to be continuous. This paper introduces a general procedure for constructing reference metrics…
This paper corrects errors in UMAP's derivation and explains its properties.
problem Errors in UMAP's derivation by McInnes et al.
method Full derivation of Spivak's functors and McInnes et al.'s finite variant.
result Corrected errors and provided an explicit description of the metric realization.
Algorithms learn and test variable partitions in various groups and error metrics.
problem Learning and testing variable partitions in different groups and error metrics.
method Algorithms for agnostically learning and testing k-partitionability over various groups and error metrics. result Learning algorithms for k-partitionability with polynomial time complexity and testing with adaptive queries. This study analyzes how well GANs approximate distributions from small samples.
problem Understanding how well GANs approximate distributions from limited data.
method Analysis of GANs using integral probability metrics and Hölder classes.
result GANs can adaptively learn low-dimensional structures or Hölder densities.
New metrics boost A/B-test power by up to 210%.
problem High cost and type-II errors in A/B-tests.
method Learn metrics from short-term signals to maximize power.
result Statistical power increased by up to 210%.
New framework optimizes random forest parameters for stability and cost.
problem Optimizing random forest parameters for industrial applications.
method Bayesian optimization framework considering error, stability, and cost.
result Parameter settings that balance error, stability, and cost.
Metrics assess uncertainty structure and distribution for regression models.
problem Quantifying uncertainty in high-dimensional and nonlinear regression tasks.
method Two bounded comparison metrics for uncertainty structure and distribution.
result DNNs and DNOs provide encouraging uncertainty metric values in high dimensions.
A new method uses coherent structure coloring to estimate model parameters more accurately than traditional methods.
problem Challenges in estimating model parameters from Lagrangian data in turbulent flows.
method Use coherent structure coloring (CSC) field to assess model skill and estimate model parameters.
result Error in the CSC field can accurately determine model parameters, while conventional methods fail.
Establishes upper bounds on generalization error in active learning.
problem Improving query algorithms in active learning.
method Derives upper bounds on generalization error using informativeness and representativeness query strategies.
result Validates the use of regularization techniques to ensure bounds' validity.
Reassesses calibration metrics in machine learning models.
problem Inconsistent reporting of calibration metrics in recent literature.
method Calibration-based decomposition of Bregman divergences, visualization of calibration and generalization error.
result New visualization technique for detecting trade-offs between calibration and generalization.
We derive a mapping between MSE and CCC, revealing counterintuitive insights.
problem Missing mapping between mean square error and concordance correlation coefficient.
method Derive mathematical formula connecting MSE and CCC, analyze graphical implications.
result Formula uncovers counterintuitive insights and precise range for CCC given MSE.
The paper develops a method to approximate arbitrary Bregman divergences from supervision.
problem Approximating an arbitrary Bregman divergence from supervision.
method Develops a formulation and algorithm for learning arbitrary Bregman divergences by approximating their convex generating function via a piecewise linear function.
result The method achieves a generalization error of Op(m−1/2) for metric learning, matching known bounds. A new metric predicts model performance on unseen data.
problem Predicting performance on out-of-distribution data without labels.
method Uses model predictions to pseudo-label data, trains a new model, and measures difference from in-distribution models.
result Empirically outperforms existing methods on image and text classification tasks.
Paper introduces a new framework to improve sample efficiency in POMDPs learning.
problem Challenges in off-policy evaluation for POMDPs, especially with hidden states.
method Exploits the metric structure of belief space to relax coverage assumptions.
result Unified analysis technique yields tighter error bounds and sample efficiency improvements.
Efficiently identifies stable manifolds from streaming data.
problem Learning reliable manifolds from streaming data is computationally expensive.
method Presented error metrics and S-Isomap algorithm for efficient manifold learning.
result Identifies the transition point for stable manifold learning.
Normalizing Flows improve prediction interval efficiency in CP.
problem Inefficient prediction intervals in CP due to non-uniform error distribution.
method Train a Normalizing Flow to optimize the distance metric between errors and inputs.
result Optimized prediction intervals are more efficient and valid.
New metric reduces estimation error in survival model evaluation.
problem Dependent censoring complicates survival model evaluation.
method Dependent Brier score based on Archimedean copula and Copula-Graphic estimator.
result Reduces estimation error by 12-16% on average.
Paper proposes a new metric to evaluate survival models, especially for censored data.
problem Challenges in evaluating survival prediction models due to censored data.
method Developed a novel approach to estimate Mean Absolute Error (MAE) for survival datasets with censored data.
result The proposed MAE metric using pseudo-observations accurately ranks model performance and closely matches true MAE.
Improved robustness in multivariate regression and classification with DRO under Wasserstein metric.
problem Outliers in covariates and responses.
method Distributionally Robust Optimization (DRO) with Wasserstein metric ambiguity set and regularization.
result Significant improvement in predictive error and robustness.
New pruning method breaks power law scaling, potentially reducing error to exponential.
problem Improving neural network performance through scaling alone is costly.
method Developed a new data pruning metric to break power law scaling.
result Pruned datasets show better than power law scaling on various image datasets.
We show that on every compact spin manifold admitting a Riemannian metric of positive scalar curvature Friedrich's eigenvalue estimate for the Dirac operator can be made sharp up to an arbitrarily small given error by choosing the metric suitably.
We study the Lipschitz metric on Teichmuller space (defined by Thurston) and compare it with the Teichmuller metric. We show that in the thin part of Teichmuller space the Lipschitz metric is approximated up to bounded additive distortion by the sup metric on a product of lower-dimensional spaces (similar to the Teichm…