Study validates metrics for offline MBO using diffusion models.
problem Evaluate metrics for offline MBO without ground truth oracle.
method Propose and quantify validation metrics over datasets.
result Identify most effective validation metrics.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,694 papers · 148 categories
Trend · papers per month
48 results for “validation metrics”
CAI automates extraction and validation of corporate GHG emission metrics.
problem Manual extraction of corporate GHG emission metrics is labor-intensive and error-prone.
method CAI uses LLMs to automate extraction and validation of metrics from corporate disclosures.
result CAI improves data collection efficiency and accuracy by automating the process.
New method uses interval-based metric to validate prediction uncertainty in machine learning.
problem Validation of prediction uncertainty in machine learning regression tasks is unreliable due to heavy-tailed distributions.
method Shift from variance-based metrics to interval-based Prediction Interval Coverage Probability (PICP).
result PICP method more quickly and reliably tests prediction intervals than variance-based metrics.
Validation of uncertainty quantification metrics: a primer based on the consistency and adaptivity conceptsphysics.chem-ph
This study revisits UQ validation methods based on consistency and adaptivity concepts.
problem Lack of comprehensive validation methods for UQ metrics across input feature ranges.
method Revisit and extend common validation methods for UQ metrics based on consistency and adaptivity concepts.
result Improved understanding and capabilities of UQ metrics validation methods.
The paper offers guidelines for validating data-driven models.
problem Ensuring reliable validation of data-driven models.
method A set of general rules for model validation.
result Helps practitioners create reliable validation plans and report results transparently.
We show that many standard results of Lorentzian causality theory remain valid if the regularity of the metric is reduced to . Our approach is based on regularisations of the metric adapted to the causal structure.
metric-learn is an open source Python package implementing supervised and weakly-supervised distance metric learning algorithms. As part of scikit-learn-contrib, it provides a unified interface compatible with scikit-learn which allows to easily perform cross-validation, model selection, and pipelining with other machi…
Active learning method improves local model validity estimation.
problem Ensuring local model validity in machine learning applications.
method Learning model error to estimate local validity using active learning.
result The proposed method can estimate local validity with a small amount of data.
We prove that `volume cone implies metric cone' in the setting of RCD spaces, thus generalising to this class of spaces a well known result of Cheeger-Colding valid in Ricci-limit spaces.
Introduces PCG for better counterfactual explanations in vision models.
problem Ambiguity in latent-space optimization methods for counterfactual explanations.
method Constructs counterfactuals by tracing geodesics under a perceptually Riemannian metric.
result PCG outperforms baselines and reveals hidden failure modes.
Adapts example weights to optimize black-box metrics.
problem Optimizing metrics defined by black-box functions.
method Adaptive example weighting and iterative post-shifting.
result Improves classification performance compared to baselines.
In this paper two metric properties on geodesic length spaces are introduced by means of the metric projection, studying their validity on Alexandrov and Busemann NPC spaces. In particular, we prove that both properties characterize the non-positivity of the sectional curvature on Riemannian manifolds. Further results …
Analyzes metric spaces homeomorphic to manifolds, proving rigidity and inequalities.
problem Analyzing metric spaces homeomorphic to manifolds.
method Geometric and analytic approaches, proving existence of integral currents, establishing rigidity and inequalities.
result Metric manifolds admit non-trivial integral currents and satisfy isoperimetric inequalities.
Study validates Michor-Mumford conjecture in infinite dimensional Hilbertian H-type groups.
problem Validation of Michor-Mumford conjecture in infinite dimensional Hilbertian H-type groups.
method Introduced infinite dimensional Hilbertian H-type groups with weak, graded, left invariant Riemannian metrics and proved the vanishing of geodesic distance and local unboundedness of sectional curvature.
result Validation of Michor-Mumford conjecture linking geodesic distance vanishing to local unboundedness of sectional curvature.
Study evaluates synthetic data augmentation for small datasets, highlighting inconsistencies in traditional metrics.
problem Inconsistent validation of synthetic data generated for small sample sizes.
method Proposes a normalized Bottleneck distance metric to evaluate synthetic tabular data.
result Common metrics like propensity scoring and MMD fail for small datasets, showing instability and high variability.
We investigate the validity of the isometry extension property for (Riemannian) Einstein metrics on manifolds with boundary. Given a metric on the boundary, this is the issue of whether any Killing field of the boundary metric extends to a Killing field of any bulk or filling Einstein metric inducing the given data on …
Machine learning (especially reinforcement learning) methods for trading are increasingly reliant on simulation for agent training and testing. Furthermore, simulation is important for validation of hand-coded trading strategies and for testing hypotheses about market structure. A challenge, however, concerns the robus…
The ability to learn disentangled representations that split underlying sources of variation in high dimensional, unstructured data is important for data efficient and robust use of neural networks. While various approaches aiming towards this goal have been proposed in recent times, a commonly accepted definition and …
Causal ML methods failed to validate their personalized treatment effects in two large trials.
problem Validating causal machine learning methods for personalized treatment effects in precision medicine.
method Assessed 17 mainstream causal heterogeneity ML methods using two large randomized controlled trials.
result None of the ML methods reliably validated their performance, internal or external, showing significant discrepancies between training and test data.
The goal of the paper is to study the angle between two curves in the framework of metric (and metric measure) spaces. More precisely, we give a new notion of angle between two curves in a metric space. Such a notion has a natural interplay with optimal transportation and is particularly well suited for metric measure …
This paper proposes a new AED framework for multi-metric experiments with fixed budget.
problem Statistical power challenges in testing multiple metrics simultaneously.
method Two-phase structure: adaptive exploration followed by validation. SHRVar algorithm with relative-variance-based sampling.
result Achieves provable error probability that decreases exponentially.
Improves test set performance and reduces out-of-sample disappointment for unstable models.
problem Ensuring strong test set performance via cross-validation for unstable models.
method Nested k-fold cross-validation with hyperparameter selection based on a weighted sum of cross-validation metric and model stability measure.
result Improves out-of-sample MSE for sparse ridge regression and CART by 4% and 2% respectively, compared to k-fold cross-validation.
Study reveals gaps between simulated and real-world treatment effect evaluation metrics.
problem Evaluation of treatment effect estimation models differs between academic and practical settings.
method Comprehensive empirical study comparing semi-simulated benchmarks and real-world datasets.
result Counterfactual metrics do not reliably predict observable metrics, and rankings from simulated benchmarks do not generalize to real-world data.
Real-world machine learning applications often have complex test metrics, and may have training and test data that are not identically distributed. Motivated by known connections between complex test metrics and cost-weighted learning, we propose addressing these issues by using a weighted loss function with a standard…
Probabilistic fair clustering tackles uncertain group membership.
problem Fair clustering with imperfect group membership.
method Probabilistic algorithms for metric graphs and metric membership.
result Approximation ratio guarantees for fair clustering.
Proposes a neural network loss function for better uncertainty estimation.
problem Challenges in estimating predictive uncertainty of neural networks.
method Bayesian Validation Metric (BVM) framework with ensemble learning.
result Competitive and robust uncertainty estimation on in-distribution and out-of-distribution data.
We extend the validity of the Penrose singularity theorem to spacetime metrics of regularity . The proof is based on regularisation techniques, combined with recent results in low regularity causality theory.
Non-Archimedean balanced metrics and their application to totally degenerate abelian varietiesmath.AG
Non-Archimedean balanced metrics approximate cscK metrics for totally degenerate abelian varieties
problem Non-Archimedean balanced metrics for polarized abelian varieties
method Non-Archimedean analogue of the cscK metric
result Uniform estimate for Calabi-Yau metrics on fibers
Normalizing Flows improve prediction interval efficiency in CP.
problem Inefficient prediction intervals in CP due to non-uniform error distribution.
method Train a Normalizing Flow to optimize the distance metric between errors and inputs.
result Optimized prediction intervals are more efficient and valid.
Counterfactual Cross-Validation: Stable Model Selection Procedure for Causal Inference Modelsstat.ML
We study the model selection problem in conditional average treatment effect (CATE) prediction. Unlike previous works on this topic, we focus on preserving the rank order of the performance of candidate CATE predictors to enable accurate and stable model selection. To this end, we analyze the model performance ranking …
Study weak geodesic lines in Kähler metric space, disproving a conjecture.
problem Disproving a conjecture about weak geodesic lines in Kähler metrics.
method Establish Ross-Witt Nyström correspondence, construct weak geodesic lines.
result Some weak geodesic lines are smooth, disproving a popular conjecture.
Paper proves Hermitian-Yang-Mills metrics for stable Kähler bundles.
problem Existence of Hermitian-Yang-Mills metrics for Kähler vector bundles.
method Analyzes slope polystability and uses Kähler currents with singularities.
result Validates existence of Hermitian-Yang-Mills metrics for stable bundles and nef-big currents.
MD tree diagnoses model failures using loss landscape metrics.
problem Diagnose model failures without knowing training configuration.
method MD tree based on loss landscape metrics.
result MD tree achieves 87.7% accuracy in dataset transfer tasks, outperforming validation-based approaches.
Design of experiments improves validation of biomolecular networks.
problem Efficiently validate non-machine learning designed biomolecular networks.
method Use Gaussian processes and Bayesian optimization to select experimental points.
result Developed a stopping criterion based on discrepancy metric and uncertainty.
Deep Neural Networks(DNN) have excessively advanced the field of computer vision by achieving state of the art performance in various vision tasks. These results are not limited to the field of vision but can also be seen in speech recognition and machine translation tasks. Recently, DNNs are found to poorly fail when …
Machine learning methods may have the potential to significantly accelerate drug discovery. However, the increasing rate of new methodological approaches being published in the literature raises the fundamental question of how models should be benchmarked and validated. We reanalyze the data generated by a recently pub…
We propose a novel classifier accuracy metric: the Bayesian Area Under the Receiver Operating Characteristic Curve (CBAUC). The method estimates the area under the ROC curve and is related to the recently proposed Bayesian Error Estimator. The metric can assess the quality of a classifier using only the training datase…
Unveiling the Potential of Robustness in Selecting Conditional Average Treatment Effect Estimatorscs.LG
Paper introduces DRM for selecting robust CATE estimators.
problem Selecting CATE estimators without counterfactual outcomes.
method Distributionally Robust Metric (DRM) for CATE estimator selection.
result DRM selects robust CATE estimators robust to distribution shift.
The paper broadens a mathematical correspondence to include more balanced metrics.
problem Extending a mathematical correspondence to a broader class of metrics.
method Using key observations and known theorems to apply results to a new class of metrics.
result The known results can be applied to a larger class of metrics, including those arising from multipolarizations.
New metrics predict human sentence comprehension across languages.
problem Predicting human sentence comprehension using computational models.
method Developed sentence-level metrics using multilingual large language models.
result Achieved high accuracy in predicting human sentence reading speeds.
Four geometries govern sequential and distribution-free inference.
problem Sequential and distribution-free inference challenges.
method Four distinct admissibility geometries.
result Four classes of admissible procedures are pairwise non-nested.
Introduces new Wasserstein distances for more intrinsic metrics.
problem Improve metric for comparing distributions.
method Introduces distances, designs algorithms for computation.
result New distances are more intrinsic and computable.
Neural nets approximate Ricci flat metrics for Calabi-Yau manifolds.
problem Lack of analytical Ricci flat metrics for Calabi-Yau threefolds.
method Employed neural network approximations for several Calabi-Yau manifolds of dimensions two and three.
result Measures of Ricci flatness improved by three orders of magnitude after training.
The paper proves a Moser-Trudinger inequality on metric spaces with curvature-dimension conditions.
problem Proving a Moser-Trudinger inequality on metric measure spaces.
method Rearrangement of functions on CD(k,n)-spaces satisfying a Polya-Szegö type inequality.
result Characterization of manifolds with lower bounded Ricci curvature admitting a Moser-Trudinger inequality.
Paper solves a metric-independent problem on almost Kähler 4-manifolds.
problem Find a metric-independent generalization of Bott-Chern and Aeppli numbers.
method Introduced a new approach to generalize Bott-Chern and Aeppli numbers.
result Found a solution valid on almost Kähler 4-manifolds.
Optimal inequalities found between Riemannian and Hilbert metrics in convex projective domains.
problem Finding optimal bounds between Riemannian and Hilbert metrics in convex projective domains.
method Optimal control techniques applied to Riemannian metrics induced by centro-affine hypersurface immersions.
result Optimal inequalities between Riemannian and Hilbert metrics for a class of convex projective domains.
Recent work in metric learning has significantly improved the state-of-the-art in k-nearest neighbor classification. Support vector machines (SVM), particularly with RBF kernels, are amongst the most popular classification algorithms that uses distance metrics to compare examples. This paper provides an empirical analy…
Paper validates ABM using stylized financial facts.
problem Validate ABM-generated financial data against real-world data.
method Compare ABM results with stylized financial facts.
result Model successfully replicates stylized financial facts.