New metrics solve machine learning limitations.
problem Previous success metrics restrict application to specific forms of machine learning.
method Define decomposable metrics as linear operations on probability distributions.
result Demonstrated theorems bounding success in various ways, generalizing existing results.
We apply machine learning to the problem of finding numerical Calabi-Yau metrics. Building on Donaldson's algorithm for calculating balanced metrics on Kähler manifolds, we combine conventional curve fitting and machine-learning techniques to numerically approximate Ricci-flat metrics. We show that machine learning is …
New machine learning approach finds Kähler metrics through Grassmannian learning.
problem Finding numerical Kähler metrics through machine learning.
method Gradient descent on the Grassmannian manifold, combining with Donaldson's algorithm.
result Observation of nontrivial local minima in moduli space.
Novel metrics improve machine learning models for ICU patient care.
problem Predicting vital sign trajectories for early detection of adverse events.
method Developed novel performance metrics aligned with clinical contexts, validated on simulated and real datasets, and optimized neural networks using these metrics.
result Neural networks trained with these metrics excel in predicting clinically significant events.
Metric learning enhances combinatorial coverage metrics' ability to predict classification errors.
problem Dataset dependence of combinatorial coverage metrics in anticipating classification errors.
method Metric learning to improve latent space separation of data classes.
result Metric learning increases SDCCMs' ability to distinguish between correctly and incorrectly classified data.
New metrics needed for streaming ML due to delayed labels.
problem Streaming ML evaluation fails to identify unexpected performance.
method Recommend additional metrics for streaming ML performance.
result New metrics are needed for streaming ML due to delayed labels.
The paper quantifies uncertainty in aggregated machine learning metrics.
problem Uncertainty in summarizing model performance across multiple tasks.
method Statistical methodologies including bootstrapping and Bayesian modeling.
result Insights into model performance dominance for specific tasks.
Efficiently tests machine learning models with minimal labeled data.
problem Guaranteeing the performance of machine learning models and preventing failures.
method Proposes a novel framework using Bayesian neural networks and data augmentation for efficient testing.
result Metrics estimations by the proposed method are significantly better than existing baselines.
The paper finds compact symbolic approximations for Ricci-flat metrics using Calabi-Yau hypersurfaces.
problem Finding explicit constructions of Ricci-flat metrics on Calabi-Yau manifolds remains challenging.
method Analysis of machine learning approximations and formalisation of symmetries.
result Ricci-flat metrics have more symmetries than the underlying manifold, leading to compact representations.
In machine learning, the choice of a learning algorithm that is suitable for the application domain is critical. The performance metric used to compare different algorithms must also reflect the concerns of users in the application domain under consideration. In this work, we propose a novel probability-based performan…
Mathematical approach defines stability conditions for ML models.
problem Ensuring stability of machine learning models.
method Adopted topological and metric spaces theory to define stability.
result Stability of ML models depends on topological properties of classification sets.
The paper clusters group fairness metrics and visualizes them using PCA.
problem Measuring discrimination against socio-demographic groups using multiple metrics.
method Empirical observation and PCA visualization of fairness metrics.
result Group fairness metrics cluster into two or three main clusters.
New metrics for information geometry and machine learning from Lie groups.
problem Traditional mean methods in data science and machine learning.
method Cartan-Schouten metrics on Lie groups.
result Cartan-Schouten metrics offer advantages over traditional means.
This thesis formalizes metric selection for machine learning applications.
problem Selecting the right performance metric for machine learning models.
method Metric Elicitation framework using pairwise preference feedback.
result Novel strategies for eliciting various classification metrics robust to noise.
The need for appropriate ways to measure the distance or similarity between data is ubiquitous in machine learning, pattern recognition and data mining, but handcrafting such good metrics for specific problems is generally difficult. This has led to the emergence of metric learning, which aims at automatically learning…
metric-learn is an open source Python package implementing supervised and weakly-supervised distance metric learning algorithms. As part of scikit-learn-contrib, it provides a unified interface compatible with scikit-learn which allows to easily perform cross-validation, model selection, and pipelining with other machi…
Novel metric space magnitude and weighting vectors improve machine learning tasks.
problem Improving machine learning algorithms using novel metric space concepts.
method Metric space magnitude and weighting vectors for better machine learning.
result The weighting vector effectively detects boundaries and improves classic machine learning tasks.
FiberNet integrates geometry into machine learning for clearer classification.
problem Lack of interpretability in traditional deep learning.
method Reformulates classification as geometric optimization on fiber bundles, introducing learnable Riemannian metrics and variational prototype optimization.
result Clear geometric interpretability and efficiency in classification.
A new stock selection strategy uses combined machine learning with dynamic weighting methods.
problem Improving stock selection accuracy and performance.
method Combined machine learning algorithms with static and dynamic weighting methods.
result IC-based dynamic weighting outperforms static evaluation metrics in backtested returns and predictive performance.
This paper compares machine and deep learning algorithms for IoT data classification.
problem Classifying IoT data using machine and deep learning algorithms.
method Evaluation of 11 machine and deep learning algorithms on six IoT datasets using multiple performance metrics.
result Random Forests outperformed other machine learning models, while ANN and CNN performed well among deep learning models.
Recent work in metric learning has significantly improved the state-of-the-art in k-nearest neighbor classification. Support vector machines (SVM), particularly with RBF kernels, are amongst the most popular classification algorithms that uses distance metrics to compare examples. This paper provides an empirical analy…
Two simple methods learn fair metrics from data to improve fairness in ML tasks.
problem Lack of widely accepted fair metrics for many ML tasks hinders individual fairness adoption.
method Presented two simple ways to learn fair metrics from various data types.
result Fair training with learned metrics improves fairness on three ML tasks.
Examines performance metrics for ML models in engineering.
problem Identifying proper assessment metrics for ML models.
method Analyzes commonly-used metrics for regression and classification.
result Identifies key performance metrics for ML models.
Interactive tool helps choose and understand classification metrics.
problem Common metrics for binary classification have limitations.
method Graphical application to visualize and explore evaluation metrics.
result Promotes careful attention to interpretation of metrics.
Matched Machine Learning combines machine learning and matching for causal inference.
problem Non-interpretable methods for causal inference.
method Combines machine learning and matching for interpretable causal inference.
result Performs as well as black-box machine learning methods and better than existing matching methods.
Permutation invariant network learns Wasserstein metrics.
problem Understanding the space of probability measures and comparing distributions.
method Permutation invariant network mapping samples to a low-dimensional space.
result Network can generalize to compute distances between unseen densities and learn moments.
Performance metrics (error measures) are vital components of the evaluation frameworks in various fields. The intention of this study was to overview of a variety of performance metrics and approaches to their classification. The main goal of the study was to develop a typology that will help to improve our knowledge a…
Riemannian metric learning improves data representation across various fields.
problem Traditional distance metrics fail to capture intrinsic data geometry.
method Leverages differential geometry to model data on Riemannian manifolds.
result Demonstrates remarkable success in diverse domains.
Neural nets approximate Ricci flat metrics for Calabi-Yau manifolds.
problem Lack of analytical Ricci flat metrics for Calabi-Yau threefolds.
method Employed neural network approximations for several Calabi-Yau manifolds of dimensions two and three.
result Measures of Ricci flatness improved by three orders of magnitude after training.
Paper introduces new importance metrics for machine learning models, linking them to CATE.
problem Interpreting black-box models' importance metrics due to data dependence and non-parametric nature.
method Introduces MVIM and CVIM, proposing permutation-based estimation and bias-variance decomposition.
result MVIM and CVIM have a quadratic relationship with CATE, addressing bias in correlated predictors.
Reassesses calibration metrics in machine learning models.
problem Inconsistent reporting of calibration metrics in recent literature.
method Calibration-based decomposition of Bregman divergences, visualization of calibration and generalization error.
result New visualization technique for detecting trade-offs between calibration and generalization.
New metrics improve quantum ensemble learning efficiency and power.
problem Quantum ensembles' distances poorly understood due to measurement constraints.
method Introduce MMD-k hierarchy of integral probability metrics for quantum ensembles. result MMD-k requires fewer samples for full discriminative power at higher k. The paper discusses the limitations of efficiency metrics in machine learning models.
problem Inadequate reporting of efficiency metrics leads to incomplete conclusions.
method Thoroughly discusses common cost indicators, their advantages and disadvantages, and how they contradict each other.
result Incomplete reporting of efficiency metrics can lead to partial conclusions and a blurred picture of model practical considerations.
New method uses interval-based metric to validate prediction uncertainty in machine learning.
problem Validation of prediction uncertainty in machine learning regression tasks is unreliable due to heavy-tailed distributions.
method Shift from variance-based metrics to interval-based Prediction Interval Coverage Probability (PICP).
result PICP method more quickly and reliably tests prediction intervals than variance-based metrics.
Machine learning solves Einstein equations without symmetry assumptions.
problem Solving the Euclidean vacuum Einstein equations with a cosmological constant.
method Semi-supervised machine learning with patch-based architecture and coordinate consistency loss.
result Hints against the existence of Ricci-flat metrics on spheres in dimensions 4 and 5.
As more researchers have become aware of and passionate about algorithmic fairness, there has been an explosion in papers laying out new metrics, suggesting algorithms to address issues, and calling attention to issues in existing applications of machine learning. This research has greatly expanded our understanding of…
CausalBench aims to advance causal learning research with a transparent platform.
problem Lack of unified benchmark datasets, algorithms, metrics, and evaluation interfaces for causal learning.
method Introduces CausalBench, a flexible benchmark framework for causal analysis and machine learning.
result Promotes scientific collaboration, reproducibility, and awareness in causal learning research.
We introduce DQFIM to quantify and improve generalization of quantum machine learning models.
problem Understanding and improving generalization of quantum machine learning models.
method Data quantum Fisher information metric (DQFIM) to quantify circuit parameters and training data.
result Improves generalization by breaking symmetries of training data and using a low number of training states.
This paper applies combinatorial testing to machine learning for robust model performance.
problem Identifying robust machine learning models using test and training sets.
method Adapting combinatorial interaction testing for machine learning, focusing on simple features.
result Combinatorial coverage can enhance model performance and robustness.
Enhanced AI analysis predicts S&P 500 stock dynamics using various financial metrics.
problem Predicting S&P 500 stock performance with complex interplay of factors.
method Advanced financial metrics, machine learning, and integration of traditional and modern analytics.
result Enhanced predictive accuracy in market behavior and investment strategies.
Paper analyzes uncertainty metrics in ensemble learning for healthcare AI.
problem Selecting appropriate uncertainty metrics for ensemble learners in healthcare AI.
method Rigorous analysis of two uncertainty metrics: ensemble mean and variance.
result Ensemble mean is preferable to ensemble variance for decision making in healthcare AI.
QCML improves bond similarity learning in illiquid markets.
problem Improving similarity learning for illiquid corporate bonds.
method Quantum Cognition Machine Learning (QCML) for supervised distance metric learning.
result QCML outperforms classical tree-based models in high-yield markets.
Interpretable machine learning tackles the important problem that humans cannot understand the behaviors of complex machine learning models and how these models arrive at a particular decision. Although many approaches have been proposed, a comprehensive understanding of the achievements and challenges is still lacking…
Uncertainty Toolbox aids in assessing and improving uncertainty quantification in machine learning.
problem Disparate evaluation metrics and implementations hinder direct comparison of uncertainty quantification results.
method Provides an open-source Python library for assessing, visualizing, and improving uncertainty quantification.
result Facilitates more accurate and comparable uncertainty quantification across different works.
Study improves machine learning models for GI tract disease detection using comprehensive evaluations and cross-dataset testing.
problem Incomplete or incorrect evaluation of machine learning models for GI tract diseases.
method Comprehensive evaluations of five machine learning models using Global Features and Deep Neural Networks, introducing performance hexagons and cross-dataset testing.
result Demonstrates the need for more sophisticated performance metrics and evaluation methods to build generalizable models.
Complex embeddings handle non-metric proximity data better than traditional methods.
problem Proximities not always metric or inner product-based, causing convergence issues.
method Proposes complex-valued embeddings for non-vectorial data.
result Complex embeddings outperform traditional techniques on benchmarks.
This paper reviews metrics for evaluating multi-class classification models.
problem Evaluating and comparing multi-class classification models.
method Review and analysis of metrics.
result Promising multi-class metrics are highlighted and their usages are demonstrated.
Quantum model improves safety in machine learning.
problem Improving safety and robustness in machine learning models.
method Variational quantum classifier with amplitude encoding and SAFE-AI metrics.
result Quantum model provides competitive performance and improved robustness.