Variable metric proximal gradient (VM-PG) is a widely used class of convex optimization method. Lately, there has been a lot of research on the theoretical guarantees of VM-PG with different metric selections. However, most such metric selections are dependent on (an expensive) Hessian, or limited to scalar stepsizes l…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “metric selection”
Proposes a method to select fair performance metrics through metric elicitation.
problem Choosing fair performance metrics in multiclass classification with multiple sensitive groups.
method Metric elicitation strategy that requires only relative preference feedback and is robust to noise.
result Elicits group-fair performance metrics for multiclass classification problems.
Counterfactual Cross-Validation: Stable Model Selection Procedure for Causal Inference Modelsstat.ML
We study the model selection problem in conditional average treatment effect (CATE) prediction. Unlike previous works on this topic, we focus on preserving the rank order of the performance of candidate CATE predictors to enable accurate and stable model selection. To this end, we analyze the model performance ranking …
New metrics using Laplace approximation improve Gaussian process model selection.
problem Finding a balance between model accuracy, interpretability, and simplicity.
method Introducing multiple metrics based on the Laplace approximation to evaluate Gaussian process models.
result Our metrics provide comparable performance to dynamic nested sampling but are significantly faster.
A stability metric compares feature selection algorithms in machine learning.
problem Stability of feature selection algorithms in machine learning.
method Rank-based instability index to compare MDA, LIME, and SHAP algorithms.
result LIME and SHAP are more stable than MDA, with LIME being best for human interpretability.
New robustness metric helps select reliable classifiers.
problem Evaluating reliability of classifier predictions.
method Proposed new robustness metric for any classifier and feature type.
result Demonstrated ability to distinguish reliable from unreliable predictions.
Proposes a Siamese NN for algorithm selection focusing on alike performing instances.
problem Lack of effective meta-features for algorithm selection via meta-learning.
method Siamese Neural Network architecture with 'Algorithm-Performance Personas' concept.
result Proposed metric outperforms standard performance metrics in training sample selection.
Testing of deep learning models is challenging due to the excessive number and complexity of computations involved. As a result, test data selection is performed manually and in an ad hoc way. This raises the question of how we can automatically select candidate test data to test deep learning models. Recent research h…
This thesis formalizes metric selection for machine learning applications.
problem Selecting the right performance metric for machine learning models.
method Metric Elicitation framework using pairwise preference feedback.
result Novel strategies for eliciting various classification metrics robust to noise.
Unveiling the Potential of Robustness in Selecting Conditional Average Treatment Effect Estimatorscs.LG
Paper introduces DRM for selecting robust CATE estimators.
problem Selecting CATE estimators without counterfactual outcomes.
method Distributionally Robust Metric (DRM) for CATE estimator selection.
result DRM selects robust CATE estimators robust to distribution shift.
A new framework selects information sources to test hypotheses robustly, even with misclassifications.
problem Robust hypothesis testing with misclassification penalties.
method Introduces a misclassification penalty framework and an efficient greedy algorithm.
result Proposes a submodular surrogate metric for better selection.
Unified query framework for active metric learning and classification.
problem Combining representation learning and task-specific goals in machine learning.
method Adaptive selection of nearest neighbor queries using information theoretic criterion.
result Actively selected nearest neighbor queries outperform recent methods in deep metric learning and classification.
BeST metric selects best source models for transfer learning.
problem Efficiently selecting top source models for a new target task.
method Quantization-level optimization procedure for similarity measure.
result BeST consistently identifies best source models for target tasks.
MDS selects assets by combining daily returns and intraday risk curves, improving portfolio performance.
problem High estimation error in large-scale asset selection.
method Metric Dependence Screening (MDS) incorporating high frequency information as object valued data.
result MDS improves portfolio performance over benchmarks by preserving intraday risk dynamics.
Minimal Morse functions on Poincaré dodecahedral space are selected via spectral properties.
problem Identifying minimal Morse functions on the Poincaré dodecahedral space.
method Spectral selection property P, obstruction principle, conformal variations, finite dimensional reduction.
result Restoration of minimal Morse selection on the Poincaré dodecahedral space via spectral mechanisms.
The predictive power of neural networks often costs model interpretability. Several techniques have been developed for explaining model outputs in terms of input features; however, it is difficult to translate such interpretations into actionable insight. Here, we propose a framework to analyze predictions in terms of …
Unsupervised clustering can reproduce categorization systems if features and metrics are correctly selected.
problem Reproducing expert-provided categorization systems using unsupervised clustering.
method Investigated using toy datasets and real-world fund categorization. Used appropriate feature selection and a supervised Random Forest-based distance metric.
result Unsupervised clustering can reproduce ground truth classes if features and metrics are correctly selected.
New method selects equivariant models using uncertainty metrics.
problem Selecting equivariant models among pretrained ones with varying symmetry biases.
method Uncertainty-aware model selection using frequentist, Bayesian, and calibration-based measures.
result Bayesian model evidence often misaligns with predictive performance.
Survey of recent metric geometry in Kähler metrics space.
problem Understanding the metric geometry of Kähler metrics space.
method Survey and highlighting of recent results.
result Highlighting of open problems in the field.
This research introduces a new strategy in cluster ensemble selection by using Independency and Diversity metrics. In recent years, Diversity and Quality, which are two metrics in evaluation procedure, have been used for selecting basic clustering results in the cluster ensemble selection. Although quality can improve …
Deep metric learning is often used to learn an embedding function that captures the semantic differences within a dataset. A key factor in many problem domains is how this embedding generalizes to new classes of data. In observing many triplet selection strategies for Metric Learning, we find that the best performance …
New metrics CWSA and CWSA+ improve model evaluation under confidence thresholds.
problem Lack of metrics capturing model reliability under confidence thresholds.
method Introducing CWSA and CWSA+ metrics that reward confident accuracy and penalize overconfident mistakes.
result CWSA and CWSA+ outperform classical metrics in trust-sensitive tests.
New metrics help rebuild trust in Active Learning for industry practitioners.
problem Misalignment of objectives between research and industry in Active Learning.
method Presentation of actionable metrics to interpret AL strategies.
result Actionable metrics improve AL performance and interpretability.
A novel criterion selects optimal distance metrics for cell profile analysis.
problem Determining the most accurate distance metric for high-dimensional cell profiles.
method Generalized proposition and corollaries to evaluate and select distance metrics.
result Wasserstein and cosine similarity metrics are optimal for general cases.
Efficient and Robust Bayesian Selection of Hyperparameters in Dimension Reduction for Visualizationstat.ML
Efficient auto-tuning for DR hyperparameters with BO.
problem Hyperparameter selection in DR for large-scale datasets.
method Bayesian optimization with surrogate model, normalization, subsampling.
result Robust and efficient hyperparameter selection for DR algorithms.
A new stock selection strategy uses combined machine learning with dynamic weighting methods.
problem Improving stock selection accuracy and performance.
method Combined machine learning algorithms with static and dynamic weighting methods.
result IC-based dynamic weighting outperforms static evaluation metrics in backtested returns and predictive performance.
Value selection reduces model size while maintaining accuracy.
problem Space efficiency in model size reduction.
method Two probabilistic methods based on information theory's metric: PVS and P + VS.
result Value selection achieves balance between accuracy and model size reduction.
Optimal subset selection for hypothesis testing with penalties.
problem Optimal subset selection of information sources for hypothesis testing with misclassification penalties.
method Proposes a misclassification penalty framework and studies two variants of subset selection problems under centralized Bayesian learning.
result Proves the submodularity of the objective and constraints of the subset selection problems and establishes performance guarantees for greedy algorithms.
Paper introduces a new performance metric for class imbalance datasets.
problem Challenges in selecting and comparing models for imbalanced datasets.
method Proposes a new performance measure based on the harmonic mean of Recall and Selectivity normalized in class labels.
result The proposed measure is less sensitive to changes in the majority class and more sensitive to changes in the minority class.
metric-learn is an open source Python package implementing supervised and weakly-supervised distance metric learning algorithms. As part of scikit-learn-contrib, it provides a unified interface compatible with scikit-learn which allows to easily perform cross-validation, model selection, and pipelining with other machi…
Investigates model selection challenges in heterogeneous treatment effect estimation.
problem Lack of validation metrics for choosing the best model in treatment effect estimation.
method Empirical investigation of different model selection criteria.
result Complex interplay between selection strategies, estimators, and data.
This is a sequel to the survey given by the author in Surveys in Differential Geometry Vol. 6 in 1999. The present survey covers selected developments since 1999 to 2011 dealing with the construction of Einstein metrics on bundles and Einstein metrics with symmetries.
Distance metric learning is an important component for many tasks, such as statistical classification and content-based image retrieval. Existing approaches for learning distance metrics from pairwise constraints typically suffer from two major problems. First, most algorithms only offer point estimation of the distanc…
The study uses unsupervised machine learning to identify top European football teams.
problem Selecting teams for the new European football Super League.
method Used Laplacian eigenmaps clustering on performance data.
result Successfully identified four clusters of teams based on performance metrics.
Investigates the use of Information Coefficient as a stock selection model performance measure.
problem The adequacy and effectiveness of Information Coefficient (IC) for evaluating stock selection models is unclear.
method Simulation and simple statistical modeling to examine IC behavior statically and dynamically.
result Proposes two practical procedures for IC-based ongoing performance monitoring of stock selection models.
We develop a statistical framework to benchmark and select large language models based on their risks.
problem Benchmarking and selecting large language models based on their associated risks.
method A distributional framework using first and second order stochastic dominance, linked to mean-risk models in finance.
result Formalizes a risk-aware approach for model selection, balancing risk and utility.
We frame the problem of selecting an optimal audio encoding scheme as a supervised learning task. Through uniform convergence theory, we guarantee approximately optimal codec selection while controlling for selection bias. We present rigorous statistical guarantees for the codec selection problem that hold for arbitrar…
Paper proposes adaptive parameter selection for KGD algorithms.
problem Improving parameter selection for kernel-based gradient descent.
method Integrates bias-variance analysis with splitting method, introduces empirical effective dimension.
result Adaptive parameter selection strategy achieves optimal generalization error bound.
A number of machine learning algorithms are using a metric, or a distance, in order to compare individuals. The Euclidean distance is usually employed, but it may be more efficient to learn a parametric distance such as Mahalanobis metric. Learning such a metric is a hot topic since more than ten years now, and a numbe…
In the panoply of pattern classification techniques, few enjoy the intuitive appeal and simplicity of the nearest neighbor rule: given a set of samples in some metric domain space whose value under some function is known, we estimate the function anywhere in the domain by giving the value of the nearest sample per the …
MICO uses mutual information co-training to improve selective search efficiency.
problem Efficiently search and route unseen queries in large-scale search systems.
method Mutual Information Co-training framework for selective search with minimal supervision.
result Significantly improves performance on multiple metrics of selective search.
This work evaluates and benchmarks calibration metrics for data-driven regression models.
problem Conflicting results from different calibration metrics make it hard to compare and interpret model performance.
method Systematically extracted and benchmarked 14 regression calibration metrics across various data types and recalibration methods.
result Many metrics disagree on the same recalibration result, highlighting the need for careful metric selection.
ICU mortality scoring systems attempt to predict patient mortality using predictive models with various clinical predictors. Examples of such systems are APACHE, SAPS and MPM. However, most such scoring systems do not actively look for and include interaction terms, despite physicians intuitively taking such interactio…
A2A metric evaluates bias correction methods, reducing ATE estimation errors.
problem Selection biases in non-randomized studies of medical treatments.
method Propensity score matching (PSM) with novel metric A2A.
result Reduces ATE estimation errors by up to 90% across synthetic and real-world datasets.
Proposes a method to choose thresholds for LLM evaluation metrics.
problem Ensuring reliable large language models (LLMs) with correct threshold selection.
method Identify risks, stakeholders' risk tolerance, and use ground-truth data to determine thresholds.
result Demonstrates a concrete example with the Faithfulness metric and HaluBench dataset.
Perplexity fails to distinguish correct predictions from incorrect ones in some cases.
problem Perplexity's limitations in model selection.
method Analyzing Transformer continuity and iso-perplexity plots.
result Perplexity does not always select the correct model.
A Cholesky decomposition-based asset selection heuristic for sparse tangent portfolio optimizationq-fin.MF
New heuristic selects fewer assets for efficient portfolios, reducing costs.
problem High transaction costs and fees from including many assets in portfolios.
method Surrogate formulation to select assets, re-optimizes portfolio with fewer assets.
result Effective in constructing portfolios with fewer assets, reducing costs.
Paper introduces a new metric to select optimal Graph Shift Operator for GNNs.
problem Empirical selection of Graph Shift Operator remains challenging.
method Introduces a novel alignment gain metric connecting geometric distortion to generalization bounds via spectral proxy.
result Provides a principled, computation-efficient criterion to rank and select optimal GSO.