The paper analyzes the Rashomon ratio for infinite classifier families and shows its importance for choosing good classifiers.
problem Analyzing the Rashomon ratio for infinite classifier families.
method Quantifying the Rashomon ratio in two examples and providing guarantees for estimating it.
result A large Rashomon ratio guarantees choosing a classifier with good empirical accuracy will not significantly increase empirical loss.
Noise increases the Rashomon ratio, leading simpler models to perform similarly to complex ones.
problem Why simpler models perform similarly to complex models on noisy datasets.
method Analyzed the data generation process and model training choices, introduced pattern diversity.
result Noisier datasets lead to larger Rashomon ratios, explaining simpler models' performance.
It is almost always easier to find an accurate-but-complex model than an accurate-yet-simple model. Finding optimal, sparse, accurate models of various forms (linear models with integer coefficients, decision sets, rule lists, decision trees) is generally NP-hard. We often do not know whether the search for a simpler m…
Introduces Rashomon Capacity to measure predictive multiplicity in probabilistic classifiers.
problem Predictive multiplicity in classification models leading to unjustified decisions.
method Introduces Rashomon Capacity, a metric for probabilistic classifiers, and provides a rigorous derivation.
result Rashomon Capacity captures nuanced score variations and provides strategies for disclosing conflicting models.
New algorithms approximate Rashomon set for sparse models, aiding expert interaction.
problem Lack of interaction between models and domain experts in classical machine learning.
method Approximate Rashomon set of sparse, generalized additive models using ellipsoids.
result Efficiently approximated Rashomon set facilitates model selection and exploration.
Framework uses dropout to efficiently explore Rashomon set for multiplicity estimation.
problem Efficiently measuring and mitigating conflicting model outputs in classification tasks.
method Dropout-based exploration of Rashomon set for multiplicity estimation.
result Framework outperforms baselines in multiplicity metric estimation with significant runtime speedup.
New method uses Rashomon sets to improve Bayesian inference in factorial designs.
problem Combustion of model uncertainty in factorial designs leads to multimodal posterior and convergence issues.
method Rashomon-seeded annealing, integrating high-performing models as warm start for AIS.
result Restores full posterior inference without exhaustive enumeration of model space.
AXE evaluates explanations to avoid misleading Rashomon set model selection.
problem Evaluating explanations for Rashomon set models to avoid false selection.
method Proposed AXE method to evaluate explanation quality.
result AXE detects adversarial fairwashing with 100% success rate.
The Rashomon effect shows many models can perform similarly, explored in this paper.
problem Why do many models perform similarly in machine learning?
method Categorized causes into statistical, structural, and procedural sources.
result Structural multiplicity persists and cannot be resolved without additional assumptions.
REALITrees uses a Rashomon ensemble approach for active learning in sparse decision trees.
problem Active learning reduces labeling costs by selecting informative samples, but current methods often sacrifice model diversity and direct characterization of the hypothesis space.
method REALITrees constructs a committee of all near-optimal sparse decision tree models using a Rashomon Set and a Gibbs posterior to weight them by empirical risk.
result REALITrees outperforms randomized ensembles, especially in noisy environments, by leveraging expanded model multiplicity.
Commentary on Rashomon Effect complicating model selection.
problem Many models equally predict data; hard to draw conclusions.
method Connections to recent ML literature exploring implications.
result Grasping Rashomon Effect can foster collaboration.
Paper explores Rashomon set models for more trustworthy medical conclusions.
problem Lack of comprehensive analysis of models in Rashomon set leads to misleading conclusions.
method Introduces Rashomon_DETECT algorithm and Profile Disparity Index (PDI).
result Combining differently behaving models in Rashomon set provides more trustworthy conclusions.
Paper presents a Rashomon Quartet to show different model explanations.
problem Models with similar performance can explain data differently.
method Synthetic dataset with four equally effective models.
result Models with similar performance can explain data differently.
Study shows more data improves model explanations, aiding reliable knowledge extraction.
problem Challenges in deriving reliable knowledge from machine learning models due to the Rashōmon effect.
method Examined the influence of sample size on explanations from models in a Rashōmon set using SHAP.
result Explanations from <128 samples are highly variable, but agreement improves with more data.
UNREAL selectively ensembles distinct models to improve active learning performance.
problem Difficulty in distinguishing genuine uncertainty from noise in limited labeled data.
method Selective ensembling of distinct models from the Rashomon set.
result UNREAL achieves faster convergence and up to 20% predictive accuracy improvement.
Leo Breiman's Rashomon Effect and Occam Dilemma are re-evaluated in the context of modern machine learning.
problem The tradeoff between model complexity and accuracy in machine learning.
method Modern perspective on Breiman's arguments using current computational capabilities.
result Algorithmic models can be accurate without being complex, nullifying the Occam Dilemma.
This paper explores how balancing and filtering techniques affect predictive multiplicity in machine learning models.
problem Predictive multiplicity due to Rashomon effect in high-stakes environments.
method Investigates the impact of balancing and filtering techniques on predictive multiplicity using 21 real-world datasets.
result Data-centric AI strategies can mitigate predictive multiplicity, but preprocessing methods may introduce it.
Bayesian framework for model uncertainty identifies complex heterogeneity without strong assumptions.
problem Identifying complex heterogeneity in factorial data with varying covariates.
method Rashomon Partition Sets (RPS) using l0 prior for robust model uncertainty.
result RPS provides a robust set of models capturing complex heterogeneity without strong assumptions.
Framework for assessing fairness across similar predictive models.
problem Fairness in predictive models across different groups.
method Develops a framework for characterizing fairness over the set of good models under selective labels.
result Framework can replace or audit models for better fairness properties.
Random projections improve classifier generalization without needing to choose the best threshold.
problem Improving classifier generalization without choosing the best threshold.
method Thresholding a random one-dimensional feature after random projection of data.
result Generalization gap is significantly smaller than linear classifiers.
New framework quantifies variable importance across all good models and is stable across data distribution.
problem Conflicting variable importance conclusions from different models trained on the same data.
method Proposes a new variable importance framework that considers all good models and is stable across data distribution.
result Framework accurately estimates true variable importance and recovers rankings for complex setups.
Breiman's data analysis dichotomy is outdated, offering a third approach: mechanistic models.
problem Data analysis dichotomy between data modelers and algorithmic modelers.
method Interpolating between simple interpretable models and flexible function approximations using mechanistic models.
result Flexible, interpretable, and scientifically-informed hybrids can provide accurate and robust predictions.
A GPU framework speeds up BnB for discrete optimization problems.
problem Optimizing large-scale discrete problems with GPU limitations.
method Parallel BnB nodes in GPU batches, using padding and custom kernels.
result One to two orders of magnitude speedup and zero optimality gap.
We discuss - in what is intended to be a pedagogical fashion - generalized "mean-to-risk" ratios for portfolio optimization. The Sharpe ratio is only one example of such generalized "mean-to-risk" ratios. Another example is what we term the Fano ratio (which, unlike the Sharpe ratio, is independent of the time horizon)…
FORE evaluates occupancy ratios without requiring Bellman completeness.
problem Offline reinforcement learning occupancy ratio estimation.
method Fitted occupancy-ratio evaluation (FORE) using adjoint Bellman recursion.
result FORE achieves convergence in KL without Bellman completeness.
Optimal option portfolios under Sharpe Ratio maximization with skew-elliptical t-distributed returns
problem Optimal option portfolios under Sharpe Ratio maximization
method Formulation for explicit portfolio weights
result Different optimal portfolios for Sharpe Ratio and return-to-Value-at-Risk (VaR) ratio
Omega ratio, defined as the probability-weighted ratio of gains over losses at a given level of expected return, has been advocated as a better performance indicator compared to Sharpe and Sortino ratio as it depends on the full return distribution and hence encapsulates all information about risk and return. We comput…
We present a new methodology of computing incremental contribution for performance ratios for portfolio like Sharpe, Treynor, Calmar or Sterling ratios. Using Euler's homogeneous function theorem, we are able to decompose these performance ratios as a linear combination of individual modified performance ratios. This a…
Survey on principles and challenges of interpretable machine learning.
problem Improving machine learning models' interpretability for high-stakes decisions.
method Identification and analysis of 10 technical challenges in interpretable machine learning.
result Identification of 10 technical challenges in interpretable machine learning.
The paper proposes an asset allocation strategy using the Sortino ratio for better performance.
problem Traditional asset allocation methods like the Sharpe ratio do not penalize negative returns adequately.
method The Sortino ratio is used to maximize asset allocation, penalizing only negative return variances.
result The Sortino ratio-based strategy outperforms traditional methods like the Kelly criterion.
A new ratio, the Hansen ratio, simplifies mean-variance portfolio theory.
problem Simplifying mean-variance portfolio theory.
method Introducing the Hansen ratio and extending mean-variance theory.
result The Hansen ratio provides a parsimonious description of the mean-variance efficient frontier.
Develops a new density ratio estimator for causal inference.
problem Estimation of density ratio functions in statistics.
method Super learning approach with a novel loss function.
result Empirical validation of the density ratio super learner's performance.
New PU ratio predicts long-term Bitcoin returns better than other methods.
problem Lack of convincing proxies for cryptocurrency fundamentals.
method Developed a new market-to-fundamental ratio (PU ratio) using blockchain accounting methods.
result PU ratio effectively predicts long-term Bitcoin returns compared to alternative methods.
The paper studies curves of constant-ratio in pseudo-Galilean space.
problem Characterizing curves of constant-ratio in pseudo-Galilean space.
method Analyzing spacelike curves with constant-ratio in terms of curvature functions.
result Characterization of special curves of constant-ratio in pseudo-Galilean space.
Unified framework for OOD detection using class ratio estimation.
problem Density-based OOD detection is unreliable for OOD images.
method Unified framework that builds energy-based models and employs differing base distributions, directly estimating the density ratio through class ratio estimation.
result Competitive results on OOD image problems compared to recent work.
Paper shows how to embed Möbius bands with many twists and small aspect ratios.
problem Finding the smallest aspect ratio for Möbius bands with many twists.
method Constructs a folded paper ribbon knot to bound the aspect ratio.
result Paper Möbius bands and annuli with any number of half-twists can be embedded with aspect ratio less than 8.
Direct neural ratio estimator for likelihood-free inference.
problem Efficient likelihood estimation for complex models.
method Amortized likelihood ratio estimation using neural networks.
result DNRE often outperforms previous ratio estimators.
Neural networks approximate likelihood ratios for complex models.
problem Difficulty in computing likelihood ratios for modern models.
method Applying the likelihood ratio trick with neural network classifiers.
result Different neural network setups can approximate likelihood ratios with varying performance.
Paper tackles unbounded density ratio estimation for covariate shift adaptation.
problem Understudied challenge in statistical learning: unbounded density ratios.
method Three-step estimation method: relative density ratio, truncation, and transformation.
result Established rigorous convergence guarantees for density ratio and regression estimators.
Calculates twist in Teichmüller space using cross ratios.
problem Calculating the Fenchel-Nielsen twist in Teichmüller space.
method Using cross ratio coordinates.
result Compact calculation of twist in Teichmüller space.
Study shows robust method for estimating density ratios even with heavy contamination.
problem Estimating density ratios in the presence of heavy contamination.
method Weighted density ratio estimation (DRE) with doubly strong robustness.
result Weighted DRE achieves sparse consistency under heavy contamination.
Meta-learning improves relative density-ratio estimation from limited data.
problem Estimating relative density-ratios from few instances.
method Meta-learning using neural networks to extract and embed dataset information for relative DRE.
result Meta-learning enables efficient and effective adaptation to few instances for relative DRE.
TRE improves density-ratio estimation for highly dissimilar densities.
problem Density-ratio estimation fails for significantly different densities.
method Telescoping density-ratio estimation (TRE) framework.
result TRE yields substantial improvements over existing methods for mutual information estimation.
New method resolves density ratio estimation saturation issues.
problem Error saturation in density ratio estimation methods.
method Iterated regularization to improve kernel methods.
result Achieves fast error rates on regular learning problems.
The paper describes a method to infer the signal-to-noise ratio in portfolio optimization.
problem Estimating the signal-to-noise ratio in portfolio optimization problems.
method A statistic similar to the Sharpe Ratio Information Criterion is used for inference.
result The method works well for reasonable sample and asset universe sizes.
Sharpe ratio (sometimes also referred to as information ratio) is widely used in asset management to compare and benchmark funds and asset managers. It computes the ratio of the (excess) net return over the strategy standard deviation. However, the elements to compute the Sharpe ratio, namely, the expected returns and …
Paper develops estimators for unbounded density ratios with applications in error control.
problem Estimating density ratios with unbounded domains and ranges.
method Least squares and logistic regression loss functions for density ratio estimation.
result Established upper bounds on estimation errors with optimal rates for unbounded density ratios.
Estimates for plate eigenvalues with nonzero Poisson's ratio.
problem Estimating eigenvalues of a free plate with nonzero Poisson's ratio.
method Using Fourier transform to derive estimates.
result Kroger-type estimates for sums of eigenvalues.