Comparison of decision curve analysis and cost curves for model evaluation.
problem Evaluating classification performance across different operating contexts.
method Comparison of Decision Curve Analysis (DCA) and Cost Curves.
result DCA and Cost Curves are closely related, with Brier curves being more generally applicable.
Deep networks have recently been shown to be vulnerable to universal perturbations: there exist very small image-agnostic perturbations that cause most natural images to be misclassified by such classifiers. In this paper, we propose the first quantitative analysis of the robustness of classifiers to universal perturba…
Geometric analysis of ROC and PR curves for binary classification.
problem Understanding classifier behavior and selection of optimal operating points.
method Geometric perspective on ROC and PR curves, focusing on the composition function G. result Many binary classification metrics are functions of G=Fp∘Fn−1, facilitating better classifier optimization. This study models Burundi's bond market yield curve using Nelson-Siegel and Svensson models.
problem Modeling the yield curve of Burundian bond market for financial analytics.
method Collected treasury securities auction reports, computed zero-coupon rates, and applied Nelson-Siegel and Svensson models.
result Nelson-Siegel model is optimal for Burundian yield curve modeling.
Deep ROC analysis improves model selection and interpretation in medical and AI applications.
problem Inadequate performance measures for binary classifiers.
method Deep ROC analysis, translating AUC and partial AUC into balanced average accuracy and post-test measures.
result Deep ROC analysis provides balanced average accuracy, average sensitivity, and average specificity.
Develops a framework to assess infrastructure reliability under natural and malicious events.
problem Assessing reliability and costs of infrastructure under various hazards.
method Coupling mechanical reliability analyses with economical reliability analyses using probabilistic considerations.
result Indicators of probable cost of failure for infrastructure, aiding safety investments.
Optimal cutoff interval for risk scores improves binary classification accuracy.
problem Improving binary classification accuracy with abstention.
method Determines optimal cutoff interval for risk scores, refraining from decisions outside this interval.
result Minimizes classification margin and maximizes accuracy within the interval.
The Receiver Operating Characteristic (ROC) curve is a representation of the statistical information discovered in binary classification problems and is a key concept in machine learning and data science. This paper studies the statistical properties of ROC curves and its implication on model selection. We analyze the …
LxCIM metric improves binary classification performance evaluation.
problem Evaluation metrics for binary classification are often not invariant to local class exchange.
method Proposes LxCIM, a rank-based metric invariant to local class exchange.
result LxCIM addresses limitations of existing metrics like AUROC.
Study finds AUC is most consistent across different prevalence in binary classification.
problem Consistency of model evaluation metrics across varying prevalence in binary classification.
method Analysis of 156 data scenarios with 18 metrics, 5 models, and a random guess model.
result AUC has the smallest variance in evaluating individual models and ranking of models.
Critiques binary classification evaluation methods, advocating for proper scoring rules.
problem The dominance of top-K metrics and fixed-threshold evaluations in machine learning.
method Introduces a decision-theoretic framework mapping evaluation metrics to their use cases, and implements a clipped Brier score variant.
result Demonstrates the clinical utility of proper scoring rules through a Python package, exttt{briertools}.
Study on error probabilities of machine learning classification techniques using large deviations theory.
problem Performance analysis of machine learning binary classification techniques.
method Large deviations theory applied to Data-Driven Decision Function (D3F) for error probability analysis.
result Classification error probabilities vanish exponentially, with an asymptotic formula providing precise error rate estimates.
In practical work with American put options, it is important to be able to know when to exercise the option, and when not to do so. In computer simulation based on the standard theory of geometric Brownian motion for simulating stock price movements, this problem is fairly easy to handle for options with a short lifesp…
The cost-benefit analysis formulates the holy trinity of objectives of project management - cost, schedule, and benefits. As our previous research has shown, ICT projects deviate from their initial cost estimate by more than 10% in 8 out of 10 cases. Academic research has argued that Optimism Bias and Black Swan Blindn…
Investment decision triggered by a convex curve in a two-factor uncertainty model.
problem Optimal irreversible investment in a company with two products whose prices follow geometric Brownian motions.
method Two-dimensional optimal stopping problem, nonlinear integral equation, convex curve characterization.
result Optimal investment decision is characterized by a convex curve, unique solution to a nonlinear integral equation.
Many problems that appear in biomedical decision making, such as diagnosing disease and predicting response to treatment, can be expressed as binary classification problems. The costs of false positives and false negatives vary across application domains and receiver operating characteristic (ROC) curves provide a visu…
A robust machine learning approach forecasts U.S. Treasury yields, reducing risk for investors.
problem Noisy and uncertain U.S. Treasury yields pose risk to forecast users.
method Formulates yield curve forecasting as a distributionally robust problem, combining factor models and machine learning.
result Robust forecast combinations improve out-of-sample performance across different maturity periods.
Transforms curves and surfaces for efficient geometric analysis.
problem Efficiently analyzing and comparing curves and surfaces.
method Square root velocity transformation for curves and intrinsic comparison for surfaces.
result Fundamental geometric properties of curves under the transformation.
Develops BPDS for better financial portfolio decisions.
problem Model uncertainty in financial time series forecasting.
method Bayesian dynamic modelling and predictive decision synthesis.
result Improved predictive and decision outcomes compared to traditional Bayesian analysis.
In many healthcare settings, intuitive decision rules for risk stratification can help effective hospital resource allocation. This paper introduces a novel variant of decision tree algorithms that produces a chain of decisions, not a general tree. Our algorithm, α-Carving Decision Chain (ACDC), sequentially carves o…
A meta-learning approach for efficient algorithm selection in budget-limited scenarios.
problem Efficiently selecting the best-performing machine learning algorithm with limited computational resources.
method A Markov Decision Process framework where an agent decides whether to train, wake up, or start new algorithms based on partial learning curves.
result Meta-learning from learning curves improves algorithm selection, especially when learning curves do not intersect frequently.
We investigate a long-debated question, which is how to create predictive models of recidivism that are sufficiently accurate, transparent, and interpretable to use for decision-making. This question is complicated as these models are used to support different decisions, from sentencing, to determining release on proba…
Capacity control, the bias/variance dilemma, and learning unknown functions from data, are all concerned with identifying effective and consistent fits of unknown geometric loci to random data points. A geometric locus is a curve or surface formed by points, all of which possess some uniform property. A geometric locus…
We propose a Bayesian model that predicts recovery curves based on information available before the disruptive event. A recovery curve of interest is the quantified sexual function of prostate cancer patients after prostatectomy surgery. We illustrate the utility of our model as a pre-treatment medical decision aid, pr…
Three geometric analysis results on curve flows and Lie groups.
problem Analyzing geometric flows and Lie groups.
method Curve-shortening flow, point-wise curvature preserving flow, Lie group analysis.
result Interpolation between Sol and hyperbolic space in Lie groups.
Study characterizes bladder motion using dynamic MRI and statistical analysis.
problem Limited volume coverage in dynamic MRI sequences hinders 3D shape reconstruction.
method 3D dense velocity measurements, LDDMM framework, statistical characterization, mean curvature changes, surface deformation analysis.
result Stable shape descriptor for characterizing bladder surface dynamics.
Curve shortening flow shrinks curves to points.
problem The behavior of curves under curve shortening flow.
method Nonlinear partial differential equations, maximum principle, monotonicity formulas, Harnack inequalities, blowup analysis.
result The curve shortening flow shrinks any closed embedded curve in the plane to a round point.
Paper introduces impact curves for evaluating binarized regression models with varying costs.
problem Evaluating binarized regression models with varying costs and instance-specific utility.
method Proposes impact curves to optimize binary decisions across different utilities.
result Impact curves identify conditions where one model is favored over another and quantify model improvement.
Multiverse analysis helps prevent fairness hacking and evaluate model design decisions.
problem Downstream effects of ADM systems depend on implicit design and evaluation decisions.
method Turn implicit decisions into explicit ones, create a grid of decision combinations, compute fairness and performance metrics.
result Decisions regarding evaluation can lead to vastly different fairness metrics for the same model.
The study evaluates AI model performance measures for medical use.
problem Selecting appropriate performance measures for AI models in medical practice.
method Assessed 32 performance measures across five domains for binary outcomes.
result 17 measures are both proper and reflect decision-analytic performance.
The Chain-of-Decision approach improves forecasting of financial professionals' trading decisions.
problem Challenges in forecasting professionals' behaviors, especially in trading decisions.
method Integrates an opinion-generator-in-the-loop to provide subjective analysis based on news items.
result Promising improvements in the proposed tasks' performance.
Principal Component Analysis (PCA) is the most common nonparametric method for estimating the volatility structure of Gaussian interest rate models. One major difficulty in the estimation of these models is the fact that forward rate curves are not directly observable from the market so that non-trivial observational e…
We use learning curves to analyze deep networks and evaluate model design.
problem Evaluate design choices in deep networks.
method Propose a method to robustly estimate learning curves, abstract their parameters, and evaluate different parameterizations.
result Interesting observations on the effectiveness of different parameterizations.
The paper certifies decision trees against evasion attacks using program analysis.
problem Vulnerability of decision tree models to evasion attacks by maliciously crafted perturbations.
method Transform decision trees into imperative programs for program analysis, leveraging abstract interpretation.
result Soundly verifies security guarantees of decision tree models, yielding minimal false positives.
New metrics on curve spaces improve shape analysis.
problem Discretization of curve spaces and metric completeness.
method Sobolev metrics on discrete regular curves, completeness analysis.
result The finite-dimensional Riemannian manifolds are complete.
Study local and global aspects of complex plane curve embeddings.
problem Local and global problems of complex plane curve embeddings.
method Braid monodromy, local and global analysis.
result Historical progress in understanding complex plane curve embeddings.
Paper proves conjecture about star-shaped curves evolving under GAPF, but not always preserves star shape.
problem What conditions guarantee global existence of Gage's area-preserving flow for nonconvex initial curves?
method Using Dittberner's singularity analysis theory, constructed a ``flying wing'' curve to show limitations.
result Gage's area-preserving flow does not always preserve star-shapedness of evolving curves.
The Large Synoptic Survey Telescope will complete its survey in 2022 and produce terabytes of imaging data each night. To work with this massive onset of data, automated algorithms to classify astronomical light curves are crucial. Here, we present a method for automated classification of photometric light curves for a…
Optimizes predictions for specific tasks using parametrized decision analysis.
problem Optimizing predictions for specific decision tasks of interest.
method Designs a class of parametrized actions for Bayesian decision analysis.
result Derives efficient and interpretable solutions for various action parametrizations and loss functions.
Study biharmonic curves in warped product manifolds with curvature analysis.
problem Characterize biharmonic curves in warped product manifolds.
method Establish a main theorem, analyze four cases, construct examples.
result Reveal curvature-related characteristics of biharmonic curves.
This paper proposes a new VoI analysis framework for complex decision problems.
problem Optimizing resource allocation for information collection in decision-making under uncertainty.
method Surrogate-based framework for Value of Information analysis, integrating knowledge sharing and adaptive training.
result Accurate and robust estimates of VoI with fewer model evaluations compared to state-of-the-art methods.
Proposes MCC-F1 curve for better binary classification evaluation.
problem Misleading performance evaluations with ROC and PR curves for imbalanced data.
method Introduces MCC-F1 curve combining MCC and F1 score.
result MCC-F1 curve provides clearer classifier differentiation.
New method uses contours of segmented images for X-ray classification.
problem Classifying X-ray images of segmented radiography.
method Develops a new approach for image analysis of multivariate planar curves, addressing alignment issues.
result Demonstrates the robustness and appeal of the proposed method through detection of cardiomegaly and numerical experiments.
Statistical approaches for Functional Data Analysis concern the paradigm for which the individuals are functions or curves rather than finite dimensional vectors. In this paper, we particularly focus on the modeling and the classification of functional data which are temporal curves presenting regime changes over time.…
Sharp bounds for curve isoperimetric deficit derived.
problem Finding sharp bounds for the isoperimetric deficit of curves.
method Fourier analysis applied to derive Wirtinger-type inequalities.
result Sharp lower and upper bounds for the isoperimetric deficit proved.
We analyze DMs using spectral methods to design effective noise schedules.
problem Lack of theoretical foundation for synthesis process decisions in DMs.
method Introduced a frequency response perspective based on Gaussianity assumption.
result Proposed a spectral transfer function to understand DM inference process.
New method simplifies ideal curve flow with length constraint.
problem Analyzing ideal curve flow with length constraint.
method Introduced length constraint to simplify sixth order curvature flow.
result Flow exists for all time and converges to a round circle.
New RL algorithms correct bias in dynamic data analysis.
problem Dynamic data generation and analysis create endogeneity issues.
method Instrument variable (IV)-based reinforcement learning (RL) algorithms.
result Established theoretical properties of IV-RL algorithms.