The study improves the assessment of fairness in face recognition using ROC curves and statistical guarantees.
problem Improving the assessment of fairness in face recognition systems.
method Proves asymptotic guarantees for empirical ROC curves and fairness metrics, and introduces a recentering technique to avoid bootstrap pitfalls.
result Demonstrates the practical relevance of the methods for assessing fairness in face recognition systems.
New method approximates CV for model assessment and selection.
problem Efficient model assessment and selection with large number of folds.
method Approximates expensive refitting with a single Newton step warm-started from full training set optimizer.
result Uniform non-asymptotic, deterministic model assessment guarantees for approximate CV.
The study simplifies assessing overlap in logistic regression models using empirical likelihood.
problem Assessing overlap in multidimensional logistic regression models.
method Translation of Silvapulle's condition to empirical likelihood maximization, mechanized with R code.
result Minimal overlapping structures are cataloged in dimensions less than four, providing rules for higher dimensions.
Risk, including economic risk, is increasingly a concern for public policy and management. The possibility of dealing effectively with risk is hampered, however, by lack of a sound empirical basis for risk assessment and management. The paper demonstrates the general point for cost and demand risks in urban rail projec…
The Basel II internal ratings-based (IRB) approach to capital adequacy for credit risk implements an asymptotic single risk factor (ASRF) model. Measurements from the ASRF model of the prevailing state of Australia's economy and the level of capitalisation of its banking sector find general agreement with macroeconomic…
Paper introduces active Bayesian method for assessing black-box classifiers efficiently.
problem Need to assess performance of black-box classifiers reliably with limited labels.
method Develops inference strategies and proposes active Bayesian framework for efficient instance selection.
result Significant gains in performance assessment with fewer labels compared to traditional methods.
Synthetic experiments are crucial for assessing causal machine learning methods.
problem Current empirical evaluations of causal machine learning methods are insufficient and unreliable.
method Propose principles for conducting rigorous empirical analyses with synthetic data.
result Rigorous synthetic experiments are essential for building trust in causal machine learning methods.
The study introduces a holdout-based framework to assess synthetic data fidelity and privacy.
problem Evaluating the quality and privacy of synthetic data solutions for mixed-type tabular data.
method Holdout-based empirical assessment framework measuring fidelity and privacy risk.
result Synthetic data samples are as close to the training as to the holdout data, indicating generalization and independence from individual records.
Develops methods to improve demand counterfactuals from imperfect proxies.
problem Imperfect proxies in demand models lead to biased counterfactuals and invalid inference.
method Practical toolkit for market-level and individual data, requiring minimal computation.
result Improves substitution prediction and counterfactual performance.
Flexible framework assesses multilevel data group heterogeneity.
problem Multilevel data structure complicates model selection.
method Flexible framework for assessing differences between levels of grouping variables.
result Framework reliably identifies relevant multilevel components.
The paper assesses machine learning robustness with covariate perturbations.
problem Ensuring robustness of machine learning models against adversarial attacks and data changes.
method Proposes a framework using covariate perturbation techniques to assess model robustness.
result Demonstrates the effectiveness of the approach in comparing robustness across models and identifying instabilities.
A cluster tree provides a highly-interpretable summary of a density function by representing the hierarchy of its high-density clusters. It is estimated using the empirical tree, which is the cluster tree constructed from a density estimator. This paper addresses the basic question of quantifying our uncertainty by ass…
This paper presents an assessment of global economic energy potentials for all major natural energy resources. This work is based on both an extensive literature review and calculations using natural resource assessment data. Economic potentials are presented in the form of cost-supply curves, in terms of energy flows …
A new method assesses regression models' global optimality.
problem Challenges in evaluating regression models without access to true data.
method Information Teacher framework based on Shannon mutual information.
result Demonstrates capability to detect global optimality.
Develops a framework to assess systemic risk in the economy using bank-firm network data.
problem Measuring systemic risk in the economy using multilayer network data.
method Unified framework combining techniques to reconstruct multilayer economy structure from bank and firm balance sheets, and dynamics of shock propagation.
result Identifies systemically important firms and banks, and assesses systemic risk determinants.
Paper assesses error estimates of Random Forests classification.
problem Quantitative assessment of Random Forests error estimates.
method Theoretical and empirical investigation of various error estimation methods.
result Random Forests' error estimates are closer to true error rate than average prediction error.
The stochastic block model accurately describes most empirical networks but struggles with large diameter and slow-mixing networks.
problem Assessing the quality of fit of the stochastic block model for empirical networks.
method Posterior predictive model checking using network descriptors.
result The stochastic block model can accurately describe most empirical networks but struggles with large diameter and slow-mixing networks.
We conduct an empirical study using the quantile-based correlation function to uncover the temporal dependencies in financial time series. The study uses intraday data for the S\&P 500 stocks from the New York Stock Exchange. After establishing an empirical overview we compare the quantile-based correlation function to…
Study assesses hyperparameter tuning for causal inference with DML.
problem Optimizing hyperparameters for causal inference with DML.
method Empirical simulation study using DML approach.
result Hyperparameter tuning crucial for causal estimation with DML.
New measures assess differences in causal graphs' separations.
problem Evaluating causal discovery algorithms' output.
method Proposes new distance measures capturing causal graphs' separations.
result Proposed distances assess differences in causal graphs' separations.
The ROC curve is widely used to assess the quality of prediction/classification/ranking algorithms, and its properties have been extensively studied. The precision-recall (PR) curve has become the de facto replacement for the ROC curve in the presence of imbalance, namely where one class is far more likely than the oth…
AgraSSt assesses graph generators using Stein operators and kernel discrepancies.
problem Assessing the quality of graph generators that are implicit or not in explicit form.
method AgraSSt uses Stein operators and kernel discrepancies to assess graph generators, providing interpretable criticisms.
result Theoretical guarantees and empirical validation for various graph models.
Develops framework for valuing and assessing risk of renewable PPAs.
problem Valuation and risk assessment of non-standard renewable PPAs.
method Formalizes payoff structures, derives fair contract prices, proposes market risk-assessment methodology.
result Fair prices and risk profiles vary across technologies and contractual structures.
New method assesses individual training points' privacy risk without retraining.
problem Privacy vulnerability of individual training points in membership inference attacks.
method Derives a closed-form decomposition of individual black-box MIA vulnerability, extending to deep networks.
result Proposes a surrogate score operating on last-layer representations that requires only a single trained model.
Study improves financial risk assessment using ARMA-APARCH-EVT models with HACs.
problem Improving risk assessment in financial portfolios.
method ARMA-APARCH-EVT-HAC model for volatility and extreme value forecasting.
result Empirical analysis shows the model's effectiveness in international stock market data.
Testing symmetry of a probability distribution is a common question arising from applications in several fields. Particularly, in the study of observables used in the analysis of stock market index variations, the question of symmetry has not been fully investigated by means of statistical procedures. In this work a di…
Objectives: Discussions of fairness in criminal justice risk assessments typically lack conceptual precision. Rhetoric too often substitutes for careful analysis. In this paper, we seek to clarify the tradeoffs between different kinds of fairness and between fairness and accuracy. Methods: We draw on the existing liter…
Proposes CCE to assess point-wise reliability of neural network predictions.
problem Overconfidence and misaligned predictive distributions in neural networks.
method Introduces Conditional Congruence (CCE) metric using conditional kernel mean embeddings.
result CCE exhibits correctness, monotonicity, reliability, and robustness in high-dimensional regression tasks.
Graph kernels assess graph similarity for various applications.
problem Assessing similarity between graphs for predictions.
method Review and comparison of existing graph kernels.
result State-of-the-art graph kernels reviewed and compared.
The paper evaluates and improves uncertainty estimates in neural networks for safety-critical applications.
problem Quantifying uncertainty in neural networks for safety-critical systems.
method Proposes a statistical test for evaluating uncertainty realism in neural networks and transfers a classification architecture to image-to-image tasks.
result The variational U-Net architecture significantly improves uncertainty realism in image-to-image tasks compared to a plain model.
The paper assesses conditions for the uniqueness of k-means clustering.
problem Conditions for the uniqueness of k-means clustering.
method Analyzes the choice of k and provides necessary and sufficient conditions for uniqueness.
result Determines the asymptotic distribution of the within cluster sum of squares (WCSS) and provides a bootstrap test for uniqueness.
The paper finds a pervasive and severe bias in accounting semi-identity models.
problem Bias in investment-cash flow sensitivity models.
method Augmented specification with a bias-capturing variable tested across multiple databases.
result The Accounting Semi-Identity (ASI) distortion is universal and severe, affecting 100% of databases and explaining more than 83% of total explained variance.
Framework for assessing fairness across similar predictive models.
problem Fairness in predictive models across different groups.
method Develops a framework for characterizing fairness over the set of good models under selective labels.
result Framework can replace or audit models for better fairness properties.
The AutoML task consists of selecting the proper algorithm in a machine learning portfolio, and its hyperparameter values, in order to deliver the best performance on the dataset at hand. Mosaic, a Monte-Carlo tree search (MCTS) based approach, is presented to handle the AutoML hybrid structural and parametric expensiv…
For nonlinear supervised learning models, assessing the importance of predictor variables or their interactions is not straightforward because it can vary in the domain of the variables. Importance can be assessed locally with sensitivity analysis using general methods that rely on the model's predictions or their deri…
Calibration without labels in multiple testing
problem Interpretable error probabilities in large-scale hypothesis testing
method Constructing pseudo-labels from spacings of ordered p-values result Finding that q-value can be severely miscalibrated In this paper we present an analysis of power law statistics on land markets. There have been no other studies that have analyzed power law statistics on land markets up to now. We analyzed a database of the assessed value of land, which is officially monitored and made available to the public by the Ministry of Land, …
Paper introduces a method to assess liquidity risk in meme tokens using entity-linked address analysis.
problem High market volatility and vulnerability to manipulation in meme tokens.
method Multi-dimensional approach integrating fund flow analysis, behavioral similarity, and anomalous transaction detection.
result Significant disparities between apparent and actual liquidity in meme token markets.
The bootstrap provides a simple and powerful means of assessing the quality of estimators. However, in settings involving large datasets, the computation of bootstrap-based quantities can be prohibitively demanding. As an alternative, we present the Bag of Little Bootstraps (BLB), a new procedure which incorporates fea…
Deep learning improves macroeconomic forecasting and risk assessment.
problem Improving accuracy in macroeconomic forecasting and sovereign risk assessment.
method Nowcasting and forecasting using deep learning techniques.
result Deep learning methods outperform traditional econometric techniques in out-of-sample performance.
Study evaluates and compares traditional and causal machine learning methods for estimating direct price effects of environmental amenities.
problem Estimating direct price effects of environmental amenities in housing markets.
method Empirical Monte Carlo simulation to compare traditional regression and causal machine learning approaches.
result Causal Machine Learning (CML) methods, particularly causal forest DID, perform comparably to generalized DID in most scenarios.
DECS tool assesses swap rates of DEXes and Fusion outperforms competitors.
problem Lack of unbiased swap rate comparisons in decentralized finance.
method Swap transaction monitoring and simulation techniques.
result 1inch Classic and Fusion consistently outperform competitors in swap rates.
Algorithmic risk assessments are increasingly used to help humans make decisions in high-stakes settings, such as medicine, criminal justice and education. In each of these cases, the purpose of the risk assessment tool is to inform actions, such as medical treatments or release conditions, often with the aim of reduci…
ScoreAG generates unrestricted adversarial images maintaining semantic integrity.
problem Limited robustness evaluations due to ℓp-norm constraints. method Score-Based Adversarial Generation (ScoreAG) using score-based generative models.
result ScoreAG improves robustness assessments across multiple benchmarks.
Recidivism prediction provides decision makers with an assessment of the likelihood that a criminal defendant will reoffend that can be used in pre-trial decision-making. It can also be used for prediction of locations where crimes most occur, profiles that are more likely to commit violent crimes. While such instrumen…
Saliency methods can aid understanding of deep neural networks. Recent years have witnessed many improvements to saliency methods, as well as new ways for evaluating them. In this paper, we 1) present a novel region-based attribution method, XRAI, that builds upon integrated gradients (Sundararajan et al. 2017), 2) int…
dcFCI discovers causal relationships robustly under latent confounding and mixed data.
problem Causal discovery under latent confounding and unfaithfulness.
method dcFCI integrates a new score to assess PAG compatibility, guided by FCI search.
result Significantly outperforms state-of-the-art methods in small and heterogeneous datasets.
A new framework assesses financial and ESG risks for sustainable investing.
problem Measuring risk and reward in sustainable investing considering environmental, social, and governance factors.
method Proposes axiomatic definitions for ESG-coherent risk measures and reward-risk ratios based on bivariate random variables.
result Empirical analysis ranks stocks using the proposed measures.