PQMass assesses generative model quality using chi-squared tests.
problem Assessing the quality of generative models without density assumptions.
method Divides sample space into regions, applies chi-squared tests to p-values.
result Effectively assesses generative model quality, novelty, and diversity.
Metrics assess uncertainty structure and distribution for regression models.
problem Quantifying uncertainty in high-dimensional and nonlinear regression tasks.
method Two bounded comparison metrics for uncertainty structure and distribution.
result DNNs and DNOs provide encouraging uncertainty metric values in high dimensions.
A novel variational inference based resampling framework is proposed to evaluate the robustness and generalization capability of deep learning models with respect to distribution shift. We use Auto Encoding Variational Bayes to find a latent representation of the data, on which a Variational Gaussian Mixture Model is a…
The paper analyzes the risk of investing in a basket of 27 cryptocurrencies using statistical distributions.
problem Risk assessment of capital allocation in a basket of cryptocurrencies.
method Used statistical tests to determine the most appropriate distribution (SDI) for modeling returns, and adapted the generalized Pareto distribution for tail risk assessment.
result Found that a combination of stable and generalized Pareto distributions provides a more accurate risk assessment for the basket of cryptocurrencies.
ABROCA assesses algorithmic bias, revealing skewed distributions that inflate results.
problem Detecting nuanced performance differences in classifier fairness.
method Study of ABROCA metric's statistical properties under various conditions.
result ABROCA distributions are skewed, inflating results by chance in imbalanced classes.
Fast risk assessment for autonomous vehicles using learned agent futures.
problem Risk assessment for autonomous vehicles given probabilistic predictions of other agents' futures.
method Non-sampling based methods using deep neural networks for probabilistic predictions, with Gaussian and non-Gaussian mixture models for agent positions and controls.
result Effective risk assessment for low probability events using learned models of agent futures.
Develops method to assess feature importance in black-box models for unconditional distribution.
problem Lack of methods to analyze feature importance in black-box models for unconditional distribution.
method Approximation method to compute feature importance curves for unconditional distribution.
result Produces sparse and faithful results, computationally efficient.
MIRA scores assess conditional distribution accuracy using joint samples.
problem Assessing the accuracy of candidate conditional distributions.
method Analytic expression for Mira score based on equal probability mass regions.
result Mira enables Bayesian model comparison by quantifying alignment with true process.
Unified four trade-off curves for assessing generative model proximity.
problem Quantitative assessment of proximity between two probability distributions.
method Unified four existing curves: PR, Lorenz, ROC, and Rényi divergence frontiers.
result Explicit relationship between PR and Lorenz curves with domain adaptation bounds.
Paper proposes a new method to assess synthetic data generators.
problem Assessing the quality of implicit generative models.
method Kernelised Stein Statistic (KSD) test based on non-parametric Stein operator.
result Improved power performance compared to existing approaches.
New method assesses financial and cyber risks under uncertainty.
problem Uncertainty in risk assessment for financial and cyber systems.
method Combines stochastic approximation and distorted mix method to compute worst case average value at risk.
result Efficient algorithm for tail uncertainty in multivariate distributions.
Paper converts quantiles to cumulative distribution functions to simplify risk measures.
problem Technical assumptions in risk measure calculations.
method Invention of converting integrated quantiles to integrated cumulative distribution functions.
result Avoids the need for probability density function existence.
Paper proposes a new model to assess risks in energy storage systems considering both exogenous and endogenous uncertainties.
problem Current risk assessment ignores the stochastic nature of energy storage availability.
method Data-driven unified model with exogenous and endogenous uncertainty description for four types of generic energy storage.
result Comparative results show more severe risks for endogenous uncertainty, suggesting new strategies for system operators.
Kernel tests assess equivalence between distributions without assuming specific moments.
problem Traditional goodness-of-fit tests fail to detect meaningful distributional differences.
method Proposes kernel-based tests using kernel Stein discrepancy and Maximum Mean Discrepancy.
result Tests assess the absence of meaningful distributional differences under controlled error rates.
BIF assesses feature importance using Dirichlet distribution and Bayesian inference.
problem Quantitative feature importance assessment in statistical models.
method Utilizes Dirichlet distribution for probabilistic feature importance assessment via approximate Bayesian inference.
result Learned importance provides relative significance and confidence quantification of features.
Study improves data quality assessment for structural monitoring data.
problem Ensuring reliability of structural health monitoring data.
method Probabilistic data quality assessment using a conditional diffusion model.
result Significantly improves accuracy of data quality assessment.
In risk management, tail risks are of crucial importance. The assessment of risks should be carried out in accordance with the regulatory authority's requirement at high quantiles. In general, the underlying distribution function is unknown, the database is sparse, and therefore special tail models are used. Very often…
The paper calculates ruin probabilities for insurers with phase-type distributed claims.
problem Calculating ruin probabilities for insurers with specific claim distributions.
method Change-of-measure technique applied to phase-type distributed claim amounts.
result The mixture of Erlangs best fits real-world loss data, improving risk assessment.
For nonlinear supervised learning models, assessing the importance of predictor variables or their interactions is not straightforward because it can vary in the domain of the variables. Importance can be assessed locally with sensitivity analysis using general methods that rely on the model's predictions or their deri…
TransCORALNet uses transformer and CORAL for supply chain credit assessment with cold start.
problem Supply chain credit assessment for new borrowers with limited data.
method Two-stream transformer CORAL networks with domain adaptation and LIME.
result TransCORALNet outperforms state-of-the-art models in accuracy.
The Surprise index assesses autonomous systems' competency in uncertain environments.
problem Evaluating competency of autonomous systems in dynamic, uncertain environments.
method Surprise index, a measure that quantifies system performance based on available data.
result The Surprise index can be computed for dynamic systems with Gaussian marginal distributions.
Study assesses health plan risk measures for Solvency Capital Requirement.
problem Assessing risk measures for health plans to meet Solvency Capital Requirement.
method Three-part regression model with three GLMs for claim counts, episode allocation, and severity.
result Reduction in regression models compared to traditional methods.
Study quantifies distribution shifts and uncertainties to improve machine learning model robustness.
problem Distribution shifts between training and test datasets impact model generalization and robustness.
method Synthetic data generation and quantitative measures (KL divergence, JS distance, Mahalanobis distance) to assess data similarity and model uncertainty.
result Utilizing statistical measures like Mahalanobis distance helps assess distribution shift and model uncertainty.
Proposes CCE to assess point-wise reliability of neural network predictions.
problem Overconfidence and misaligned predictive distributions in neural networks.
method Introduces Conditional Congruence (CCE) metric using conditional kernel mean embeddings.
result CCE exhibits correctness, monotonicity, reliability, and robustness in high-dimensional regression tasks.
The paper analyzes heavy-tailed multivariate distributions in non-stationary systems using random matrix theory.
problem Risk assessment for rare events in complex, non-stationary systems.
method Generalized scalar product between correlation matrices, model for non-stationary fluctuations.
result Formulae for multivariate distributions with reduced parameters, facilitating applications.
Bayesian framework uses unlabeled data to improve fairness assessment.
problem Reliable fairness assessment with limited labeled data.
method Hierarchical latent variable model with Bayesian inference.
result Significant reduction in estimation error for fairness metrics.
The paper introduces a method to assess machine translation quality with confidence intervals.
problem Evaluating the uncertainty and quality of machine translation.
method Utilizes conformal predictive distributions to produce prediction intervals with guaranteed coverage.
result The method outperforms a baseline on six language pairs in terms of coverage and sharpness.
Study improves statistical power for detecting algorithmic bias in educational data.
problem Challenges in measuring algorithmic bias using ABROCA due to skewed distribution.
method Investigates ABROCA's distributional properties and proposes nonparametric randomization tests.
result ABROCA-based bias assessments are underpowered in typical EDM sample sizes.
BREEDS benchmarks assess model robustness to subpopulation shifts.
problem Measuring model robustness to novel subpopulation shifts.
method Controlled synthesis of realistic distribution shifts using class structure.
result Validated model sensitivity and effectiveness of robustness interventions.
With the deployment of online monitoring systems in distribution networks, massive amounts of data collected through them contains rich information on the operating states of the networks. By leveraging the data, an unsupervised approach based on bidirectional generative adversarial networks (BiGANs) is proposed for op…
CPA framework assesses conditional validity of conformal prediction.
problem Challenges in evaluating conditional validity of conformal prediction.
method Reframes conditional coverage evaluation as a supervised learning task.
result Establishes convergence rates and proves CVI consistency.
In retailer management, the Newsvendor problem has widely attracted attention as one of basic inventory models. In the traditional approach to solving this problem, it relies on the probability distribution of the demand. In theory, if the probability distribution is known, the problem can be considered as fully solved…
Paper introduces symmetric divergence link models for probability distributions.
problem Symmetric divergence measures for probability distributions.
method Two general classes of link models: one for survival functions and another for cumulative probability distribution functions.
result Advantages of symmetric divergence measures over asymmetric measures for model averaging and feature assessment.
PS-DME evaluates model performance and reliability after data-dependent selection.
problem Evaluating model performance and reliability when data is used for selection and evaluation.
method Post-selection distributional model evaluation (PS-DME) using e-values to control false coverage rate.
result PS-DME provides reliable comparison of model configurations across different reliability levels.
WRSE predicts dynamic survival distributions in ICU patients.
problem Dynamic assessment of ICU patient mortality risk.
method Non-parametric weighted-resolution ensemble model combining binary classifiers.
result Competitive results with state-of-the-art models, reducing training time.
Computer vision model automates residual plot assessment for diagnosing model assumptions.
problem Automating residual plot assessment for model diagnostics.
method Trains a computer vision model to predict disparity between residual distributions and reference distributions using Kullback-Leibler divergence.
result Computer vision model is less sensitive to non-linearity but more sensitive than human judgment and conventional tests.
DRF improves confidence and uncertainty assessment for multivariate conditional distributions.
problem Estimating multivariate conditional distributions with confidence and uncertainty.
method Developed a bootstrap approximation of the asymptotic distribution of DRF to derive inferential tools.
result Asymptotic coverage guarantees for confidence regions and hypothesis testing.
Fermat-Torricelli points help assess investment risks by smoothing series data.
problem Analyzing investment risks in series with large variance, nonlinear trends, or non-normal distributions.
method Construct Fermat-Torricelli points to reduce random component influence.
result Smoothing series by Fermat-Torricelli points reduces risk assessment errors.
The scaled complex Wishart distribution is a widely used model for multilook full polarimetric SAR data whose adequacy has been attested in the literature. Classification, segmentation, and image analysis techniques which depend on this model have been devised, and many of them employ some type of dissimilarity measure…
New framework assesses deep learning models for spatio-temporal data with missing data.
problem Challenges in assessing deep learning models for spatio-temporal data with missing and heterogeneous data.
method Residual correlation analysis framework using spatio-temporal graphs and asymptotically distribution-free summary statistics.
result Identification and localization of regions where predictive performance can be improved.
Paper proposes a framework to assess model robustness against adversarial actions.
problem Ensuring model reliability in deployment with varied adversarial conditions.
method Developed a versatile framework for evaluating SVR and relaxed optimization models' robustness.
result Demonstrates model vulnerability without requiring additional test data.
The paper proposes using entropy to assess model confidence.
problem Discarded information in probability distributions hinders model confidence assessment.
method Entropy methods applied to assess model confidence.
result Theoretical explanation of confidence degradation in Naive Bayes model.
The paper proposes a method to assess survey data credibility without needing many samples, regardless of data dimension.
problem Assessing the credibility of survey data across different dimensions.
method Task-based approach and model-specific distance metric for verifying survey data credibility in regression models.
result The sample complexity of the proposed algorithm is independent of the data dimension, making it more efficient.
We show how to reduce the problem of computing VaR and CVaR with Student T return distributions to evaluation of analytical functions of the moments. This allows an analysis of the risk properties of systems to be carefully attributed between choices of risk function (e.g. VaR vs CVaR); choice of return distribution (p…
New method allocates capital based on tail central moments for financial risk assessment.
problem Inability of CTE-based capital allocation to reflect tail behavior of losses.
method Developed TCM-based capital allocation for normal mean-variance mixture distributions.
result TCM-based method captures tail risk contributions not detected by CTE.
The paper examines how machine learning tools in justice settings can unfairly affect different racial groups.
problem Machine learning tools in justice settings can unfairly affect different racial groups.
method Exploring different ideas of racial equity and their computational trade-offs.
result Computation alone is unlikely to solve the unfairness in machine learning tools for justice settings.
Distributed computing offers a high degree of flexibility to accommodate modern learning constraints and the ever increasing size of datasets involved in massive data issues. Drawing inspiration from the theory of distributed computation models developed in the context of gradient-type optimization algorithms, we prese…
CREDO assesses decision optimality under uncertainty without assuming a model.
problem Uncertainty in decision-making without reliable quantification of optimality.
method CREDO uses the inverse feasible region and conformal prediction balls to estimate decision optimality probability.
result CREDO provides accurate, efficient, and reliable evaluations of decision optimality.