The paper proposes a test to assess rater accuracy while accounting for rater covariates.
problem Assessing the accuracy of raters in medical imaging and forensic studies.
method Covariate-adjusted homogeneity test to determine differences in accuracy among multiple rater groups.
result The proposed test identifies statistically significant differences among five participant groups in a face recognition study.
PerSense assesses personality traits from text for commonsense reasoning.
problem Estimating human personality traits from text for mental health analysis.
method Aggregated Probability Density Functions (PDF) and Machine Learning (ML) models.
result PerSense algorithms achieve comparable results to ground truth data, with high accuracy for personality assessment and commonsense prediction.
Contrast trees assess machine learning accuracy, boosting improves performance.
problem Lack of accuracy assessment for machine learning results.
method Contrast trees and distribution boosting methods.
result Distribution boosting provides an assumption-free method for estimating outcome distributions.
Objectives: Discussions of fairness in criminal justice risk assessments typically lack conceptual precision. Rhetoric too often substitutes for careful analysis. In this paper, we seek to clarify the tradeoffs between different kinds of fairness and between fairness and accuracy. Methods: We draw on the existing liter…
Unified model predicts stock and systemic risks from diverse financial data.
problem Isolating financial tasks leads to missed cross-scale dependencies.
method Shared Transformer backbone with modular task heads for cross-modal attention and multi-task optimization.
result Uni-FinLLM significantly outperforms baselines in stock forecasting, credit-risk assessment, and systemic-risk detection.
Automatically assesses the quality of online health articles.
problem Lack of automated tools to evaluate the quality of online health information.
method Data mining approach using 10 quality criteria and feature selection.
result Classifier achieved 84%-90% accuracy on 10 criteria.
Study develops a new algorithm for assessing clinical trial abstracts.
problem Limited tools for assessing qualitative statements in clinical research.
method Developed a three-class sentiment classification algorithm using BERT model.
result Algorithm achieved 91.3% classification accuracy and 0.92 macro F1-Score.
Protein structure prediction has been a grand challenge problem in the structure biology over the last few decades. Protein quality assessment plays a very important role in protein structure prediction. In the paper, we propose a new protein quality assessment method which can predict both local and global quality of …
Paper introduces active Bayesian method for assessing black-box classifiers efficiently.
problem Need to assess performance of black-box classifiers reliably with limited labels.
method Develops inference strategies and proposes active Bayesian framework for efficient instance selection.
result Significant gains in performance assessment with fewer labels compared to traditional methods.
Framework for automatically assessing and correcting data quality issues without domain knowledge.
problem Ensuring data quality in datasets across various domains.
method Hybrid approach combining statistical and machine learning methods.
result Effective detection and correction of missing values, duplicates, and typographical errors.
MIRA scores assess conditional distribution accuracy using joint samples.
problem Assessing the accuracy of candidate conditional distributions.
method Analytic expression for Mira score based on equal probability mass regions.
result Mira enables Bayesian model comparison by quantifying alignment with true process.
The paper assesses quality measures for machine learning models using cross-validation.
problem Evaluating the accuracy and robustness of quality measures for machine learning models.
method Cross-validation approach to estimate prediction error and quantify explained variation. Confidence bounds and local quality measures derived from residuals.
result The reliability and robustness of quality measures are assessed through numerical examples and confidence bounds.
Dual-stage sEMG classification improves gesture recognition accuracy.
problem Improving accuracy in hand gesture recognition from sEMG signals.
method Dual-stage classification approach: first stage groups similar activities, second stage classifies within groups.
result Dual-stage classification yields significantly higher accuracy than single-stage approach.
Study evaluates deep learning methods for dermatology, finding they perform poorly under non-ideal conditions.
problem Lack of robustness of deep learning methods in dermatology under real-world conditions.
method Simulated non-ideal conditions on user-submitted dermatology images.
result Deep learning methods show significant drop in accuracy and prediction changes under non-ideal conditions.
TARP tests accuracy of generative posterior estimators.
problem Assessing the accuracy of posterior estimators from generative models.
method TARP coverage testing method.
result TARP can detect inaccurate inferences in high-dimensional spaces.
Flexible framework assesses multilevel data group heterogeneity.
problem Multilevel data structure complicates model selection.
method Flexible framework for assessing differences between levels of grouping variables.
result Framework reliably identifies relevant multilevel components.
This paper reviews metrics to assess AI model calibration accuracy.
problem AI model probabilities do not always match their true accuracy.
method Comprehensive review of 82 probability calibration metrics.
result Identified 4 classifier families and 1 object detection family of metrics.
Model assesses credit risk using behavioral data from Experian and Bank of Italy.
problem Improving credit risk assessment in financial institutions.
method Statistical and machine learning techniques applied to behavioral data from Experian and Bank of Italy.
result Demonstrates transferability of the model from private to central data.
Automated rock fragmentation assessment using deep learning and spatial statistics.
problem Assessing post-blast rock fragmentation in real-time.
method Fine-tuned YOLO12l-seg model for instance segmentation, followed by spatial statistics.
result Framework accurately assesses rock fragmentation patterns in real-time.
FinBERT-XRC model assesses financial report risk, offering transparent explanations.
problem Assessing post-event return volatility risk in financial reports.
method Deep-learning model FinBERT-XRC with explainability at word, sentence, and corpus levels.
result FinBERT-XRC outperforms state-of-the-art models in predictive accuracy.
Applying deep learning methods to mammography assessment has remained a challenging topic. Dense noise with sparse expressions, mega-pixel raw data resolution, lack of diverse examples have all been factors affecting performance. The lack of pixel-level ground truths have especially limited segmentation methods in push…
First European crop map created using satellite data.
problem Need for detailed parcel-level crop type mapping for EU.
method Used Sentinel-1 radar observations and LUCAS in-situ data.
result 80.3% overall accuracy for 19 crop types, highest for rape and turnip rape.
Satellite imagery improves house price prediction models.
problem Improving accuracy of housing price estimation models.
method Transfer learning from ImageNet-pretrained Inception-v3 model to satellite images.
result Achieved a 10% improvement in R-squared score.
Pairwise ranking aligns subjective clinical evaluations with objective indicators.
problem Aligning subjective clinical evaluations with objective indicators for improved diagnosis.
method Pairwise ranking methods to align subjective evaluations with objective indicators.
result The resulting score improves classification accuracy and provides a nuanced severity assessment.
In this paper, we propose a new measure to gauge the complexity of image classification problems. Given an annotated image dataset, our method computes a complexity measure called the cumulative spectral gradient (CSG) which strongly correlates with the test accuracy of convolutional neural networks (CNN). The CSG meas…
Uncertainty of labels in clinical data resulting from intra-observer variability can have direct impact on the reliability of assessments made by deep neural networks. In this paper, we propose a method for modelling such uncertainty in the context of 2D echocardiography (echo), which is a routine procedure for detecti…
Study combines quantum and classical deep learning for better credit risk assessment.
problem Enhancing accuracy and efficiency in credit risk evaluation.
method Hybrid Quantum-Classical Deep Neural Network for Row-Type Dependent Predictive Analysis.
result Proposed framework enhances predictive models for different loan categories.
Study improves data quality assessment for structural monitoring data.
problem Ensuring reliability of structural health monitoring data.
method Probabilistic data quality assessment using a conditional diffusion model.
result Significantly improves accuracy of data quality assessment.
Deep learning method improves risk assessment for small loan portfolios.
problem Measuring name concentration risk in small loan portfolios.
method Deep learning approach using Monte Carlo simulations with importance sampling.
result New method outperforms existing analytical methods for small portfolios.
New measures assess differences in causal graphs' separations.
problem Evaluating causal discovery algorithms' output.
method Proposes new distance measures capturing causal graphs' separations.
result Proposed distances assess differences in causal graphs' separations.
TransCORALNet uses transformer and CORAL for supply chain credit assessment with cold start.
problem Supply chain credit assessment for new borrowers with limited data.
method Two-stream transformer CORAL networks with domain adaptation and LIME.
result TransCORALNet outperforms state-of-the-art models in accuracy.
Fast risk assessment for autonomous vehicles using learned agent futures.
problem Risk assessment for autonomous vehicles given probabilistic predictions of other agents' futures.
method Non-sampling based methods using deep neural networks for probabilistic predictions, with Gaussian and non-Gaussian mixture models for agent positions and controls.
result Effective risk assessment for low probability events using learned models of agent futures.
A new measure identifies clusters without assuming data distribution.
problem Identifying the correct number of clusters in data without distribution assumptions.
method Nonparametric interpoint distance-based approach.
result Superior to existing clustering measures, validated on synthetic and real data.
In risk management, tail risks are of crucial importance. The assessment of risks should be carried out in accordance with the regulatory authority's requirement at high quantiles. In general, the underlying distribution function is unknown, the database is sparse, and therefore special tail models are used. Very often…
This paper develops a machine learning model to assess credit risk in UAE commercial banks.
problem Lack of precision in conventional credit rating tools for accurate credit risk prediction.
method Constructs a credit risk assessment model using Linear Discriminant Analysis.
result Demonstrates improved accuracy in predicting good and bad creditors compared to conventional methods.
This paper addresses clustering with missing data using Rubin's rules.
problem How to pool partitions and assess instability when data are incomplete after multiple imputation.
method Consensus clustering and bootstrap theory are used to address the problem.
result New rules for pooling partitions and assessing instability are proposed and validated.
Estimating and assessing the risk of a large portfolio is an important topic in financial econometrics and risk management. The risk is often estimated by a substitution of a good estimator of the volatility matrix. However, the accuracy of such a risk estimator for large portfolios is largely unknown, and a simple ine…
Study improves financial risk assessment using ARMA-APARCH-EVT models with HACs.
problem Improving risk assessment in financial portfolios.
method ARMA-APARCH-EVT-HAC model for volatility and extreme value forecasting.
result Empirical analysis shows the model's effectiveness in international stock market data.
Recall assistance methods are among the key aspects that improve the accuracy of online dietary assessment surveys. These methods still mainly rely on experience of trained interviewers with nutritional background, but data driven approaches could improve cost-efficiency and scalability of automated dietary assessment.…
Bayesian design improves accuracy without extra cost.
problem Nested inference in complex systems limits BED accuracy and efficiency.
method Grouped geometric pooled posterior with EKI formulation.
result Improved accuracy and stable estimators at comparable cost.
New metrics needed for streaming ML due to delayed labels.
problem Streaming ML evaluation fails to identify unexpected performance.
method Recommend additional metrics for streaming ML performance.
result New metrics are needed for streaming ML due to delayed labels.
Active sampling algorithm improves accuracy of inferred scores from pairwise comparisons.
problem Inference of accurate scores from time-consuming pairwise comparisons.
method Approximate message passing and expected information gain maximization.
result ASAP offers the highest accuracy of inferred scores compared to existing methods.
Enhances Transformers for better risk assessment in finance.
problem Transformer models lack sensitivity to extreme financial losses.
method Integrates Loss-at-Risk function with Value at Risk (VaR) and Conditional Value at Risk (CVaR).
result Improves risk prediction and management in financial datasets.
We propose a novel classifier accuracy metric: the Bayesian Area Under the Receiver Operating Characteristic Curve (CBAUC). The method estimates the area under the ROC curve and is related to the recently proposed Bayesian Error Estimator. The metric can assess the quality of a classifier using only the training datase…
Magnetic resonance (MR) imaging offers a wide variety of imaging techniques. A large amount of data is created per examination which needs to be checked for sufficient quality in order to derive a meaningful diagnosis. This is a manual process and therefore time- and cost-intensive. Any imaging artifacts originating fr…
Measuring the corporate default risk is broadly important in economics and finance. Quantitative methods have been developed to predictively assess future corporate default probabilities. However, as a more difficult yet crucial problem, evaluating the uncertainties associated with the default predictions remains littl…
TinyXRA assesses financial risks from 10-K reports using a lightweight transformer model.
problem Comprehensive risk assessment from financial reports, distinguishing between upside and downside risk.
method Lightweight transformer model with dynamic attention, incorporating skewness, kurtosis, and Sortino ratio.
result State-of-the-art predictive accuracy and transparent risk assessments.
Time series are ubiquitous, and a measure to assess their similarity is a core part of many computational systems. In particular, the similarity measure is the most essential ingredient of time series clustering and classification systems. Because of this importance, countless approaches to estimate time series similar…