Develops methods for AI self-assessment to improve trustworthiness.
problem Uncertainty in AI predictions and lack of trust in AI systems.
method Uncertainty estimation techniques considering practical impacts and costs.
result Guidelines for selecting and designing effective AI self-assessment methods.
EduQG generates better educational questions by pre-training on scientific text.
problem Improving the quality of educational questions for scalable self-assessment.
method Adapting a large language model for educational question generation, pre-trained on scientific text.
result EduQG produces superior educational questions compared to baseline models.
This paper investigates uncertainty calibration in multimodal large language models.
problem Challenges in properly calibrating uncertainty in multimodal large language models.
method Investigation of representative MLLMs across various scenarios, including visual fine-tuning and multimodal training.
result MLLMs tend to give answers rather than admit uncertainty, but this self-assessment improves with proper prompt adjustments.
Estimates classifier errors without ground truth using algebraic geometry.
problem Lack of ground truth in real-world production systems.
method Non-parametric estimation using algebraic geometry to solve the self-assessment problem.
result Accuracy estimators are better than one part in a hundred.
A key aspect of automating predictive machine learning entails the capability of properly triggering the update of the trained model. To this aim, suitable automatic solutions to self-assess the prediction quality and the data distribution drift between the original training set and the new data have to be devised. In …
The study infers risk preferences from portfolio choices and measures portfolio efficiency.
problem Measuring the efficiency of household investment portfolios based on risk preferences.
method Statistical analysis of portfolio choices and demographic information over six years.
result Implied risk aversion increases with wealth and financial literacy, impacting portfolio efficiency.
Study shows GPT's earnings forecasts are human-like but not always accurate.
problem Information friction in AI-generated financial analysis.
method Examined GPT's earnings forecasts following corporate earnings releases and proposed a diagnostic framework.
result GPT's narrative attention is consistent and human-like but not always associated with higher forecast accuracy.
Experiment shows author rankings can improve peer review scores.
problem Improving accuracy in machine learning conference peer review.
method Used Isotonic Mechanism to calibrate review scores using author rankings.
result Calibrated scores outperform raw scores in estimating ground truth review scores.
Automated prediction of valence, one key feature of a person's emotional state, from individuals' personal narratives may provide crucial information for mental healthcare (e.g. early diagnosis of mental diseases, supervision of disease course, etc.). In the Interspeech 2018 ComParE Self-Assessed Affect challenge, the …
Compared to in-clinic balance training, in-home training is not as effective. This is, in part, due to the lack of feedback from physical therapists (PTs). Here, we analyze the feasibility of using trunk sway data and machine learning (ML) techniques to automatically evaluate balance, providing accurate assessments out…
Peer-induced fairness framework audits algorithmic fairness in AI applications.
problem Current auditing methods lack robustness and fail to distinguish between algorithmic discrimination and subject limitations.
method Combines counterfactual fairness and peer comparison strategy for a reliable auditing tool.
result Demonstrates significant unfairness in micro-firms compared to non-micro firms, highlighting the framework's potential.
Study evaluates machine learning methods for uncertainty quantification in complex systems.
problem Accurately quantify epistemic and aleatoric uncertainties in complex dynamical systems.
method Examined Gaussian processes, UQ-augmented neural networks (ENN, BNN, D-NN, G-NN) on two model data sets.
result Concluded on model architecture and hyperparameter tuning for improved UQ accuracy.
New method calibrates LLMs for safety-critical tasks with scalable Bayesian inference.
problem Overconfidence in LLMs after fine-tuning for specific tasks.
method Orthogonalized Low-Rank Adapters (PoLAR) with variational Bayesian inference.
result Scalable and well-calibrated uncertainty estimation for LLMs.