The paper assesses quality measures for machine learning models using cross-validation.
problem Evaluating the accuracy and robustness of quality measures for machine learning models.
method Cross-validation approach to estimate prediction error and quantify explained variation. Confidence bounds and local quality measures derived from residuals.
result The reliability and robustness of quality measures are assessed through numerical examples and confidence bounds.
Protein structure prediction has been a grand challenge problem in the structure biology over the last few decades. Protein quality assessment plays a very important role in protein structure prediction. In the paper, we propose a new protein quality assessment method which can predict both local and global quality of …
The paper presents a method for generating well-calibrated prediction intervals using quality-driven deep ensembles.
problem Generating reliable prediction intervals for regression analysis.
method A multi-objective loss function combining quality measures for prediction intervals and point estimates, with a penalty function to ensure semantic integrity and stability.
result The method produces well-calibrated prediction intervals and point estimates, capturing both aleatoric and epistemic uncertainty.
Research develops a water quality prediction model using LSTM.
problem Global degradation of water resources and need for optimal water quality monitoring.
method Developed a multivariate water quality prediction model using LSTM and historical data.
result Multiple step LSTM model achieved RMSE of 0.227 mg/L.
The paper introduces a method to assess machine translation quality with confidence intervals.
problem Evaluating the uncertainty and quality of machine translation.
method Utilizes conformal predictive distributions to produce prediction intervals with guaranteed coverage.
result The method outperforms a baseline on six language pairs in terms of coverage and sharpness.
DCK improves air quality index prediction with probabilistic spatial models.
problem Non-Gaussian, complex spatial structure of air quality index.
method Deep classifier kriging (DCK) for non-Gaussian, nonlinear spatial prediction.
result DCK outperforms conventional methods in predictive accuracy and uncertainty quantification.
Low-cost sensors improve air quality prediction accuracy significantly.
problem Improving air quality monitoring networks with affordable sensors.
method Developed a high-resolution air quality prediction engine using low-cost sensors and official data.
result The use of low-cost sensors improves prediction accuracy by 25% and 15% for PM2.5 and PM10 respectively in densely monitored areas.
Paper uses SSL models' uncertainty to predict audio quality efficiently.
problem Efficiently predicting audio quality in low-resource settings.
method Leverages self-supervised learning models' uncertainty measures.
result Uncertainty measures correlate with MOS scores in SSL models.
Engine predicts real-time air quality with high resolution.
problem Real-time prediction of air pollutants for health monitoring.
method Combines official data, models, land cover, traffic data for high-resolution predictions.
result Engine produces predictions with resolution of a few dozen meters.
The UN Sustainable Development Goals allude to the importance of infrastructure quality in three of its seventeen goals. However, monitoring infrastructure quality in developing regions remains prohibitively expensive and impedes efforts to measure progress toward these goals. To this end, we investigate the use of wid…
CALLISTO generates tests and assesses ML data quality using prediction entropy.
problem Validating ML systems for accuracy and data quality.
method Entropy-based test generation and data quality assessment framework.
result CALLISTO detects up to 20x more errors than traditional methods.
FAB-PPI uses prior knowledge to improve prediction-powered inference.
problem Improving statistical inference with machine learning predictions.
method Informing PPI with prior knowledge on prediction quality.
result FAB-PPI improves inference accuracy and confidence intervals.
Research predicts rice prices in Banda Aceh post-COVID using ARIMA models.
problem Forecasting rice prices in Banda Aceh post-COVID-19.
method Used LOCF imputation for missing data and auto-ARIMA for forecasting.
result ARIMA model (0,0,5) best for all rice qualities, showing price decline and then stability.
Generative models predict page quality without training, useful for low-resource settings.
problem Detecting low-quality content in web articles.
method Human evaluation and analysis of 500 million web articles.
result Generative models can predict page quality without training, useful for low-resource settings.
MRI image quality affects statistical and predictive analysis of brain morphology.
problem Impact of MRI image quality on statistical and predictive analysis of brain morphology.
method Systematic testing of image quality on univariate statistics and machine learning classification using three large datasets.
result Low-quality MRI data significantly affects detecting significant sex/gender differences in smaller samples, but not in larger ones.
LightGBM outperforms other models in predicting pH values in Georgia, USA.
problem Accurate water quality prediction for effective resource management and pollution mitigation.
method Five distinct predictive models (linear regression, Random Forest, XGBoost, LightGBM, MLP neural network) were assessed for pH value forecasting in Georgia, USA.
result LightGBM achieved the highest average precision in predicting pH values.
Online audio advertising is a particular form of advertising used abundantly in online music streaming services. In these platforms, which tend to host tens of thousands of unique audio advertisements (ads), providing high quality ads ensures a better user experience and results in longer user engagement. Therefore, th…
The paper proposes a method to predict the performance of data-driven algorithms using surrogate models.
problem Improving the performance prediction of data-driven knowledge discovery algorithms.
method Surrogate-assisted performance prediction using evolutionary modeling of clinical pathways.
result The proposed approach provides interpretable prediction of algorithm performance and quality.
The economic and social impact of poor air quality in towns and cities is increasingly being recognised, together with the need for effective ways of creating awareness of real-time air quality levels and their impact on human health. With local authority maintained monitoring stations being geographically sparse and t…
This work improves explanation quality for time series predictions by learning perturbations.
problem Explaining predictions on multivariate time series data with time dependencies.
method Learning both masks and associated perturbations to explain predictions.
result Learning perturbations significantly improves explanation quality on time series data.
A new model combines data-driven and model-based methods for accurate air quality prediction.
problem Accurate air quality forecasts are crucial for public health.
method Combines model-based strategy and data-driven method using PTC model.
result The PTC model achieves excellent performance compared to baseline models.
Study predicts coastal water quality using machine learning, identifying salinity as key factor.
problem Predicting and managing coastal water quality for public health and tourism.
method Machine learning models (Catboost, Xgboost, Random Forests, Support Vector Regression, Artificial Neural Networks) trained on environmental data.
result Catboost algorithm performed best, with R² values of 0.71 and 0.68 for E. Coli and enterococci predictions.
Growing amounts of online user data motivate the need for automated processing techniques. In case of user ratings, one interesting option is to use neural networks for learning to predict ratings given an item and a user. While training for prediction, such an approach at the same time learns to map each user to a vec…
MMformer improves forecasting of environmental time series data.
problem Accurately forecasting environmental change trends for policy-making.
method Meta-learning MTS model combining self-attention and adaptive transferable multi-head attention.
result MMformer outperforms other models in air quality and climate datasets, reducing prediction errors by 50% in MSE and 20% in MAE.
FPPI selectively uses predictions to improve inference efficiency.
problem Improving statistical inference with limited labeled data and heterogeneous prediction quality.
method Filtered Prediction-Powered Inference (FPPI) framework.
result FPPI achieves strictly improved asymptotic efficiency compared to existing methods.
In chemistry, deep neural network models have been increasingly utilized in a variety of applications such as molecular property predictions, novel molecule designs, and planning chemical reactions. Despite the rapid increase in the use of state-of-the-art models and algorithms, deep neural network models often produce…
BeMF improves recommendation reliability in recommender systems.
problem Improving reliability in recommender systems beyond accuracy.
method Bernoulli Matrix Factorization (BeMF) for model-based collaborative filtering.
result BeMF selects more reliable predictions, improving recommendation quality.
Geometrical and appearance quality requirements set the limits of the current industrial performance in injection molding. To guarantee the product's quality, it is necessary to adjust the process settings in a closed loop. Those adjustments cannot rely on the final quality because a part takes days to be geometrically…
This paper presents a method to automatically generate high-quality prediction intervals for neural networks.
problem Accurate uncertainty quantification for deep learning models in real-world applications.
method Dual neural network approach with a novel loss function to balance prediction interval width and coverage.
result Our method produces significantly narrower prediction intervals with higher probability coverage compared to state-of-the-art methods.
By seeking the narrowest prediction intervals (PIs) that satisfy the specified coverage probability requirements, the recently proposed quality-based PI learning principle can extract high-quality PIs that better summarize the predictive certainty in regression tasks, and has been widely applied to solve many practical…
The PARAFAC tensor decomposition has enjoyed an increasing success in exploratory multi-aspect data mining scenarios. A major challenge remains the estimation of the number of latent factors (i.e., the rank) of the decomposition, which yields high-quality, interpretable results. Previously, we have proposed an automate…
An automated metric to evaluate dialogue quality is vital for optimizing data driven dialogue management. The common approach of relying on explicit user feedback during a conversation is intrusive and sparse. Current models to estimate user satisfaction use limited feature sets and employ annotation schemes with limit…
SupRB learns rules for continuous decision problems from examples.
problem Learning from continuous choices and explaining decisions to operators.
method SupRB is a supervised rule-based learning system for multi-dimensional continuous problems.
result SupRB provides human-understandable rules for optimal choices and quality predictions.
The Neural Testbed evaluates joint predictions of neural agents, revealing their limitations.
problem Evaluating the quality of joint predictions generated by neural agents.
method Developed an open-source benchmark (The Neural Testbed) to assess agents' marginal and joint predictions.
result Popular Bayesian deep learning agents perform poorly on joint predictions, even with accurate marginal predictions.
Enhances statistical inference using synthetic data.
problem Limited labeled data for statistical inference.
method GESPI framework that combines synthetic and real data.
result Error rate remains below a user-specified bound and decreases with synthetic data quality.
Enhances early-exit neural networks for anytime classification.
problem Lack of guaranteed prediction quality improvement with longer computation time.
method Post-hoc modification based on Product-of-Experts to enforce conditional monotonicity.
result Achieves conditional monotonicity in prediction quality, enabling anytime classification.
The paper introduces recklessness to improve recommendation quality and quantity.
problem The reliability/coverage dilemma in recommender systems limits the number of recommended items.
method Incorporates a new term (recklessness) into matrix factorization-based recommender systems to address the dilemma.
result Recklessness improves the quantity and quality of recommendations by allowing for risk regulation.
The study forecasts water quality from satellite data using machine learning.
problem Predicting future water quality from satellite data for coastal regions.
method Decomposed time series into components and used machine learning models (SARIMA, regression, neural network).
result Regression and neural network models are best at predicting Chl-a, SARIMA model best at FLH and SST.
Study evaluates margin parameter effects on knowledge embedding quality.
problem Understanding margin parameter's impact on embedding quality.
method Examined margin parameter values for multi-relational categorized data.
result Lower margin values are insufficient, while larger values cause noise.
A key aspect of automating predictive machine learning entails the capability of properly triggering the update of the trained model. To this aim, suitable automatic solutions to self-assess the prediction quality and the data distribution drift between the original training set and the new data have to be devised. In …
Study finds visual explanations do not significantly improve human accuracy or trust in model predictions.
problem Measuring the impact of visual explanations on human accuracy and trust in model predictions.
method Randomized controlled trial with image-based age prediction task, varying levels of explanation quality.
result Visual explanations do not significantly alter human accuracy or trust in the model.
Generative model improves time series prediction quality.
problem Lack of effective generative models for time series data.
method Proposed XIRP for time series, trained with WGAN-GP.
result XIRP significantly outperforms RNN-based models in predictive ability.
This study uses machine learning to predict sovereign credit ratings and identifies key factors.
problem Predicting sovereign credit ratings and identifying important factors.
method Used Multilayer Perceptron (MLP), Classification and Regression Trees (CART), Support Vector Machines (SVM), Naïve Bayes (NB), and Ordered Logit (OL) models.
result MLP is the best model for predicting sovereign credit ratings with a 68% accuracy.
Machine learning algorithms have been effectively applied into various real world tasks. However, it is difficult to provide high-quality machine learning solutions to accommodate an unknown distribution of input datasets; this difficulty is called the uncertainty prediction problems. In this paper, a margin-based Pare…
New algorithm reduces regret and constraint violation in online convex optimization with predictions.
problem Online convex optimization with time-varying constraints and predictions.
method Primal-dual algorithm combining Follow-The-Regularized-Leader with adaptive steps.
result Achieves O(T43−β) regret and O(T21+β) constraint violation bounds. The study analyzes how probabilistic forecasts improve battery trading strategies in electricity markets.
problem Improvements in statistical forecast quality do not directly translate to economic value in battery trading strategies.
method The study frames battery optimization as a stochastic program based on fully probabilistic forecasts and examines decision quality under different uncertainty models.
result The study identifies two critical flaws in quantile-based trading strategies and provides theoretical justification and empirical evidence.
A new approach to newsvendor problems reduces loss by up to 40%.
problem Model misspecification in feature-based newsvendor problems.
method Conformal prediction framework combining selective training and calibration phases.
result Statistical guarantees and improved confidence intervals for critical quantiles.
Engine forecasts NO2, O3, PM2.5, PM10 with high accuracy.
problem Accurate long-term air quality forecasting.
method Convolutional LSTM network trained on grid data.
result 4-day forecasts significantly outperform simple benchmarks.