Dual quality assessment method tackles adversarial robustness issues across various metrics.
problem Varying robustness levels and bias in adversarial attacks and defenses.
method Model agnostic dual quality assessment method, including robustness levels.
result Current networks and defenses are vulnerable at all robustness levels, highlighting the need for a dual approach.
Protein structure prediction has been a grand challenge problem in the structure biology over the last few decades. Protein quality assessment plays a very important role in protein structure prediction. In the paper, we propose a new protein quality assessment method which can predict both local and global quality of …
The paper assesses quality measures for machine learning models using cross-validation.
problem Evaluating the accuracy and robustness of quality measures for machine learning models.
method Cross-validation approach to estimate prediction error and quantify explained variation. Confidence bounds and local quality measures derived from residuals.
result The reliability and robustness of quality measures are assessed through numerical examples and confidence bounds.
Paper tightens optimization bounds using conformal prediction.
problem Lack of practical informiveness in dual bounds from optimization solvers.
method Introduces conformal prediction framework to tighten loose primal and dual bounds.
result Proposed method produces tighter, more informative prediction intervals.
Automatically assesses the quality of online health articles.
problem Lack of automated tools to evaluate the quality of online health information.
method Data mining approach using 10 quality criteria and feature selection.
result Classifier achieved 84%-90% accuracy on 10 criteria.
Magnetic resonance (MR) imaging offers a wide variety of imaging techniques. A large amount of data is created per examination which needs to be checked for sufficient quality in order to derive a meaningful diagnosis. This is a manual process and therefore time- and cost-intensive. Any imaging artifacts originating fr…
Study improves data quality assessment for structural monitoring data.
problem Ensuring reliability of structural health monitoring data.
method Probabilistic data quality assessment using a conditional diffusion model.
result Significantly improves accuracy of data quality assessment.
A framework assesses the quality of crowdsourced weather data.
problem Quality control and assessment of crowdsourced weather data from third-party stations.
method Proposes a simple, scalable, and interpretable AI/Stats/ML framework to assess TPAWS data.
result Demonstrates the performance of the framework using synthetic and real data.
Paper proposes a method to estimate intra-observer variability in echocardiography quality assessment.
problem Intra-observer variability in echocardiography quality assessment impacts deep neural network reliability.
method Modeling intra-observer variability as aleatoric uncertainty in a regression problem.
result The proposed method reduces error from 0.11 to 0.09, improving test accuracy by 5.7%.
Generative models assess quality on time-series data using ITS and FITD.
problem Lack of consensus for quality assessment of class-conditional generative models on time-series data.
method Introduced InceptionTime Score (ITS) and Frechet InceptionTime Distance (FITD) to evaluate generative models.
result ITS and FITD combined with TSTR can accurately assess generative model performance on time-series data.
The paper defines and assesses the quality of datasets using a novel expected diameter metric.
problem Lack of rigorous methods to assess data quality.
method Formal definition of data quality, expected diameter metric, Fourier analysis, algebraic methods, probabilistic analysis.
result The expected diameter metric provides theoretical guarantees and practical solutions for data quality assessment.
Framework for automatically assessing and correcting data quality issues without domain knowledge.
problem Ensuring data quality in datasets across various domains.
method Hybrid approach combining statistical and machine learning methods.
result Effective detection and correction of missing values, duplicates, and typographical errors.
The paper introduces a method to assess machine translation quality with confidence intervals.
problem Evaluating the uncertainty and quality of machine translation.
method Utilizes conformal predictive distributions to produce prediction intervals with guaranteed coverage.
result The method outperforms a baseline on six language pairs in terms of coverage and sharpness.
PQMass assesses generative model quality using chi-squared tests.
problem Assessing the quality of generative models without density assumptions.
method Divides sample space into regions, applies chi-squared tests to p-values.
result Effectively assesses generative model quality, novelty, and diversity.
A popular tool for unsupervised modelling and mining multi-aspect data is tensor decomposition. In an exploratory setting, where and no labels or ground truth are available how can we automatically decide how many components to extract? How can we assess the quality of our results, so that a domain expert can factor th…
Dual-stage sEMG classification improves gesture recognition accuracy.
problem Improving accuracy in hand gesture recognition from sEMG signals.
method Dual-stage classification approach: first stage groups similar activities, second stage classifies within groups.
result Dual-stage classification yields significantly higher accuracy than single-stage approach.
Re-speaking is a mechanism for obtaining high quality subtitles for use in live broadcast and other public events. Because it relies on humans performing the actual re-speaking, the task of estimating the quality of the results is non-trivial. Most organisations rely on humans to perform the actual quality assessment, …
This paper uses Monte Carlo simulation to value quality options in agricultural futures contracts.
problem Valuation of quality options in agricultural futures to prevent manipulation and improve hedging performance.
method Monte Carlo simulation with antithetic variables for efficiency.
result Demonstrates a method to estimate the value of quality options in agricultural futures contracts.
CALLISTO generates tests and assesses ML data quality using prediction entropy.
problem Validating ML systems for accuracy and data quality.
method Entropy-based test generation and data quality assessment framework.
result CALLISTO detects up to 20x more errors than traditional methods.
Paper proposes a method to protect DCNNs from adversarial attacks and poor quality images.
problem DCNNs are vulnerable to adversarial examples and perceptual quality issues.
method Combines image quality assessment with knowledge distillation.
result Detector effectively identifies poor quality and adversarial images.
Study subjective perception of low light restored images and develop an unsupervised QA model.
problem Lack of subjective QA for low light restored images and challenges in collecting human opinion scores.
method Create a dataset, conduct subjective QA study, develop self-supervised contrastive learning technique to extract features.
result Unsupervised NR QA model achieves state-of-the-art performance for low light restored images.
Computer-aided assessment of physical rehabilitation entails evaluation of patient performance in completing prescribed rehabilitation exercises, based on processing movement data captured with a sensory system. Despite the essential role of rehabilitation assessment toward improved patient outcomes and reduced healthc…
Enhances renewable energy forecasts and trading profits.
problem Improving accuracy and economic benefits of renewable energy forecasts and trading.
method Data-driven newsvendor model using features from multiple sources.
result Significant reduction in balancing costs and improvement in forecast accuracy.
C-VAE improves VAE by resolving prior issues and generating better samples.
problem Low-quality samples from VAE due to prior issues.
method Formulates VAE as OT, allows flexible priors, and uses OT formulations.
result C-VAE generates higher quality samples and latent representations.
We introduce a fully probabilistic framework of consumer product choice based on quality assessment. It allows us to capture many aspects of marketing such as partial information asymmetry, quality differentiation, and product placement in a supermarket.
The exploitation of large-scale population data has the potential to improve healthcare by discovering and understanding patterns and trends within this data. To enable high throughput analysis of cardiac imaging data automatically, a pipeline should comprise quality monitoring of the input images, segmentation of the …
New dual approach for hedging Bermudan options efficiently.
problem Computing efficient hedging portfolios for Bermudan options.
method Pure dual approach, rewriting dual pricing formula as excess reward representation, strict convexification, Monte Carlo method.
result Convergence and effectiveness of the new algorithm tested on various Bermudan options.
Study assesses 'big data' in materials science, highlighting challenges.
problem Understanding what constitutes 'big data' in materials science.
method Selected examples of machine learning models, data quality, and infrastructure requirements.
result Big data presents unique challenges in materials science.
AgraSSt assesses graph generators using Stein operators and kernel discrepancies.
problem Assessing the quality of graph generators that are implicit or not in explicit form.
method AgraSSt uses Stein operators and kernel discrepancies to assess graph generators, providing interpretable criticisms.
result Theoretical guarantees and empirical validation for various graph models.
In this paper, as a first step in examining the properties of a feasible portfolio subset that is characterized by budget and risk constraints, we assess the maximum and minimum of the investment concentration using replica analysis. To do this, we apply an analytical approach of statistical mechanics. We note that the…
A framework uses POMDPs to assess hierarchical clustering quality.
problem No agreed methodology for evaluating hierarchical clustering without ground-truth labels.
method Modelled as a POMDP, assessing search support for hierarchical structures.
result Proposes a novel measure for assessing hierarchical clustering quality.
Stabilizes training of DNN for speech enhancement using PESQ scores.
problem Stability issues in training DNNs using non-differentiable OSQA scores.
method Approximate OSQA scores with a differentiable auxiliary DNN and stabilize training with reinforcement learning techniques.
result Stable training of DNN to achieve state-of-the-art PESQ scores and better sound quality.
The paper explores dual learning, a technique that improves machine translation and image transformation.
problem Understanding and improving dual learning's effectiveness and conditions.
method Theoretical analysis and algorithmic extension of dual learning.
result Multi-step dual learning boosts performance under mild conditions.
Paper explores physics-informed deep learning for system reliability assessment.
problem Limited study on deep learning for system reliability assessment.
method Physics-informed deep learning approach for system reliability assessment.
result Physics-informed deep learning can alleviate computational challenges and combine measurement data and mathematical models.
FinSphere improves stock analysis quality with AI and expert-curated data.
problem Lack of objective evaluation metrics and depth in stock analysis by FinLLMs.
method Developed AnalyScore, curated Stocksis dataset, and FinSphere AI agent.
result FinSphere outperforms general and domain-specific LLMs in generating high-quality stock analysis reports.
Voice conversion (VC) aims at conversion of speaker characteristic without altering content. Due to training data limitations and modeling imperfections, it is difficult to achieve believable speaker mimicry without introducing processing artifacts; performance assessment of VC, therefore, usually involves both speaker…
Paper proposes a new method to assess synthetic data generators.
problem Assessing the quality of implicit generative models.
method Kernelised Stein Statistic (KSD) test based on non-parametric Stein operator.
result Improved power performance compared to existing approaches.
Study assesses impact of CBDC on financial stability in dual-currency economy.
problem Impact of CBDC on financial stability in dual-currency economy (Romania).
method Integrated analytical framework combining econometrics, machine learning, and behavioural modelling. CBDC adoption probabilities estimated using XGBoost and logistic regression models. Liquidity stress simulations and VAR, MSVAR, SVAR models capture macro-financial transmission.
result CBDC uptake would be moderate, primarily driven by digital readiness and trust in the central bank.
SONA improves conditional generation by balancing authenticity and alignment.
problem Challenges in balancing authenticity and conditional alignment in conditional generative models.
method SONA integrates unconditional discrimination, matching-aware supervision, and adaptive weighting to balance authenticity and alignment.
result SONA achieves superior sample quality and conditional alignment compared to state-of-the-art methods.
The bootstrap provides a simple and powerful means of assessing the quality of estimators. However, in settings involving large datasets, the computation of bootstrap-based quantities can be prohibitively demanding. As an alternative, we present the Bag of Little Bootstraps (BLB), a new procedure which incorporates fea…
Developers of text-to-speech synthesizers (TTS) often make use of human raters to assess the quality of synthesized speech. We demonstrate that we can model human raters' mean opinion scores (MOS) of synthesized speech using a deep recurrent neural network whose inputs consist solely of a raw waveform. Our best models …
Develops methods to improve demand counterfactuals from imperfect proxies.
problem Imperfect proxies in demand models lead to biased counterfactuals and invalid inference.
method Practical toolkit for market-level and individual data, requiring minimal computation.
result Improves substitution prediction and counterfactual performance.
A method uses CG to create efficient channels for ideal observers.
problem Computational intractability of ideal observers for high-dimensional image data.
method Conjugate gradient (CG) method for constructing efficient channels.
result CG-based channels approximate IO and HO performance efficiently.
Adopting a zonal structure of electricity market requires specification of zones' borders. In this paper we use social welfare as the measure to assess quality of various zonal divisions. The social welfare is calculated by Market Coupling algorithm. The analyzed divisions are found by the usage of extended Locational …
This paper argues against using calibration metrics for assessing posterior probabilities and proposes expected proper scoring rules instead.
problem The assessment of posterior probabilities generated by machine learning classifiers using calibration metrics is flawed and should be replaced with expected proper scoring rules.
method The paper reviews proper scoring rules from a practical perspective, explains why expected PSRs are a principled measure of posterior quality, and introduces a new calibration metric called calibration loss.
result Calibration loss is superior to expected calibration error and expected score divergence calibration metrics for assessing posterior probabilities.
Linear NDCG is used for measuring the performance of the Web content quality assessment in ECML/PKDD Discovery Challenge 2010. In this paper, we will prove that the DCG error equals a new pair-wise loss.
MEx dataset benchmarks HAR and multi-modal fusion for exercise quality.
problem Recognizing and evaluating exercise quality for Musculoskeletal Disorders patients.
method Multi-sensor, multi-modal dataset with four sensors (pressure mat, depth camera, accelerometers) for HAR and exercise quality assessment.
result Reference performance for each sensor identified, exposing their strengths and weaknesses.
Study identifies pitfalls in assessing hierarchies for multi-class classification.
problem Lack of understanding in selecting hierarchies for multi-class classification.
method Analyzed and compared popular approaches to extracting hierarchies.
result Hierarchy quality becomes irrelevant when using powerful classifiers.