Rapidly estimating the remaining wall thickness (RWT) is paramount for the non-destructive condition assessment evaluation of large critical metallic pipelines. A robotic vehicle with embedded magnetism-based sensors has been developed to traverse the inside of a pipeline and conduct inspections at the location of a br…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “non-destructive evaluation”
Study examines human factors in radiographic testing to improve inspection performance.
problem Insufficient consideration of human and organizational factors in NDT.
method CREAM method applied to analyze and model HOF on radiogram interpretation tasks.
result Model CREAM well-adapted for estimating HOF impact on NDT performances.
Paper introduces ML tools for guided wave behaviour in composite materials.
problem Difficult assessment of guided wave behaviour in complex materials.
method Data-driven model using Gaussian processes with physical constraints.
result Structured machine learning models offer advantages like extrapolation and physical interpretation.
Meta-models predict model hyperparameters for NDT experiments.
problem Non-destructive testing experiments are isolated; this work connects them.
method Bayesian multilevel approach, capturing inter-task relationships.
result Meta-models encode knowledge between and within tasks for transfer learning.
In this paper, we study the problem of locating a predefined sequence of patterns in a time series. In particular, the studied scenario assumes a theoretical model is available that contains the expected locations of the patterns. This problem is found in several contexts, and it is commonly solved by first synthesizin…
SVM methods improve tack coat classification in French pavements.
problem Classifying thin tack coat layers in GPR data.
method Inverse Machine Learning approach applied to SVM/SVR methods.
result SVM/SVR methods accurately classify tack coat emulsion proportions.
Understanding the adaptation process of plants to drought stress is essential in improving management practices, breeding strategies as well as engineering viable crops for a sustainable agriculture in the coming decades. Hyper-spectral imaging provides a particularly promising approach to gain such understanding since…
Artificial Intelligence (AI) systems sometimes make errors and will make errors in the future, from time to time. These errors are usually unexpected, and can lead to dramatic consequences. Intensive development of AI and its practical applications makes the problem of errors more important. Total re-engineering of the…
Low-cost sensor fusion for organic substance classification.
problem Non-destructive classification and quality control of organic substances.
method Arduino Mega 2560 microcontroller with three sensors, supervised learning models, hyperparameter tuning, PCA/LDA, ensemble voting classifier.
result Test accuracies of 93-94% for identifying organic compounds.
Multi-output Gaussian processes for inverse uncertainty quantification in neutron noise analysisstat.CO
The paper tackles inverse uncertainty quantification in neutron noise analysis.
problem Uncertainty in estimating material properties from noisy neutron correlation measurements.
method Surrogate models and inverse uncertainty quantification to account for measurement error and model bias.
result Improved prediction of neutron correlations and quantification of uncertainties.
Improved prediction of soil properties with Multi-target Stacked Generalisation on EDXRF spectrastat.ML
Improved prediction of soil parameters using Multi-target Stacked Generalisation on EDXRF spectra.
problem Challenges in predicting multiple soil parameters accurately from EDXRF spectra.
method Multi-target Stacked Generalisation (MTSG) method combining multiple regression models.
result MTSG significantly improved prediction accuracy for multiple soil parameters, reducing average error from 0.67 to 0.64.
A computational theory reduces agent evaluation errors and speeds up processes.
problem Efficient evaluation of mini agents at reduced cost.
method Developed a computational theory and a meta-learner to handle heterogeneous agents.
result Reduced evaluation errors by 24.1% to 99.0% across various scenarios.
Develops a method to evaluate OPE robustness to hyperparameters and policies.
problem Difficulty in selecting and tuning OPE estimators due to limited experimental evaluations.
method Introduces IEOE (Interpretable Evaluation for Offline Evaluation) to assess robustness.
result Demonstrates improved evaluation of OPE estimators' reliability.
This paper evaluates and validates cluster results using external and internal evaluation methods.
problem Evaluating and validating the quality of clustering results.
method External evaluation using Homogeneity, Correctness, and V-measure scores; internal evaluation using Silhouette Index and Sum of Square Errors.
result Validation of the number of clusters using dendrogram and statistical frequency distribution.
Optimizes crowdsourced preference-based subjective evaluation with online learning.
problem Large-scale evaluation of generative media using crowdsourcing due to combinatorial explosion.
method Automatic optimization of pair combination selections and evaluation volumes with online learning.
result Optimizes evaluation by reducing pair combinations and allocating optimal evaluation volumes.
We conduct a large-scale, systematic study to evaluate the existing evaluation methods for natural language generation in the context of generating online product reviews. We compare human-based evaluators with a variety of automated evaluation procedures, including discriminative evaluators that measure how well machi…
Our research extends the Bilingual Evaluation Understudy (BLEU) evaluation technique for statistical machine translation to make it more adjustable and robust. We intend to adapt it to resemble human evaluation more. We perform experiments to evaluate the performance of our technique against the primary existing evalua…
Study finds AUC is most consistent across different prevalence in binary classification.
problem Consistency of model evaluation metrics across varying prevalence in binary classification.
method Analysis of 156 data scenarios with 18 metrics, 5 models, and a random guess model.
result AUC has the smallest variance in evaluating individual models and ranking of models.
Cramming method evaluates learned policies from contextual bandits efficiently.
problem Evaluating final learned policies from contextual bandit algorithms.
method On-policy evaluation using a single pass of data, ensuring consistency and asymptotic normality.
result Cramming method reduces evaluation standard error by approximately 40% compared to off-policy methods.
Cer-Eval saves LLM evaluation costs while maintaining accuracy.
problem Challenges in evaluating large language models due to large dataset requirements.
method Adapts to different evaluation objectives, uses test sample complexity, and develops a partition-based algorithm.
result Cer-Eval can save 20-40% test points with comparable accuracy and 95% confidence guarantee.
The study addresses biases in evaluating molecular optimization methods and proposes methods to reduce these biases.
problem Biases in in silico evaluation of molecular optimization methods.
method Discussion and empirical investigation of bias reduction methods for predictor misspecification and sample reuse.
result Empirical investigation of bias reduction methods for predictor misspecification and sample reuse.
Paper addresses off-policy evaluation and learning with covariate shift.
problem Evaluating and training a new policy using historical data with a covariate shift.
method Derives efficiency bounds and proposes doubly robust estimators for OPE and OPL under covariate shift.
result Proposes estimators for off-policy evaluation and learning under covariate shift.
Proposes clustering as a new evaluation method for clinical knowledge embedding.
problem Traditional Link Prediction evaluation protocol loses information and harms model accuracy.
method Proposes Clustering Evaluation Protocol as an alternative.
result Experimental results show the proposed protocol can potentially replace Link Prediction.
AXE evaluates explanations to avoid misleading Rashomon set model selection.
problem Evaluating explanations for Rashomon set models to avoid false selection.
method Proposed AXE method to evaluate explanation quality.
result AXE detects adversarial fairwashing with 100% success rate.
We consider evaluation methods for payoffs with an inherent financial risk as encountered for instance for portfolios held by pension funds and insurance companies. Pricing such payoffs in a way consistent to market prices typically involves combining actuarial techniques with methods from mathematical finance. We prop…
New method for evaluating LLMs reduces bias in open-ended evaluations.
problem Bias in Elo-based ratings of LLMs due to data redundancies.
method Proposes evaluation as a 3-player game and introduces novel solution concepts.
result Novel method leads to more robust and intuitive ratings.
Progress in machine learning is measured by careful evaluation on problems of outstanding common interest. However, the proliferation of benchmark suites and environments, adversarial attacks, and other complications has diluted the basic evaluation model by overwhelming researchers with choices. Deliberate or accident…
New method improves consistency of reinforcement learning performance evaluations.
problem Inconsistent performance results in reinforcement learning due to flawed evaluation metrics.
method Proposes a new comprehensive evaluation methodology for reinforcement learning algorithms.
result Demonstrates improved reliability of performance measurements for reinforcement learning algorithms.
This paper considers the challenge of evaluating a set of classifiers, as done in shared task evaluations like the KDD Cup or NIST TREC, without expert labels. While expert labels provide the traditional cornerstone for evaluating statistical learners, limited or expensive access to experts represents a practical bottl…
Bayesian approach quantifies uncertainty in LLM evaluations.
problem Statistical uncertainty in evaluating LLM behavior.
method Bayesian evaluation of LLM behavior using probabilistic text generation strategies.
result Bayesian approach provides useful uncertainty quantification about LLM behavior.
Combines curvature descriptors with TDA for graph model evaluation.
problem Evaluating graph generative models efficiently and accurately.
method Combines graph curvature descriptors with topological data analysis.
result Robust, expressive descriptors for graph generative models.
We present and prove properties of a new offline policy evaluator for an exploration learning setting which is superior to previous evaluators. In particular, it simultaneously and correctly incorporates techniques from importance weighting, doubly robust evaluation, and nonstationary policy evaluation approaches. In a…
Paper tackles efficient evaluation of natural stochastic policies in offline RL.
problem Efficiency issues in evaluating natural stochastic policies due to unknown evaluation policy.
method Derive efficiency bounds for tilting and modified treatment policies, propose nonparametric estimators.
result Proposed estimators attain efficiency bounds under lax conditions and enjoy partial double robustness.
CONE evaluates treatment assignment functions using networked observational data to mitigate hidden confounding bias.
problem Evaluate treatment assignment functions using networked observational data with hidden confounders.
method CONE framework that learns partial representations of latent confounders and combines them for counterfactual evaluation.
result Network information mitigates hidden confounding bias in counterfactual evaluation.
MLModelScope streamlines ML/DL model evaluation and benchmarking.
problem Challenges in evaluating and benchmarking ML/DL models.
method Open-source, framework/hardware agnostic platform with distributed design.
result Demonstrates the impact of model evaluation pipelines and HW/SW choices.
Study evaluates saliency maps on artificial data with different backgrounds.
problem Objective evaluation of saliency methods on artificial data with varying backgrounds.
method Developed a framework to generate artificial data with synthetic lesions and a known ground truth map, evaluated two data sets with different backgrounds (Perlin noise and 2D brain MRI slices).
result Heatmaps vary strongly between saliency methods and backgrounds.
Tiny benchmarks reduce LLM evaluation costs by using fewer examples.
problem Expensive evaluation of LLMs with tens of thousands of examples.
method Developed evaluation tools and tiny versions of popular benchmarks.
result Accurately estimate LLM performance with just 100 curated examples.
A new method for faster multi-objective optimization by evaluating objectives separately.
problem Finding the Pareto front of trade-offs between multiple objectives efficiently.
method Knowledge Gradient with decoupled evaluations, accounting for different costs.
result The method significantly outperforms existing approaches in terms of evaluation cost.
The Vulnerability of Language Model Benchmarks: Do They Accurately Reflect True LLM Performance?cs.CL
Language model benchmarks often misrepresent true understanding, revealing vulnerabilities in evaluation methods.
problem Language model benchmarks fail to accurately reflect true language understanding and adaptability.
method Systematic analysis of NLP evaluation frameworks, identifying vulnerabilities in static benchmarks, human evaluation protocols, and LLM-as-judge frameworks.
result Current evaluation methods are unreliable and need improvement to accurately assess LLM performance.
New framework assesses graph-learning datasets for better evaluation.
problem Insufficient evaluation of graph-learning datasets and methods.
method Introduces Rings framework for dataset ablations and proposes performance separability and mode complementarity measures.
result Demonstrates utility of Rings framework for graph-learning dataset evaluation.
This work shows how evaluation metrics can be seen as fair gambles.
problem The relationship and evaluation of machine learning forecasts.
method Using game-theoretic probability, the authors show evaluation metrics as fair gambles.
result Standard evaluation metrics are fair gambler outcomes, with calibration and regret metrics on two dimensions.
SureMap estimates model performance across subpopulations efficiently.
problem Estimating model performance across subpopulations with scarce data.
method Simultaneous Gaussian mean estimation with external data.
result High accuracy in both multi-task and single-task disaggregated evaluations.
Murakami-Ohtsuki-Yamada introduced an evaluation of certain oriented planar trivalent graphs with colored edges. This evaluation plays a key role in the evaluation of the colored HOMFLY polynomial of a link in 3-space and its Khovanov-Rozansky categorification. Our goal is is to give a generating series formula for the…
Study evaluates new models using human feedback from another model.
problem Evaluate a new model using human feedback collected for another model.
method Formalize problem, propose model-based and model-free estimators, analyze unbiasedness, and empirically evaluate.
result Proposed estimators can predict absolute values, rank, and optimize evaluated policies.
Corrects bias in LLM-as-a-judge evaluations using adaptive calibration.
problem Bias in LLM evaluations due to imperfect sensitivity and specificity.
method Plug-in framework with confidence intervals accounting for test and calibration dataset uncertainties.
result LML-based evaluation yields more reliable estimates than human-only evaluation.
This work studies the problem of batch off-policy evaluation for Reinforcement Learning in partially observable environments. Off-policy evaluation under partial observability is inherently prone to bias, with risk of arbitrarily large errors. We define the problem of off-policy evaluation for Partially Observable Mark…
New method extends fitted Q-evaluation for distributional off-policy reinforcement learning.
problem Estimating return distribution in reinforcement learning using offline data.
method Developed a set of guiding principles and new FDE methods with theoretical justification.
result FDE methods outperform existing approaches in simulations and real-world games.
Geospatial ML models need special evaluation methods due to their unique challenges.
problem Evaluating geospatial machine learning models is challenging due to their specific characteristics.
method Delineated unique challenges and proposed concrete takeaways for improving geospatial model evaluations.
result Concrete takeaways for improving evaluations of geospatial model performance.