A novel correction algorithm is proposed for multi-class classification problems with corrupted training data. The algorithm is non-intrusive, in the sense that it post-processes a trained classification model by adding a correction procedure to the model prediction. The correction procedure can be coupled with any app…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “data correction”
New method combines machine learning with data assimilation for model error correction.
problem Correcting model errors using sparse and noisy observations.
method Hybrid machine learning and data assimilation methods.
result Tendency correction outperforms resolvent correction in data assimilation experiments.
Deep learning improves low-fidelity dynamical models with scarce high-fidelity data.
problem Improving low-fidelity models with limited high-fidelity data.
method Transfer learning using a deep neural network to correct a low-fidelity model.
result An improved DNN model with high accuracy to underlying dynamics.
This paper corrects climate model biases using a factor model approach.
problem Systematic biases in GCM outputs due to unobserved confounders.
method Factor model approach to learn latent confounders from historical data and apply them to enhance bias correction.
result Significant improvements in the accuracy of precipitation outputs.
DisCor corrects reinforcement learning issues by re-weighting collected data.
problem Reinforcement learning algorithms struggle with instability and sensitivity to hyperparameters.
method DisCor reweights collected data to mitigate issues caused by the distribution of experience.
result DisCor improves reinforcement learning in challenging settings like multi-task learning and noisy reward signals.
Sig-PCA integrates model outputs and observations to correct model biases.
problem Improving model accuracy and reliability by correcting biases and numerical approximations.
method Sig-PCA framework that combines summary statistics from model outputs with localized observations via a neural network.
result Corrects model outputs to align closely with observational data, preserving essential statistical information.
System predicts ice formation to improve road safety.
problem Improving road safety through accurate ice formation prediction.
method Combining physical models and machine learning for data correction.
result Machine learning effectively corrects incorrect data in forecasts.
Paper stabilizes generative model training with synthetic data.
problem Self-consuming loops in generative model training.
method Introducing an idealized correction function and self-correction functions.
result Self-consuming loops can be exponentially more stable with the right correction.
In Stochastic blockmodels, which are among the most prominent statistical models for cluster analysis of complex networks, clusters are defined as groups of nodes with statistically similar link probabilities within and between groups. A recent extension by Karrer and Newman incorporates a node degree correction to mod…
A neural framework corrects bias in estimating individual treatment effects.
problem Estimating individual treatment effects from observational data.
method An anchored neural architecture and precision-corrected intersection-bound inference.
result Corrected bias and maintained nominal coverage in high-dimensional settings.
Nyquist ghost artifacts in EPI are originated from phase mismatch between the even and odd echoes. However, conventional correction methods using reference scans often produce erroneous results especially in high-field MRI due to the non-linear and time-varying local magnetic field changes. Recently, it was shown that …
WeMix improves data augmentation by correcting bias in deep learning.
problem Data augmentation's effectiveness is limited by data bias.
method Developed AugDrop and MixLoss algorithms to correct data bias.
result WeMix improves data augmentation performance through bias correction.
Improved variational inference for geophysical inverse problems with data correction.
problem High computational cost and accuracy issues in Bayesian inference for geophysical inverse problems.
method Amortized variational inference with latent distribution correction using physics-based priors.
result Improved robustness of amortized variational inference under data distribution shifts.
The study examines methods to correct measurement error in nutritional epidemiology studies.
problem Measurement error in nutritional studies leads to biased and underconfident estimates.
method The article reviews various bias-correction models for exposure variables in nutritional epidemiology.
result Bias-correction methods are essential for accurate inference in nutritional studies.
Correctness of Backtest Enginesq-fin.TR
In recent years several trading platforms appeared which provide a backtest engine to calculate historic performance of self designed trading strategies on underlying candle data. The construction of a correct working backtest engine is, however, a subtle task as shown by Maier-Paape and Platen (cf. arXiv:1412.5558 [q-…
New methods correct spectral distortions using known analyte concentrations.
problem Distorted spectral shapes from absorbing and scattering contributions.
method Modified penalized baseline correction methods that incorporate known analyte concentrations.
result Improved prediction performance on near infra-red data sets.
A method for inferring ground-truth signals from degraded sensor data.
problem Inferring ground-truth signals from multiple degraded sensor signals.
method Iterative correction of degraded signals using a Bayesian multi-sensor data fusion method.
result The method effectively infers ground-truth signals from noisy and degraded sensor data.
Bias correction needed after deep learning regression training.
problem Systematic error accumulation in deep learning regression models.
method Adjust bias of the machine learning model post-training.
result Bias correction efficiently solves error accumulation.
The paper corrects bias in synthetic data for imbalanced learning.
problem Challenges in balancing false positive and negative rates in imbalanced data.
method Proposes a bias correction procedure to generate synthetic data for minority groups.
result Enhances prediction accuracy while avoiding overfitting.
Noise-corrected Langevin algorithm improves sampling from noisy data.
problem Sampling from noisy data with biased score function.
method Noise-corrected Langevin algorithm using noisy score function.
result Bias due to noisy data is removed, improving sampling accuracy.
Paper corrects bias in online learning algorithms with endogenous data.
problem Dynamic selection problems in online learning algorithms with endogenous data.
method Instrumental-variable-based algorithm to correct bias, proving central limit theorem.
result Obtains true parameter values and low regret levels.
Spectrally-Corrected and Regularized Linear Discriminant Analysis for Spiked Covariance Modelstat.ML
Improved LDA method for better classification and dimensionality reduction.
problem Improving linear discriminant analysis for better classification performance.
method Integrates spectrally-corrected covariance matrix and regularized discriminant analysis.
result SRLDA has a linear classification global optimal solution under spiked model assumption.
New methods reduce bias in machine learning predictions for causal inference without extra data.
problem Machine learning predictions from satellite data shrink toward the mean, leading to biased causal estimates.
method Two post-hoc correction methods: Linear Calibration Correction (LCC) and Tweedie's approach, reduce shrinkage-induced bias.
result Tweedie's method yields nearly unbiased treatment-effect estimates, enabling multiple trials with a single map.
Visual analytics tool detects and corrects concept drift in data streams.
problem Concept drift causes inaccurate predictions in evolving data.
method DriftVis combines drift detection and visualization.
result Visual analytics supports detection, examination, and correction of concept drift.
BC-ACI corrects time series forecast bias, improving prediction intervals.
problem Persistent bias in time series forecasts leads to overly conservative prediction intervals.
method Augments ACI with an EWM estimate of forecast bias to correct nonconformity scores and re-center intervals.
result Reduces Winkler interval scores by 13-17% under distribution shifts, improving calibration.
New method corrects missing data bias in dimension reduction.
problem Missing data complicates high-dimensional data analysis.
method Developed a bias-corrected Gram matrix for heterogeneous missingness.
result Proposed method improves dimension reduction techniques significantly.
The paper corrects biases in estimating intrinsic dimension and differential entropy.
problem Systematic bias in estimating intrinsic dimension and differential entropy.
method A bias-corrected estimator for both measures is proposed, highlighting shared steps and useful consequences.
result Simultaneous estimation of differential entropy and intrinsic dimension provides complementary perspectives on underlying manifolds.
Proposes PA-DSL for correcting noisy human labels in automated data labeling.
problem Noisy human labels in automated data labeling.
method Uses adjudicated cases to correct noisy human labels and debias analyses.
result Maintains nominal coverage and reduces RMSE by 10-17% relative to using only adjudicated labels.
This work uses adversarial learning to detect and correct feature shifts in various datasets.
problem Detecting and correcting feature shifts in real-world datasets.
method Adversarial learning applied to multiple discriminators to detect and correct feature shifts.
result Mainstream classifiers can effectively localize and correct feature shifts, outperforming existing techniques.
The paper examines when importance weighting is needed for nonparametric and misspecified models.
problem When is importance weighting correction needed for covariate shift adaptation?
method Analysis of IW-corrected kernel ridge regression in various settings.
result The importance weighting correction is needed for nonparametric and misspecified models to obtain the best approximation of the true unknown function.
QC-ST and CoCo methods correct batch effects in metabolomics data.
problem Batch effects in metabolomics data obscure biological variations.
method QC-ST for simultaneous detection of QC samples' mean vectors and covariance matrices, CoCo for covariance correction.
result QC-ST and CoCo improve batch effect correction in metabolomics datasets.
Logit correction improves model performance by correcting spurious correlations.
problem Spurious correlations lead to poor model performance during inference.
method Proposes logit correction (LC) loss to mitigate spurious correlations.
result LC loss outperforms state-of-the-art solutions by 5.5% absolute improvement.
New method preserves GCM spatial dependencies for better climate projections.
problem Systemic biases in GCM output and loss of spatial/temporal dependencies.
method SPECD approach using Vecchia approximation and semi-parametric quantile regression.
result SPECD preserves key marginal and joint distribution properties of precipitation and temperature.
Resampling outperforms reweighting for correcting biased data in machine learning models.
problem Correcting sampling bias in machine learning models trained on biased data sets.
method Compared resampling and reweighting techniques, focusing on their performance with stochastic gradient algorithms.
result Resampling outperforms reweighting when combined with stochastic gradient algorithms.
Local surrogate model improves time series forecasts and provides interpretable explanations.
problem Improving time series forecasting accuracy while maintaining interpretability.
method A local surrogate model is used to correct the base model's predictions, making the corrections interpretable by re-fitting the base model to the error-predicted data.
result The method can discover and explain underlying patterns in the data, improving both accuracy and interpretability.
Improved estimation for imbalanced data using log odds correction and optimal sampling.
problem Parameter estimation with nonuniform negative sampling for imbalanced data.
method Derive asymptotic distribution of IPW estimator, derive optimal sampling probability, propose likelihood-based estimator.
result Improved estimator has the smallest asymptotic variance.
Derives log-corrections in AdS4/CFT3 using supergravity localization.
problem Factorizing log-corrections in AdS4/CFT3.
method Supergravity localization, Atiyah-Singer index theorem, fixed points (NUTs), fixed two-manifolds (Bolts).
result General fixed-point formula for log-corrections in large N expansion.
Proposes CC-NMDF for analyzing manifold-valued data.
problem Nonlinear structure in manifold-valued data requires new analysis methods.
method Curvature-corrected nonnegative manifold data factorization (CC-NMDF) with an iterative algorithm.
result Demonstrates CC-NMDF on real-world diffusion tensor MRI data.
We use tools from geometric statistics to analyze the usual estimation procedure of a template shape. This applies to shapes from landmarks, curves, surfaces, images etc. We demonstrate the asymptotic bias of the template shape estimation using the stratified geometry of the shape space. We give a Taylor expansion of t…
A new framework for fair representation learning using correction vectors.
problem Learning fair representations that are independent of sensitive features.
method Introducing correction vectors to neural network features for fair representation learning.
result The approach does not impact performance while ensuring fairness.
Missing data are ubiquitous in many domains including healthcare. When these data entries are not missing completely at random, the (conditional) independence relations in the observed data may be different from those in the complete data generated by the underlying causal process. Consequently, simply applying existin…
Subtract the Corruption: Training-Data-Free Corrective Machine Unlearning using Task Arithmeticcs.LG
CUTS removes corruption from models without clean data, improving utility and security.
problem Removing corruption from models without access to clean training data.
method CUTS uses a proxy set to amplify corruption and subtract it from model weights.
result CUTS recovers a large fraction of lost utility and nearly eliminates attacks with minimal damage.
With broad applications in various public services like aviation management and urban disaster warning, numerical precipitation prediction plays a crucial role in weather forecast. However, constrained by the limitation of observation and conventional meteorological models, the numerical precipitation predictions are o…
New estimator corrects bias in CKA for sparsely sampled neurons.
problem Bias in CKA for sparsely sampled neurons.
method Novel estimator that corrects for input and feature sampling.
result Reliable model-to-brain alignment with sparsely sampled neurons.
Improved GEC models use scored data from large pretraining to outperform.
problem Addressing data sparsity in Grammatical Error Correction.
method Derive example-level scores from a smaller, higher-quality dataset and incorporate delta-log-perplexity into training schedules.
result Models trained on scored data achieve state-of-the-art results.
AI-generated variables bias regression estimates; methods correct for invalid inference.
problem Bias in regression estimates due to AI-generated variables.
method Two methods: bias correction and joint estimation.
result Valid inference restored through proposed methods.
A new method corrects bias in high-dimensional ridge regression.
problem Inherent bias in ridge regression limits statistical efficiency and scalability.
method Iterative bias correction strategy for and Ridge-Screening method for .
result Valid inferences and asymptotic properties established for de-biased ridge estimators.
A topological approach to stratification learning is developed for point cloud data drawn from a stratified space. Given such data, our objective is to infer which points belong to the same strata. First we define a multi-scale notion of a stratified space, giving a stratification for each radius level. We then use met…