Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,695 papers · 148 categories

Trend · papers per month

10.8%21.6%32.5%43.3% · Jun 202019922001200920172026
48 results for clean data refinement

Neural networks learn clean data patterns first, then noisy data, leading to improved performance initially but deteriorating later.

problem Improvement in prediction error on clean data during early training of neural networks with noisy labels.
method Theoretical analysis and experiments to explore the dynamics of gradient descent and the impact of clean and noisy data.
result Neural networks prioritize learning clean data patterns first, leading to improved performance initially but deteriorating later due to diminishing gradient dominance of clean samples over noisy ones.

PClean automates Bayesian data cleaning for specific datasets.

problem Bayesian inference for diverse and complex data cleaning.
method Domain-specific probabilistic programming language with custom models and inference.
result PClean programs outperform general-purpose PPLs in accuracy and runtime.

We apply basic statistical reasoning to signal reconstruction by machine learning -- learning to map corrupted observations to clean signals -- with a simple and powerful conclusion: it is possible to learn to restore images by only looking at corrupted examples, at performance at and sometimes exceeding training using…

2018-03-12abs ↗pdf ↗

Adversarial training leads to clean data generalization with significant robust overfitting gap.

problem Significant robust generalization gap in adversarial training.
method Two theoretical views: representation complexity and training dynamics.
result ReLU nets with O(ND)O(N D) extra parameters can achieve CGRO.

Study improves resilience against adversarial clean-label attacks in real and noisy settings.

problem Ensuring accurate predictions in the presence of adversarial clean-label samples.
method Sequential learning from a stream of i.i.d. data, allowing abstention for uncertain predictions.
result Theoretical analysis and adaptations for the agnostic setting with a clean-label adversary and noise.

Self-supervised method predicts clean signal and noise distribution from noisy images.

problem Blind denoising and noise estimation in biomedical images with limited clean data.
method Two neural networks jointly predict clean signal and noise distribution from noisy observations.
result Significantly outperforms state-of-the-art algorithms on six biomedical image datasets.

SSMs can be poisoned with clean labels, leading to generalization failure.

problem The implicit bias of SSMs can be manipulated by including special training examples with clean labels.
method Formal proof and empirical demonstration of the phenomenon.
result SSMs can fail to generalize even with clean labels, due to the inclusion of special training examples.

Cincer cleans both new and past data by identifying and relabeling suspicious and counter-examples.

problem Sequential learning under label noise, especially in applications with human supervision.
method Cincer uses example-based explanations to identify and relabel suspicious and counter-examples, leveraging Fisher information matrix approximation.
result Cincer achieves better data and models by clarifying the model's suspicions, especially with FIM approximation.

Enhances deep learning robustness to noise without sacrificing clean data accuracy.

problem Robustness of deep neural networks to input noise.
method Discriminative loss at penultimate layer and class-wise feature alignment with Gaussian noise.
result Improves robustness to various perturbations without degrading clean data accuracy.

The US Census Bureau corrupts data to protect privacy, but we show how to clean and analyze it effectively.

problem Analyzing Census data with intentional corruption to maintain privacy.
method Formulated a semiparametric model, proposed data cleaning, estimation, and inference procedures.
result Demonstrated that data cleaning can maintain precision and provided theoretical and empirical support.

New method generates clean data from corrupted observations.

problem Generating clean data from corrupted observations.
method Iterative update of a transport map using black-box corruption channel access.
result Converges to a self-consistent transport map that effectively inverts the corruption channel.

CLSVAE repairs systematic errors in images with minimal labeled data.

problem Repairing systematic errors in data, especially in images.
method CLSVAE models inliers as a smaller latent space representation, separating inlier and outlier patterns.
result CLSVAE achieves superior repairs with less than 2% labeled data, outperforming other methods.

Image classification problems are typically addressed by first collecting examples with candidate labels, second cleaning the candidate labels manually, and third training a deep neural network on the clean examples. The manual labeling step is often the most expensive one as it requires workers to label millions of im…

2019-10-20abs ↗pdf ↗

Study proposes a clustering and logistic regression algorithm for PU classification under Non-SCAR.

problem PU classification under Non-SCAR condition when SCAR condition is unsatisfied.
method 2-means clustering followed by logistic regression.
result Efficacy of the proposed algorithm demonstrated on 11 real data sets and a synthetic set.

New method improves false-/true-positive-rate estimation in fraud detection with noisy labels.

problem Estimating FPR/TPR in fraud detection with class-conditional label noise.
method Directly cleaning model's validation data to de-correlate cleaning error with model scores.
result Improves accuracy of FPR/TPR estimates, especially in asymmetric label noise scenarios.

Efficient method for training deep learning models with human validation and statistical analysis.

problem Challenges in labeling medical images for deep learning, including time and cost.
method Four-step method using automated data and human visual checks for iterative refinement and statistical validation.
result Initial model accuracy improved from 92% to 98% with statistical validation.

Method cleans noisy training labels for biomedical data.

problem Accurately labeling biomedical data is challenging.
method Reliability-based training data cleaning with inductive conformal prediction.
result Significant enhancements in classification performance across multiple tasks.

Data quality issues have attracted widespread attention due to the negative impacts of dirty data on data mining and machine learning results. The relationship between data quality and the accuracy of results could be applied on the selection of the appropriate algorithm with the consideration of data quality and the d…

2018-03-16abs ↗pdf ↗

The paper cleans label noise in supervised classification using Bernoulli sampling.

problem Label noise degrades supervised classifier performance.
method Proposes a label noise cleaning method based on Bernoulli random sampling.
result The method separates clean and noisy observations without prior label information.

The report presents the theory of harmonic maps from Kähler manifolds.

problem Understanding harmonic maps from Kähler manifolds.
method Reviewing and specializing the theory of harmonic maps between Riemannian manifolds, introducing pluriharmonic maps, and proving refined Bochner formulas.
result Strong rigidity results and applications to symmetric spaces of noncompact type.

EntProp increases entropy of clean samples to generate out-of-distribution data for better DNN performance.

problem Improving deep neural networks' accuracy and robustness to out-of-distribution data.
method High entropy propagation using data augmentation and free adversarial training.
result EntProp achieves higher standard accuracy and robustness with lower training cost.

Deep learning models are vulnerable to various adversarial manipulations of their training data, parameters, and input sample. In particular, an adversary can modify the training data and model parameters to embed backdoors into the model, so the model behaves according to the adversary's objective if the input contain…

2019-05-31abs ↗pdf ↗

Data poisoning attacks compromise the integrity of machine-learning models by introducing malicious training samples to influence the results during test time. In this work, we investigate backdoor data poisoning attack on deep neural networks (DNNs) by inserting a backdoor pattern in the training images. The resulting…

2019-12-03abs ↗pdf ↗

New method tackles label noise on imbalanced datasets by considering class-specific uncertainty.

problem Label noise and class imbalance in imbalanced datasets.
method Epistemic and aleatoric uncertainty-aware class-specific noise modeling.
result Proposed ULC framework improves performance on imbalanced datasets.

It is known that evaluating a certain approximation to the Jones polynomial for the plat closure of a braid is a BQP-complete problem. That is, this problem exactly captures the power of the quantum circuit model. The one clean qubit model is a model of quantum computation in which all but one qubit starts in the maxim…

2007-07-19abs ↗pdf ↗

PrIU optimizes machine learning model updates after data cleaning.

problem Incrementally updating machine learning models after removing problematic training samples.
method Provenance-based approach for efficient model parameter updates.
result PrIU-opt achieves up to two orders of magnitude speed-up compared to retraining from scratch.

When considering answering important questions with data, unsupervised data offers extensive insight opportunity and unique challenges. This study considers student survey data with a specific goal of clustering students into like groups with underlying concept of identifying different poverty levels. Fuzzy logic is co…

2018-11-28abs ↗pdf ↗

Clean-label poisoning attacks inject innocuous looking (and "correctly" labeled) poison images into training data, causing a model to misclassify a targeted image after being trained on this data. We consider transferable poisoning attacks that succeed without access to the victim network's outputs, architecture, or (i…

2019-05-15abs ↗pdf ↗