Unsupervised preprocessing can bias cross-validation estimates in regression models.
problem Bias in cross-validation estimates due to unsupervised preprocessing.
method Analysis of three preprocessing procedures: feature selection, grouping, and rescaling.
result Unsupervised preprocessing can introduce substantial bias into cross-validation estimates.
Unsupervised anomaly detection aids doctors in evaluating X-ray images of hands.
problem Lack of labeled medical data and ambiguous symptoms in X-ray images.
method Adopting state-of-the-art unsupervised learning methods for anomaly detection and preprocessing.
result Without preprocessing, unsupervised methods perform randomly on X-ray images of hands.
Improved unsupervised anomaly detection using Random Forest.
problem Enhancing unsupervised anomaly detection accuracy.
method Training Random Forest to distinguish real and synthetic data, then applying transformed distances.
result Significant improvement in anomaly detection accuracy compared to other methods.
In this paper, we investigate the potential of estimating the soil-moisture content based on VNIR hyperspectral data combined with LWIR data. Measurements from a multi-sensor field campaign represent the benchmark dataset which contains measured hyperspectral, LWIR, and soil-moisture data conducted on grassland site. W…
Tactile information is important for gripping, stable grasp, and in-hand manipulation, yet the complexity of tactile data prevents widespread use of such sensors. We make use of an unsupervised learning algorithm that transforms the complex tactile data into a compact, latent representation without the need to record g…
This study evaluates clustering algorithms on high-dimensional data.
problem Comparing clustering algorithms on high-dimensional datasets.
method Evaluation of K-means, DBSCAN, and Spectral Clustering using PCA, t-SNE, UMAP, and multiple metrics.
result UMAP preprocessing improves clustering quality across all algorithms, with Spectral Clustering excelling.
A multi-step framework tackles online unsupervised domain adaptation with novel mean-target subspace computation.
problem Online unsupervised domain adaptation with unlabelled target data arriving sequentially.
method Multi-step framework with a novel mean-target subspace computation and temporal coherency consideration.
result Improved performance over previous approaches on four datasets.
This study automates blood cell classification using computer vision.
problem Automated detection and classification of white blood cells.
method Color-based segmentation, morphological processing, feature extraction, PCA, Unsupervised and Supervised learning algorithms, Deep Convolutional Neural Networks.
result Identification of robust algorithms for automated blood cell classification.
DMGI embeds multiplex networks with node attributes without supervision.
problem Existing methods fail to handle node attributes and multiple relation types in multiplex networks.
method Inspired by DGI, DMGI maximizes mutual information between local and global graph representations, integrating node embeddings from multiple graphs.
result DMGI outperforms state-of-the-art methods on various downstream tasks.
RECol generates error columns to improve outlier detection.
problem Outlier detection in data with complex relationships.
method Generates reconstruction error columns for leave-one-out feature sets.
result Improves ROC-AUC and PR-AUC values of common outlier detection methods.
DPASF stream preprocesses Big Data streams efficiently.
problem Efficient preprocessing of streaming Big Data.
method Implemented six preprocessing algorithms in Apache Flink.
result Preprocessing improves data accuracy in streaming Big Data.
TzK model learns tight conditional priors using side information.
problem Learning tight conditional priors using side information.
method TzK is a conditional probability flow-based model that exploits attributes to learn tight conditional priors around target observations. It trains via approximated ML and supports supervised, unsupervised, and semi-supervised learning.
result TzK produces efficient and stable approximations of arbitrary data distributions, comparable to state-of-the-art models.
Data preprocessing improves data quality for robust data mining.
problem Noisy and incomplete data hinders data mining models.
method Overview of data cleaning, transformation, and preprocessing methods.
result Preprocessing significantly affects data mining model performance.
Automated trading system with preprocessing and reinforcement learning.
problem Efficient portfolio trading for individual investors.
method Feature preprocessing, recurrent reinforcement learning, automated trading algorithm.
result System outperforms other strategies in profit and drawdown.
A framework uses preprocessing to improve psychiatric questionnaire predictions while maintaining interpretability.
problem Weak predictive accuracy and limited interpretability of psychiatric questionnaires.
method Two-stage method: stable preprocessing followed by a linear mapping.
result REFINE outperforms other interpretable approaches in psychiatric and non-psychiatric prediction tasks.
Paper introduces probabilistic methods to approximate archetypal analysis, reducing complexity.
problem Inherent computational complexity of archetypal analysis limits its practical applicability.
method Two preprocessing techniques: dimensionality reduction and representation cardinality reduction, using probabilistic geometry.
result The method effectively reduces scaling and provides near-optimal solutions for prediction errors.
Metalearning finds preprocessing often hinders accuracy, but can improve pipelines.
problem The role of preprocessing in machine learning pipelines is underexplored.
method Extensive empirical study over various algorithms and preprocessors using metalearning.
result Metalearning suggests preprocessing should be used in ML pipeline design.
Enhances solar flare prediction with advanced preprocessing and contrastive learning.
problem Accurate prediction of solar flares to mitigate risks to astronauts and equipment.
method Advanced data preprocessing pipeline and contrastive learning with GRU regression model.
result Exceptional True Skill Statistic (TSS) scores, surpassing previous methods.
This paper analyzes dynamic ensemble selection and preprocessing for multi-class imbalanced datasets.
problem Class imbalance in multi-class datasets where majority classes have more instances.
method Examined dynamic selection techniques and data preprocessing methods for multi-class imbalanced problems.
result Dynamic ensemble improves AUC and G-mean compared to static ensemble.
Enhances diffusion models by preprocessing data to improve reconstruction quality.
problem Slow sampling and poor reconstruction quality in diffusion models, especially for small-scale networks.
method Applying Gaussianization preprocessing to the training data to make the target distribution more Gaussian-like.
result Improves generation quality, especially in the early stages of reconstruction with small networks.
This paper uses feature preprocessing and RRL to automate profitable financial trading.
problem Automating profitable financial trading strategies.
method Feature preprocessing (PCA, DWT) followed by Recurrent Reinforcement Learning (RRL).
result The proposed strategy is effective, robust, and mitigates RRL's drawbacks.
This study compares different data preprocessing methods for gait analysis.
problem Improving the generalizability of machine learning models in gait analysis.
method Compared various data preprocessing steps including filtering, time derivative, normalization, and scaling.
result Different preprocessing combinations affect gait classification performance.
Proposes a new feature preprocessing method using kernel density integral transformation.
problem Feature preprocessing for tabular data in machine learning and statistics.
method Kernel density integral transformation as a drop-in replacement or improved alternative to min-max scaling and quantile transformation.
result Frequently outperforms min-max scaling and quantile transformation with hyperparameter tuning.
Two preprocessing techniques reduce neural network training cost.
problem Training over-parameterized neural networks efficiently.
method Two novel preprocessing techniques to reduce training cost.
result Training cost reduced to sublinear per iteration.
This paper investigates the impact of unconventional preprocessors on deep convolutional neural networks for face identification.
problem The impact of unconventional preprocessors on deep convolutional neural networks for face identification.
method Varying the preprocessing module across different approaches while keeping other network architecture facets constant, using transfer learning from Inception-V3 network features.
result Improvement in discriminative capability of deep networks through preprocessing with HE, quantization, and other methods.
New method speeds up neural network training by preprocessing weight-data correlation.
problem Slow neural network training due to high time complexity.
method Stores weight-data correlation in a tree structure for quick detection of firing neurons.
result Achieves o(nmd) time per iteration with only O(nmd) preprocessing time. Recent advances in weakly supervised classification allow us to train a classifier only from positive and unlabeled (PU) data. However, existing PU classification methods typically require an accurate estimate of the class-prior probability, which is a critical bottleneck particularly for high-dimensional data. This pr…
Paper proposes protecting DNN models with secret key preprocessing.
problem Protecting deep learning models from unauthorized access.
method Block-wise pixel shuffling with secret key for preprocessing.
result Protected models maintain close performance to non-protected models with correct key, but accuracy drops significantly with incorrect key.
Paper presents a data preprocessing method for PHM models.
problem Lack of consistent data preprocessing for PHM applications.
method Comprehensive pipeline for sensor data preprocessing.
result Creation of clean data sets for training machinery health state classifiers.
A new R package for ordinal classification and preprocessing.
problem Lack of proper methods for ordinal data in machine learning.
method Developed an R package named ocapis in Scala.
result Improves classification and preprocessing of ordinal data.
RODMAN improves ML-based disk failure prediction accuracy in cloud environments.
problem Imperfect data quality in real-world cloud environments degrades ML-based disk failure prediction accuracy.
method RODMAN uses three data preprocessing techniques: failure-type filtering, spline-based data filling, and automated pre-failure backtracking.
result RODMAN significantly improves prediction accuracy compared to no preprocessing.
Class-imbalance refers to classification problems in which many more instances are available for certain classes than for others. Such imbalanced datasets require special attention because traditional classifiers generally favor the majority class which has a large number of instances. Ensemble of classifiers have been…
This paper highlights the overlooked role of preprocessing hyperparameters in machine learning model performance.
problem The neglect of preprocessing hyperparameters in machine learning model evaluation.
method Empirical review and illustration of different procedures for generating and evaluating prediction models.
result Users may fail to report or account for preprocessing hyperparameter optimization, leading to exaggerated performance claims.
Within a supervised classification framework, labeled data are used to learn classifier parameters. Prior to that, it is generally required to perform dimensionality reduction via feature extraction. These preprocessing steps have motivated numerous research works aiming at recovering latent variables in an unsupervise…
Recently, deep neural networks have demonstrated excellent performances in recognizing the age and gender on human face images. However, these models were applied in a black-box manner with no information provided about which facial features are actually used for prediction and how these features depend on image prepro…
Develops algorithm to make fair decisions from biased data.
problem Ethical concerns in machine learning fairness.
method Fair Learning through Data Preprocessing (FLAP) algorithm.
result Counterfactual fairness equivalent to conditional independence.
New algorithm samples determinantal point processes with sublinear preprocessing time.
problem Sampling from determinantal point processes efficiently with small expected subset size.
method Proposes an algorithm with sublinear preprocessing and independent sampling cost.
result Achieves sublinear preprocessing time and independent sampling cost.
MobileNet CNN achieves high accuracy in skin disease classification on Android.
problem Skin disease classification using smartphone technology.
method Transfer learning on MobileNet, imbalanced dataset handling (sampling and preprocessing), and data augmentation.
result Oversampling and data augmentation on preprocessing input data achieved 94.4% accuracy.
End-to-end solution for recognizing handwritten numerals, avoiding traditional preprocessing steps.
problem Handwritten numeral string recognition with traditional preprocessing steps.
method YoLo-based model for automatic detection and recognition, avoiding heuristic-based preprocessing and segmentation.
result Proposed method reduces complexity and is a feasible end-to-end solution for numeral string recognition.
CDL index improves clustering validation for non-convex data.
problem Selecting clustering algorithms and hyperparameters without labeled data.
method CDL uses compactness, centers, and covariances to compute a probabilistic description length bound.
result CDL outperforms conventional CVIs on synthetic and image benchmarks.
This study improves quantum classifiers by optimizing data preprocessing.
problem Quantum Machine Learning advantages are not yet clearly demonstrated.
method Used Linear Discriminant Analysis (LDA) for data preprocessing.
result Variational Quantum Algorithm (VQA) outperforms classical classifiers.
Commercial activity trackers are set to become an essential tool in health research, due to increasing availability in the general population. The corresponding vast amounts of mostly unlabeled data pose a challenge to statistical modeling approaches. To investigate the feasibility of deep learning approaches for unsup…
fastml guards against data leakage in automated machine learning.
problem Data leakage during preprocessing before resampling inflates apparent performance.
method fastml uses guarded resampling to re-estimate preprocessing inside each resample.
result Guarded resampling reduces apparent performance compared to global preprocessing.
A new method learns to segment and represent objects jointly without supervision.
problem Learning to segment and represent objects jointly without supervision.
method Iterative variational inference for disentangled object representations.
result System learns to inpaint occluded parts and extrapolates to unseen objects.
DDVFA learns and retrieves clusters without order dependence, outperforming other methods.
problem Order dependence in clustering methods.
method DDVFA combines distributed higher-order activation and match functions with dual vigilance parameters.
result DDVFA outperforms other clustering methods in online learning mode.
This paper compares preprocessing techniques for XGBoost models on various data sets.
problem Improving predictive performance of XGBoost models through optimal data preprocessing.
method Comparison of feature selection, categorical handling, and null imputation methods.
result XGBoost importance by gain is the most consistent and highest-performing method for feature selection.
Study creates a dataset for fraud detection in online auctions.
problem Difficulty in detecting Shill Bidding (SB) due to similarity with normal bidding.
method Scraped eBay auctions, preprocessed data, created SB dataset.
result Shared preprocessed auction dataset for fraud detection.
Value selection reduces model size while maintaining accuracy.
problem Space efficiency in model size reduction.
method Two probabilistic methods based on information theory's metric: PVS and P + VS.
result Value selection achieves balance between accuracy and model size reduction.