This study compares machine learning methods for high-cardinality categorical variables.
problem Machine learning struggles with high-cardinality categorical variables.
method Empirical comparison of tree-boosting, deep neural networks, and linear mixed effects models.
result Tree-boosting with random effects outperforms deep neural networks with random effects.
CardiCat generates synthetic data for high-cardinality tabular datasets.
problem Learning complexities of high-cardinality categorical features in tabular data.
method Substitutes one-hot encoding with regularized dual encoder-decoder embedding layers.
result Generates high-quality synthetic data with a smaller parameter space.
Improves probability estimation for high-cardinality data using graphical models and count sketches.
problem Estimating probabilities in high-cardinality data with structured models.
method Combining graphical models and count-min sketches with random projections.
result Significantly improved error bounds and computational efficiency over existing methods.
New methods encode high-cardinality string variables efficiently.
problem Efficient encoding of high-cardinality string categorical variables.
method Two approaches: Gamma-Poisson matrix factorization and min-hash encoder.
result Improves supervised learning with high-cardinality categorical variables.
Novel GLMMNet model tackles high-cardinality categorical features in actuarial applications.
problem Inadequate encoding methods for high-cardinality categorical features in actuarial data.
method Generalised Linear Mixed Model Neural Network (GLMMNet) integrating a generalised linear mixed model in a deep learning framework.
result GLMMNet often outperforms or performs comparably with entity embedded neural networks, providing transparency.
Proposes a new DNN framework for count data with high-cardinality features.
problem Real-world data often have correlations and high-cardinality categorical features that traditional DNNs overlook.
method Introduces a hierarchical likelihood learning framework with gamma random effects for Poisson DNNs.
result Improves prediction performance by capturing nonlinear effects and subject-specific cluster effects.
Regularized target encoding beats traditional methods for high cardinality features in ML.
problem Efficiently encoding high cardinality categorical variables for ML algorithms.
method Regularized target encoding compared to traditional encodings like integer and one-hot encoding.
result Regularized target encoding consistently provided the best results in a large-scale benchmark experiment.
Similarity encoding improves learning from messy categorical data.
problem Learning from categorical variables with high cardinality and redundancy.
method Similarity encoding, a generalization of one-hot encoding that uses similarities between categories.
result Similarity encoding significantly outperforms traditional encoding methods in prediction accuracy.
Combines boosting with Gaussian process and mixed effects models.
problem Model misspecifications and independence assumptions in boosting.
method Relaxes zero or linearity assumption in Gaussian process and mixed effects models, and independence assumption in boosting.
result Increased prediction accuracy compared to existing approaches.
Improved classification model for high-cardinality categorical predictors.
problem Handling high-cardinality categorical predictors and non-linear data.
method Data-driven binning of spline functions and shrinkage estimators.
result Improved classification precision with interpretable predictors.
Unified comparison of gradient boosting algorithms for insurance claims.
problem Improving predictive accuracy and computational efficiency in insurance claim prediction.
method Unified notation and comprehensive numerical study comparing 12 gradient boosting algorithms on 5 datasets.
result No trade-off between model adequacy and predictive accuracy.
Method reduces categorical data to lower dimensions using density matrices.
problem Dimensionality reduction for categorical data.
method Density-matrix construction from class-conditional frequencies; spectral embedding.
result Low-dimensional spectral embeddings with controlled rank.
Boosting trees predict Twitch subscriptions from user activity.
problem Predicting Twitch user subscriptions from activity data.
method Used boosting trees and target-encodings for high cardinality categoricals.
result User activity can be better predicted than content alone.
HNHN learns from hypergraphs with hyperedge neurons for better classification.
problem Learning from hypergraphs with complex relationships.
method Hypergraph convolution network with hyperedge neurons and adaptive normalization.
result Improved classification accuracy and speed compared to state-of-the-art methods.
New Krylov subspace methods speed up mixed-effects models with crossed random effects.
problem Slow computations for high-dimensional crossed random effects in mixed-effects models.
method Krylov subspace-based methods for generalized mixed-effects models with cross effects.
result Speedups by factors of up to 10,000 in computations for mixed-effects models.
Combines boosting and latent Gaussian models for better predictions.
problem Boosting's assumptions and latent Gaussian models' limitations.
method Integrates tree-boosting and latent Gaussian models.
result Increased prediction accuracy in simulations and real-world data.
MMbeddings reduces categorical embeddings by treating them as latent effects, significantly decreasing parameters and mitigating overfitting.
problem Large cardinalities in categorical embeddings lead to high parameter counts and overfitting.
method MMbeddings treats embeddings as latent random effects in a variational autoencoder framework, reducing parameter count and mitigating overfitting.
result MMbeddings consistently outperforms traditional embeddings across various tasks, demonstrating its potential in machine learning applications.
MC-GMENN improves neural networks for clustered data using Monte Carlo methods.
problem Improving neural network performance on clustered data with correlations.
method MC-GMENN employs Monte Carlo methods to train generalized mixed effects neural networks.
result MC-GMENN outperforms existing models in generalization and quantifying inter-cluster variance.
This paper proposes a method to reduce complexity in GLMs with categorical predictors.
problem Wasteful, hard-to-interpret, and prone to overfitting of traditional one-hot encoding for high-cardinality categorical predictors.
method Clustering categories of categorical predictors through a numerical method that preserves or improves accuracy while reducing the number of coefficients.
result Clustering categories of categorical predictors reduces complexity substantially without harming accuracy.
Study investigates how preprocessing, feature selection, and model selection affect performance on imbalanced genetic data.
problem Challenges in using machine learning on imbalanced genetic datasets.
method Comparative analysis of data preprocessing, feature selection techniques, and machine learning models on imbalanced genetic data.
result Class-imbalanced target variables and skewed predictors have little to no impact on classification performance.
This paper simplifies OPE in large state spaces using state abstractions.
problem Accurately evaluating policies offline in large state spaces.
method Developed a backward-model-irrelevance condition and an iterative state abstraction procedure.
result Deeply-abstracted states substantially simplify OPE sample complexity.
CoSMIC extends flow-based SVI to transdimensional problems.
problem Bayesian structure learning and model selection with multi-model parameter spaces.
method Normalizing flows with a combined stochastic variational transdimensional inference approach.
result Improved performance on high-cardinality model spaces.
LambdaOpt automatically optimizes regularization hyperparameters for better recommendation model performance.
problem Data sparsity and high-cardinality issues in recommendation models.
method LambdaOpt automatically updates regularization coefficients based on validation data, allowing fine-grained regularization.
result LambdaOpt leads to better generalized models with improved recommendation performance.
Bayesian encoding improves lead scoring for WeWork using conjugate models.
problem Encoding high-cardinality categorical features for machine learning.
method Conjugate Bayesian models for categorical features, ensemble learning.
result AUC improved from 0.87 to 0.97 for WeWork's lead scoring engine.
SYNTHONY selects tabular synthesizers based on stress profiling and user intent.
problem Non-uniform performance of tabular generative models across datasets.
method Stress profiling and intent-conditioned tabular synthesis selection.
result Meta-features predict synthesizer performance, improving selection accuracy.
This research compares two encoding methods for categorical attributes in machine learning, affecting model fairness.
problem The impact of encoding protected categorical attributes on fairness in machine learning models.
method Comparison of one-hot encoding and target encoding methods.
result Target encoding can lead to more unfair models compared to one-hot encoding due to induced bias.
Bayesian optimization tackles expensive discrete and mixed parameter spaces.
problem Optimizing expensive functions with discrete and mixed parameters.
method Probabilistic reparameterization to maximize expectation of AF over continuous parameters.
result Our approach provably converges to a maximizer of the AF and enjoys the same regret bounds as standard BO.
Proposes RSP model for efficient big data analysis.
problem Efficiently partitioning big data sets for analysis.
method Random sample partition (RSP) data model and block-level sampling.
result RSP data blocks can estimate statistics and build models equivalent to whole data set.
Data preprocessing improves data quality for robust data mining.
problem Noisy and incomplete data hinders data mining models.
method Overview of data cleaning, transformation, and preprocessing methods.
result Preprocessing significantly affects data mining model performance.
A new method for handling imbalanced big data using ensembles and smart data.
problem Imbalanced data distribution in big data scenarios.
method Smart Data driven Decision Trees Ensemble (SD_DeTE) methodology.
result SD_DeTE outperforms Random Forest in handling imbalanced binary classification problems in big data.
Prevents sensitive data generation in diffusion models using labeled and unlabeled data.
problem Generating sensitive data in diffusion models using unlabeled data.
method Positive-Unlabeled Diffusion Models, approximating ELBO with labeled and unlabeled data.
result Prevents the generation of sensitive data without compromising image quality.
Study reveals Data Shapley's inconsistent performance in data selection tasks.
problem Inconsistency of Data Shapley's performance in data selection across different settings.
method Hypothesis testing framework and identification of utility functions.
result Data Shapley's performance is no better than random selection without specific constraints.
Survey on data collection challenges in machine learning.
problem Data scarcity and need for labeled data in machine learning.
method Comprehensive study of data acquisition, labeling, and improvement techniques.
result Identification of research challenges in data collection.
PRRO generates synthetic tabular data that improves SL performance and class distribution.
problem Low SL utility of synthetic data due to class imbalance and overlooked data relationships.
method Data pruning and column reordering to optimize SL utility.
result Synthetic data generated with PRRO enhances predictive performance and class distribution.
Defines data science as a natural ecosystem with challenges and missions.
problem Challenges and missions in data science due to 5D complexities and data life cycle phases.
method Systemic and data-centric view of data science as a fusion of data universe and its challenges, formalizing a general-purpose architecture.
result Essential data science as a natural ecosystem integrating specific disciplines and high-impact applications.
Synthetic data enhances analytics but requires careful volume management.
problem Accuracy of statistical methods on synthetic data vs. raw data.
method Synthetic Data Generation for Analytics framework using tabular diffusion models.
result Error rate decreases with more synthetic data but may stabilize or increase.
Data science redefines causal inference from observational data, classifying tasks into description, prediction, and counterfactual prediction.
problem Widespread misunderstandings about data science's role in causal inference from observational data.
method Organizing data science tasks into three classes: Description, prediction, and counterfactual prediction (including causal inference).
result The necessity of subject-matter expert knowledge for causal analyses in data science.
This paper evaluates how dirty data affects data mining and machine learning results.
problem Negative impacts of dirty data on data mining and machine learning results.
method Experimental comparison of missing, inconsistent, and conflicting data on classification and clustering algorithms.
result Guidelines for algorithm selection and data cleaning based on experimental findings.
DPASF stream preprocesses Big Data streams efficiently.
problem Efficient preprocessing of streaming Big Data.
method Implemented six preprocessing algorithms in Apache Flink.
result Preprocessing improves data accuracy in streaming Big Data.
This paper introduces C-DSL to improve data mining outcomes by considering context.
problem Data collection ambiguities, data imbalance, hidden biases, lack of domain info, and data incompleteness.
method Developed Context-Driven Data Science Lifecycle (C-DSL) to address data quality issues.
result Tangible improvements to data mining outcomes were achieved through C-DSL.
Proposes using probabilistic models for privacy-preserving synthetic data.
problem Designing high-quality synthetic data for privacy preservation.
method Formulate the problem through probabilistic modelling, choosing a model for the data.
result Statistical discoveries can be reliably reproduced from synthetic data.
Unlabeled data helps stop active learning better than labeled data.
problem Reducing the need for manual annotation in text classification.
method Compared stopping methods based on labeled, unlabeled, and training data.
result Stopping methods using unlabeled data are more effective.
New test ensures quality of shared data in machine learning.
problem Ensuring quality of external data in machine learning tasks.
method Distribution-free two-sample testing procedures grounded in conformal outlier detection.
result Identifies valuable external data agents for model personalization.
Paper creates fair synthetic data ensuring equal predictions across sensitive attributes.
problem Ensuring fair predictions across sensitive attributes in synthetic data.
method Equalizing target probability distributions across sensitive attributes in synthetic data generation.
result Synthetic data provides strong fair predictions, equal across all thresholds.
A new method classifies multiple correlated data streams simultaneously.
problem Classifying multiple correlated data streams in practical scenarios.
method Double-Coupling Support Vector Machines (DC-SVM) considers both internal and external correlations.
result The proposed method outperforms traditional methods on artificial and real-world data streams.
This paper improves neural machine translation training by selecting and denoising data.
problem Reduces negative impact of noisy data on neural machine translation training.
method Measures and selects domain data, applies denoising curriculum using online data selection.
result Significant effectiveness for training on noisy data.
DPA preserves data distribution in reduced dimensions.
problem Loss of data distribution in dimension reduction.
method DPA combines encoder and decoder to match data distribution.
result DPA successfully reconstructs data distribution.
Framework captures missing data in sparse data sets.
problem Capturing missing data in extremely sparse data sets.
method Coupled compound Poisson factorization with stochastic variational inference.
result Explicitly modeling missing data improves results in clustering, prediction, and matrix factorization.