Study finds real-world datasets contain natural experiments that can improve model performance.
problem Detecting natural experiments in real-world datasets for causal inference.
method Synthetic graph simulation and feature selection based on causal links.
result Real-world datasets contain natural experiments that can be exploited for improved model performance.
Study real-world noisy labels from human annotations for better understanding.
problem Understanding and modeling real-world label noise in machine learning.
method Developed two new benchmark datasets (CIFAR-10N, CIFAR-100N) with human-annotated real-world noisy labels.
result Real-world noisy labels exhibit instance-dependent patterns, not class-dependent as previously assumed.
Study introduces a benchmark suite for evaluating neural MI estimators on real-world unstructured datasets.
problem Lack of comprehensive evaluation methods for neural MI estimators on real-world unstructured datasets.
method Developed a benchmark suite using same-class sampling and a binary symmetric channel trick.
result Showed accurate manipulation of true MI values of real-world datasets.
Improved tabular models learn better from real-world data.
problem Tabular models perform poorly on real-world datasets when trained only on synthetic data.
method Continued pre-training on a curated set of real-world datasets.
result Real-TabPFN achieves superior predictive accuracy on 29 datasets.
Federated learning is a new machine learning paradigm which allows data parties to build machine learning models collaboratively while keeping their data secure and private. While research efforts on federated learning have been growing tremendously in the past two years, most existing works still depend on pre-existin…
Paper addresses challenges in benchmarking stream learning algorithms with real-world data.
problem Lack of publicly available non-stationary real-world datasets for evaluating stream algorithms.
method Proposes a new public data repository for benchmarking stream algorithms with real-world data.
result Mitigates problems related to dataset choice in experimental evaluation of stream classifiers and drift detectors.
Study evaluates scalability and real-world impact of disentangled representations.
problem Scalability and real-world impact of disentangled representations.
method New high-resolution dataset and architectures for disentangled representation learning.
result Disentanglement predicts out-of-distribution task performance.
This paper analyzes and improves active learning techniques for real-world projects.
problem Reducing labelling effort in machine learning models with real-world constraints.
method Systematic study of active learning issues, proposing techniques to address model convergence, annotation error, and dataset imbalance.
result Presentation of two techniques to speed up active learning: partial uncertainty sampling and larger query size.
Study reveals how spectral bias affects learnability on real-world data.
problem Understanding how well complex datasets can be learned using kernel methods.
method Use eigenvalues and eigenfunctions from idealized data to reveal spectral bias on real-world data.
result Bound learnability on real-world data using symmetries of realistic kernels.
Clustering analysis has become a ubiquitous information retrieval tool in a wide range of domains, but a more automatic framework is still lacking. Though internal metrics are the key players towards a successful retrieval of clusters, their effectiveness on real-world datasets remains not fully understood, mainly beca…
New model detects crying in real-world settings with improved accuracy.
problem Generalization of cry detection models to real-world environments.
method Evaluated machine learning approaches on a novel dataset of real-world infant crying.
result Improved F1 score of 0.613 for crying event recognition in real-world settings.
We uncover scaling laws and statistical structure in complex datasets.
problem Understanding universal traits in complex datasets.
method Analogizing data to physical systems, using statistical physics and RMT.
result Real-world datasets and Gaussian data with long-range correlations share the same RMT universality class.
In federated learning, a central server coordinates the training of a single model on a massively distributed network of devices. This setting can be naturally extended to a multi-task learning framework, to handle real-world federated datasets that typically show strong statistical heterogeneity among devices. Despite…
HNHN learns from hypergraphs with hyperedge neurons for better classification.
problem Learning from hypergraphs with complex relationships.
method Hypergraph convolution network with hyperedge neurons and adaptive normalization.
result Improved classification accuracy and speed compared to state-of-the-art methods.
SurvHTE-Bench benchmarks HTE estimation in survival analysis with diverse datasets.
problem Challenges in estimating HTEs from right-censored survival data.
method Modular synthetic datasets, semi-synthetic datasets, and real-world datasets.
result First rigorous comparison of survival HTE methods under diverse conditions.
We created financial benchmarks for distribution shifts in crude oil prices and volatility.
problem Scarcity of task-labeled time-series benchmarks in finance.
method Transformed asset price data into volatility proxies, generated task labels based on distribution shifts, and made datasets publicly available.
result Inclusion of task labels improves continual learning algorithms' performance on real-world data.
New algorithms improve federated learning accuracy and stability with real-world data.
problem Real-world data diversity and imbalance challenge federated learning.
method Developed new algorithms (FedVC, FedIR) to resample and reweight data.
result Significant improvements in accuracy and stability of federated learning.
XAI-Bench releases synthetic datasets for evaluating feature attribution methods.
problem Evaluating and comparing feature attribution methods is challenging.
method Released synthetic datasets and benchmarking library.
result Efficiently evaluates feature attribution methods across various metrics.
Modern CATE models often fail to outperform a trivial zero-effect predictor, highlighting significant challenges.
problem Lack of robustness in CATE models when applied to real-world data.
method Large-scale benchmark study using diverse observational sampling strategies and novel statistics.
result 62% of CATE estimates have higher MSE than a trivial zero-effect predictor, indicating poor performance.
Body-worn cameras are now commonly used for logging daily life, sports, and law enforcement activities, creating a large volume of archived footage. This paper studies the problem of classifying frames of footage according to the activity of the camera-wearer with an emphasis on application to real-world police body-wo…
Learning meaningful and compact representations with disentangled semantic aspects is considered to be of key importance in representation learning. Since real-world data is notoriously costly to collect, many recent state-of-the-art disentanglement models have heavily relied on synthetic toy data-sets. In this paper, …
Framework for applying GPs to real-world data with scalability guidelines.
problem Deployment of Gaussian Processes (GPs) is hindered by computational costs and lack of guidelines.
method Proposed a framework for identifying GP suitability and setting up robust models, formalizing decisions of experienced practitioners.
result More accurate results at test time for glacier elevation change case study.
PNNs improve treatment outcomes in TAVR and liver trauma.
problem Improving treatment outcomes in medical procedures.
method Multimodal Prescriptive Neural Networks (PNNs) combining optimization and machine learning.
result PNNs significantly improve estimated outcomes in medical procedures.
DNN2LR bridges DNN power and LR interpretability.
problem Combining DNN power and LR interpretability for real-world tabular data.
method Automatic feature crossing method based on DNN interpretation inconsistencies.
result DNN2LR outperforms complex DNN models and feature crossing methods.
Advances in sensor technology have enabled the collection of large-scale datasets. Such datasets can be extremely noisy and often contain a significant amount of outliers that result from sensor malfunction or human operation faults. In order to utilize such data for real-world applications, it is critical to detect ou…
Adaptive regularization tackles heteroskedastic and imbalanced datasets in deep learning.
problem Heteroskedastic and imbalanced datasets challenge deep learning due to varying label uncertainty and long-tailed label distributions.
method Data-dependent adaptive regularization that applies stronger regularization to higher-uncertainty, lower-density regions.
result Significant improvement in noise-robust deep learning over other methods on benchmark tasks.
FairJob dataset for job recommendations in advertising, preserving fairness and utility.
problem Fairness in online job recommendations for advertising.
method Anonymized dataset with proxy for sensitive attributes, fairness metric computation.
result Demonstrated potential improvements in fairness with trade-offs in utility.
Study evaluates methods for improving model robustness to various real-world distribution shifts.
problem Improving model robustness to real-world distribution shifts like geographic changes.
method Introduced new datasets and evaluated existing methods on four types of shifts (style, blurriness, location, camera operation).
result Data augmentations and larger models can improve robustness on real-world distribution shifts, contrary to prior claims.
FinDiff generates synthetic financial data for regulatory tasks.
problem Sharing microdata for research due to privacy regulations.
method Diffusion model using embedding encodings for mixed modality financial data.
result FinDiff excels in generating high-fidelity, privacy-preserving synthetic financial data.
In many real-world applications, we want to exploit multiple source datasets of similar tasks to learn a model for a different but related target dataset -- e.g., recognizing characters of a new font using a set of different fonts. While most recent research has considered ad-hoc combination rules to address this probl…
An approximate method for conducting resampling in Lasso, the ℓ1 penalized linear regression, in a semi-analytic manner is developed, whereby the average over the resampled datasets is directly computed without repeated numerical sampling, thus enabling an inference free of the statistical fluctuations due to sam…
Deep learning models for survival analysis have gained significant attention in the literature, but they suffer from severe performance deficits when the dataset contains many irrelevant features. We give empirical evidence for this problem in real-world medical settings using the state-of-the-art model DeepHit. Furthe…
A new imputation method considers feature importance for missing value completion.
problem Missing values in datasets reduce accuracy and increase processing difficulty.
method Iterative matrix completion with feature importance learning.
result The method outperforms existing imputation algorithms on various datasets.
Discovering causal relationships from data is the ultimate goal of many research areas. Constraint based causal exploration algorithms, such as PC, FCI, RFCI, PC-simple, IDA and Joint-IDA have achieved significant progress and have many applications. A common problem with these methods is the high computational complex…
Paper proves mathematically that poisoning datasets can be detected.
problem Detecting data poisoning attacks in datasets.
method Mathematical definition and Conformal Separability Test.
result Dataset poisoning can be effectively detected.
Paper tackles domain adaptation for contextual bandits with sub-linear regret.
problem Adapting contextual bandit algorithms across domains with distribution shift.
method Learn a bandit model for the target domain using feedback from the source domain.
result Sub-linear regret bound maintained across domains.
Evaluates transfer learning methods in dynamic data availability scenarios.
problem Real-world data availability varies over time, leading to unrealistic TL method evaluations.
method Proposes a data manipulation framework to simulate varying data availability and domain transformations.
result Demonstrates the usefulness of the framework on proprietary and publicly available datasets.
Enhances GNNs for causal relationship learning.
problem Lack of robust causal modeling in GNNs.
method Synthesized dataset with known causal relationships, lightweight GNN module.
result Empirically validated GNN module improves causal learning.
Enhanced Extended Isolation Forest (EIF+) improves anomaly detection and provides interpretable explanations.
problem Detecting anomalies in complex datasets and explaining model predictions.
method Extended Isolation Forest (EIF) and Extended Isolation Forest Feature Importance (ExIFFI) methods.
result EIF+ outperforms EIF in detecting unseen anomalies and provides better generalization.
A tool predicts BN performance for real-world datasets.
problem Lack of validation for BN results in real-world applications.
method Synthetic datasets and structure learning algorithms to estimate BN performance.
result Automatic recommendations for BN performance based on synthetic data.
The research community has increasing interest in autonomous driving research, despite the resource intensity of obtaining representative real world data. Existing self-driving datasets are limited in the scale and variation of the environments they capture, even though generalization within and between operating regio…
Synthetic data mimics real-world demographics for fairness testing.
problem Lack of complete, representative datasets for fairness testing.
method Construct synthetic datasets using overlapping real and separate datasets.
result Synthetic data yields consistent fairness metrics with real data.
Proposes PUUPL for PUL in imbalanced datasets, boosting minority class signals.
problem Imbalanced datasets and model calibration in PUL.
method Uncertainty-aware pseudo-labeling procedure (PUUPL).
result Substantial performance gains in highly imbalanced settings.
Automatic analysis of teacher and student interactions could be very important to improve the quality of teaching and student engagement. However, despite some recent progress in utilizing multimodal data for teaching and learning analytics, a thorough analysis of a rich multimodal dataset coming for a complex real lea…
GNNs improve supply chain analytics with real-world benchmarks.
problem Limited research on applying GNNs to supply chain management.
method Conceptual discussions, detailed formulations, examples, mathematical definitions, and task guidelines.
result GNN-based models outperform other methods by 10-40% in various supply chain tasks.
Noisy labels are ubiquitous in real-world datasets, which poses a challenge for robustly training deep neural networks (DNNs) as DNNs usually have the high capacity to memorize the noisy labels. In this paper, we find that the test accuracy can be quantitatively characterized in terms of the noise ratio in datasets. In…
A faster method for visualization recommendations on large datasets.
problem Infeasibility of state-of-the-art vis-rec models on large datasets due to high computational time.
method Reinforcement-learning (RL) framework that identifies optimal statistics within a time budget.
result Significantly reduces time-to-visualize with minimal error compared to baseline approaches.
IKD uses eigen-decomposition for nonlinear dimensionality reduction.
problem Lack of sophisticated and nonlinear dimensionality reduction methods.
method Inverse Kernel Decomposition (IKD) based on eigen-decomposition of sample covariance matrix.
result IKD achieves comparable performance to optimization-based methods with faster running speeds.