New method improves ensemble diversity and generalization.
problem Ensemble diversity does not guarantee practical generalization.
method Introduced a new diversity metric and training method for extrapolating differently on local data patches.
result Improves generalization and diversity in practical settings, especially under data limits and covariate shift.
New framework shows diverse training data improves subgroup and overall performance.
problem Lack of understanding how diverse training data affects subgroup and overall performance.
method Casts data collection as part of the learning process, analyzes dataset compositions, and guides dataset design.
result Diverse representation in training data improves subgroup and overall performance.
Proposes Ada-Sit method for mortality prediction of rare diseases.
problem Data insufficiency and clinical diversity of rare diseases make mortality prediction hard.
method Initialization-sharing multi-task learning method (Ada-Sit) for fast adaptation to similar tasks.
result Experimental results show the proposed model is effective for mortality prediction of diverse rare diseases.
SharpBalance improves deep ensemble performance by balancing sharpness and diversity.
problem Improving deep ensemble performance in both in-distribution and out-of-distribution scenarios.
method Introducing SharpBalance, a novel training approach that balances sharpness and diversity within ensembles.
result SharpBalance effectively improves the sharpness-diversity trade-off and ensemble performance in ID and OOD scenarios.
Mixreg improves RL generalization by mixing diverse training environments.
problem Overfitting in RL agents trained in limited environments.
method Trains on a mixture of diverse observations and imposes linearity constraints.
result Mixreg outperforms baselines on unseen testing environments.
A method for efficient reinforcement learning query reformulation.
problem Efficiently learn diverse strategies for query reformulation.
method A framework with specialized sub-agents and a meta-agent trained on full data.
result Improved generalization performance and diversity of reformulation strategies.
Generative model learns diverse generations using DPP to avoid mode collapse.
problem Mode collapse in generative models, limiting output to few modes.
method Inspired by DPP, proposes an unsupervised penalty loss to enhance diversity.
result Generative DPP approach consistently resists mode-collapse and outperforms state-of-the-art methods.
MO-PaDGAN generates diverse, high-performance designs with multiple metrics.
problem Challenges in generating diverse, high-performance designs with multiple metrics.
method MO-PaDGAN uses a new Determinantal Point Processes based loss function for probabilistic modeling of diversity and performances.
result MO-PaDGAN expands the design space towards high-performance regions and generates new designs with high diversity and performances.
Unified framework for supervised classification with diverse training data.
problem Handling different types of training data for supervised classification.
method Generalized robust risk minimization (GRRM) with probabilistic transformations.
result GRRM can handle various training data types and new supervision schemes.
New analysis shows diverse classes in pre-training boost NLP performance.
problem Improving sample efficiency in downstream NLP tasks.
method Proved that diverse classes in pre-training lead to better performance, using a large last linear layer singular value.
result Transfer learning excess risk improves with large ildeν and $O\left(\frac{1}{ ildeν \sqrt{n}}
ight)$ rate. PaDGAN generates diverse, high-quality designs with improved performance.
problem Lack of diversity and performance improvement in generated designs.
method Integrates Determinantal Point Processes for diversity and quality, using GAN framework.
result PaDGAN generates higher quality designs with better diversity and without mode collapse.
Training task diversity improves ICL with linear attention.
problem Understanding the impact of training task diversity on in-context learning.
method Modeling training task vectors as a mixture of low-rank Gaussians.
result Our model explains why training with task diversity shortens the ICL plateau and achieves out-of-distribution generalization.
A new confidence measure improves self-training in biased data.
problem Improving self-training in biased data.
method Proposes a new confidence measure, T-similarity, based on ensemble diversity of linear classifiers.
result Empirically shows the benefit of T-similarity for pseudo-labeling policies on various datasets.
New measures quantify how data augmentation improves model performance.
problem Understanding the effectiveness of data augmentation in deep learning.
method Introduced Affinity and Diversity measures to quantify augmentation performance.
result Augmentation performance is best achieved by optimizing both Affinity and Diversity.
Method learns diverse models to resist spurious correlations.
problem Models learn spurious correlations in i.i.d. training data.
method Multiple models with minimized conditional total correlation.
result Framework significantly outperforms baseline on distribution shift.
Lung segmentation accuracy varies little across diverse datasets.
problem Limited clinical applicability of automated lung segmentation methods.
method Comparison of four deep learning approaches and two standard algorithms on diverse datasets.
result Standard U-net approach yields higher accuracy on routine imaging data.
DEN learns diverse tasks to generalize to unseen tasks.
problem Generalization from a diverse set of classification tasks with limited data.
method Three-block architecture: covariate transformation, distribution embedding, and classification.
result DEN outperforms existing methods in various synthetic and real tasks.
This paper improves ensemble learning for vision tasks by encouraging diversity in predictions.
problem Generating effective ensembles of neural networks for multi-modal data.
method Explicitly optimize a diversity inducing adversarial loss for learning stochastic latent variables.
result Significant improvements in classification accuracy and out-of-distribution detection compared to baselines.
DIVA generates diverse tasks for complex simulators, enabling adaptive agent training.
problem Lack of diverse training data for complex, open-ended simulators.
method Evolutionary approach using domain randomization and procedural generation.
result Successfully trains adaptive agent behavior in complex simulators.
New method improves text classification without labeled target data.
problem Improving text classification under domain shift without labeled target data.
method Diversity-based generalization using multi-head attention with diversity constraints.
result Method matches state-of-the-art performance without labeled target data.
New method selects diverse mini-batches for active learning.
problem Reduce labeled data for deep learning models.
method Sequential selection of diverse mini-batches using K-means clustering.
result Achieves comparable or better performance than previous methods.
This work studies the impact of intra-/inter-class diversity on pre-training datasets and finds a balance for optimal performance.
problem The impact of intra-/inter-class diversity on supervised pre-training datasets and their effect on downstream tasks.
method Empirical study and theoretical analysis of the relationship between diversity types and downstream performance.
result The optimal class-to-sample ratio is invariant to the size of the pre-training dataset and can be predicted.
New findings show that minimal feature diversity at neural network initialization is harmful but can be mitigated with noise.
problem The importance of feature diversity at neural network initialization.
method A series of experiments comparing different initialization schemes, including adding noise.
result Minimal feature diversity is harmful but can be mitigated with noise, even standard GPU noise is sufficient.
TSFMs improve financial forecasting from diverse datasets.
problem Challenges in forecasting financial time series due to noisy, non-stationary, and heterogeneous data.
method Empirical study of TSFMs in global financial markets, evaluating zero-shot inference, fine-tuning, and pre-training from scratch.
result Pre-trained TSFMs on financial data achieve substantial forecasting and economic improvements, highlighting the value of domain-specific adaptation.
DVERGE diversifies adversarial vulnerabilities to enhance robust ensemble models.
problem Diverse adversarial vulnerabilities for robust ensemble models.
method Isolates and diversifies adversarial vulnerabilities through distillation and training.
result Achieves higher robustness against transfer attacks compared to previous methods.
Modern machine learning systems such as image classifiers rely heavily on large scale data sets for training. Such data sets are costly to create, thus in practice a small number of freely available, open source data sets are widely used. We suggest that examining the geo-diversity of open data sets is critical before …
SplitNN-driven Vertical Partitioning enables distributed learning from diverse data sources.
problem Learning from vertically distributed features across institutions.
method A configuration of SplitNN that does not share raw data or model details.
result Flexibility in merging split model outputs and resource efficiency.
Transformers learn to use induction heads or shortcuts based on data diversity.
problem How data diversity influences the behavior of transformers.
method Gradient-based training of a single-layer transformer on a minimal task.
result Data diversity steers transformers toward induction heads or shortcuts.
Deep learning dynamics and NTK evolution studied through diverse measures.
problem Understanding the training dynamics of deep neural networks and their loss landscapes.
method Phenomenological analysis of training dynamics in multiple architectures and datasets.
result Training dynamics exhibit a chaotic initial transient followed by a stable phase, with the NTK evolving to match full network performance.
Improved GANs model geological facies with diversity and unbiased distribution.
problem Generating unbiased and representative geological models from training images.
method Info-WGAN combining InfoGAN, Wasserstein distance, and Gradient Penalty.
result Generated samples have equal probability distribution as training data.
Improves exploration in reinforcement learning with diverse population.
problem Lack of diversity in reinforcement learning agents leads to limited exploration.
method Optimizes a population of agents using volume of behavioral manifold and online learning techniques.
result Improves exploration without sacrificing performance.
Ensembles of deep neural networks significantly improve generalization accuracy. However, training neural network ensembles requires a large amount of computational resources and time. State-of-the-art approaches either train all networks from scratch leading to prohibitive training cost that allows only very small ens…
A co-evolutionary approach for Heston model calibration reduces overfitting with diverse datasets.
problem Overfitting and lack of generalization in Heston model calibration.
method Coupling a genetic algorithm with an evolving neural inverse map, using both GA-history sampling and Latin hypercube sampling.
result Diverse datasets improve out-of-sample stability and calibration accuracy.
METASET selects diverse unit cells for efficient data-driven metamaterial design.
problem Imbalanced datasets in unit cells can bias data-driven metamaterial design.
method METASET uses similarity metrics and DPPs to select diverse subsets of unit cells.
result Smaller, diverse subsets improve search process and structural performance.
Meta-learning techniques like MAML and Reptile fail to generalize well to out-of-distribution tasks compared to simple finetuning.
problem Generalization of meta-learning techniques in low-data scenarios.
method Investigation of MAML, Reptile, and finetuning on various tasks.
result MAML and Reptile specialize for fast adaptation but fail to generalize to out-of-distribution tasks.
Improved transfer learning with expert models, reducing compute and speeding up performance.
problem Improving sample efficiency and reducing computational requirements for new tasks.
method Training a diverse set of experts using existing label structures and performance proxies to select the relevant expert for each target task.
result Significant speed-up of 2-3 orders of magnitude compared to competing approaches.
New algorithms improve federated learning accuracy and stability with real-world data.
problem Real-world data diversity and imbalance challenge federated learning.
method Developed new algorithms (FedVC, FedIR) to resample and reweight data.
result Significant improvements in accuracy and stability of federated learning.
Fine-tunes diffusion models to generate diverse samples with high genuine rewards.
problem Reward collapse in finetuning diffusion models.
method Entropy-regularized control against pretrained diffusion models.
result Efficient generation of diverse samples with high genuine rewards.
Meta-CoTGAN improves adversarial text generation by preventing mode collapse.
problem Mode collapse in adversarial text generation.
method Meta-Cooperative Training Paradigm with a language model.
result Meta-CoTGAN effectively slows down mode collapse and improves generation quality and diversity.
DivDis learns diverse hypotheses from underspecified data to improve robustness.
problem Learning from underspecified datasets leads to multiple equally viable solutions, causing out-of-distribution issues.
method DivDis framework: 1) learns diverse hypotheses using unlabeled test data, 2) selects one hypothesis with minimal additional supervision.
result DivDis finds robust features in image and natural language processing problems.
New method shows random, diverse initializations are not essential for deep neural networks.
problem The necessity of random, diverse initializations in deep neural networks.
method Constructed a deep convolutional network with identical features by initializing weights to 0, enabling signal propagation and stable gradients.
result Random, diverse initializations are not necessary for training neural networks.
MOD improves ensemble-based uncertainty estimates by encouraging larger diversity.
problem Improving model uncertainty estimates for inputs not seen during training.
method Maximize Overall Diversity (MOD) approach to encourage larger diversity in ensemble predictions.
result Significantly improves predictive performance for out-of-distribution test examples.
Supervised machine learning based state-of-the-art computer vision techniques are in general data hungry and pose the challenges of not having adequate computing resources and of high costs involved in human labeling efforts. Training data subset selection and active learning techniques have been proposed as possible s…
COLD optimizes samples by projecting them into a latent space, ensuring they are distinct from training data.
problem Optimizing discrete data for specific characteristics using gradient-based methods often leads to dissimilar samples.
method Constrained Optimisation with Latent Distributions (COLD) to find optimal samples similar to but distinct from training data.
result COLD generates diverse, high-quality samples with similar properties to training data.
Picket guards against corrupted data in machine learning models.
problem Data corruption biases models and invalidates predictions.
method PicketNet detects corrupted data using self-supervised deep learning; flags corrupted queries online.
result Picket consistently protects models from corrupted data during training and deployment.
Unified framework for portfolio optimization using multiple hypotheses.
problem Risk diversification in portfolio allocation.
method Structured ensemble learning approach with diversity control.
result Structured ensembles link predictor diversity to risk diversification.
Single model learns physics from diverse data.
problem Lack of universal physics models for diverse applications.
method General Physics Transformer (GPhyT) trained on diverse physics data.
result Single model achieves superior performance across multiple physics domains.
Study evaluates reinforcement learning for trading diverse stocks, finds Q-learning outperforms.
problem Evaluating reinforcement learning for trading diverse stocks.
method Implemented Value Iteration (VI), State-action-reward-state-action (SARSA), and Q-Learning on a diverse stock portfolio dataset.
result Q-learning performs better than VI and SARSA during testing, but performance varies based on market conditions.