New model improves diversity in question generation over text.
problem Inadequate evaluation metrics for diverse questions.
method Proposed a variational encoder-decoder model and evaluation scheme.
result Improved diversity in generated questions without loss of quality.
High-capacity neural network ensembles often benefit more from high-capacity models than from increased diversity.
problem The performance of high-capacity neural network ensembles is often harmed by interventions that promote predictive diversity.
method A large-scale study of nearly 600 neural network classification ensembles, examining various interventions and architectures.
result Discouraging predictive diversity can be benign in large-network ensembles, and higher-capacity models often yield better performance than diverse architectures.
Pantypes improve prototypical models by capturing diverse input distributions.
problem Prototypical models lack sufficient data representation in low density regions.
method Introducing pantypes, a sparse set of diverse objects to represent the full diversity of input distribution.
result Pantypes empower prototypical models to foster high diversity, interpretability, and fairness.
A stock market is called diverse if no stock can dominate the market in terms of relative capitalization. On one hand, this natural property leads to arbitrage in diffusion models under mild assumptions. On the other hand, it is also easy to construct diffusion models which are both diverse and free of arbitrage. Can o…
This paper improves model generalization by integrating diverse pretrained models.
problem Leveraging diverse pretrained models for robust out-of-distribution generalization.
method Characterize and integrate diverse pretrained models based on diversity and correlation shifts.
result Demonstrates state-of-the-art out-of-distribution generalization performance.
Unified theory explains diversity in ensemble learning.
problem Explaining diversity in ensemble learning across various scenarios.
method Developed a framework revealing diversity as a hidden dimension in bias-variance decomposition.
result Proved exact bias-variance-diversity decompositions for multiple losses in regression and classification.
Proposes Vendi Score for evaluating diversity in ML models.
problem Lack of flexible diversity evaluation metrics in ML.
method Integrates ecological and quantum statistical mechanics concepts to define Vendi Score.
result Vendi Score enables flexible diversity evaluation without requiring a reference dataset.
Ensembles depend on diversity for improved performance. Many ensemble training methods, therefore, attempt to optimize for diversity, which they almost always define in terms of differences in training set predictions. In this paper, however, we demonstrate the diversity of predictions on the training set does not nece…
MO-PaDGAN generates diverse, high-performance designs with multiple metrics.
problem Challenges in generating diverse, high-performance designs with multiple metrics.
method MO-PaDGAN uses a new Determinantal Point Processes based loss function for probabilistic modeling of diversity and performances.
result MO-PaDGAN expands the design space towards high-performance regions and generates new designs with high diversity and performances.
Product diversity of large US firms has declined steadily since 1997.
problem Lack of data on global product diversity makes investigation difficult.
method Text mining of US firms' product descriptions from 1997-2017.
result Product diversity of large US firms has been declining since 1997.
Study shows diverse data sources improve cryptocurrency forecasting models.
problem Improving cryptocurrency market forecasting accuracy.
method Integrating various data types, including on-chain metrics, traditional indices, and macroeconomic indicators.
result Data source diversity significantly enhances forecasting model performance.
This paper presents a general framework for studying diverse beliefs in dynamic economies. Within this general framework, the characterization of a central-planner general equilbrium turns out to be very easy to derive, and leads to a range of interesting applications. We show how for an economy with log investors hold…
Ensemble learning is a methodology that integrates multiple DNN learners for improving prediction performance of individual learners. Diversity is greater when the errors of the ensemble prediction is more uniformly distributed. Greater diversity is highly correlated with the increase in ensemble accuracy. Another attr…
PaDGAN generates diverse, high-quality designs with improved performance.
problem Lack of diversity and performance improvement in generated designs.
method Integrates Determinantal Point Processes for diversity and quality, using GAN framework.
result PaDGAN generates higher quality designs with better diversity and without mode collapse.
Generating molecules with desired chemical properties is important for drug discovery. The use of generative neural networks is promising for this task. However, from visual inspection, it often appears that generated samples lack diversity. In this paper, we quantify this internal chemical diversity, and we raise the …
In 1999 Robert Fernholz observed an inconsistency between the normative assumption of existence of an equivalent martingale measure (EMM) and the empirical reality of diversity in equity markets. We explore a method of imposing diversity on market models by a type of antitrust regulation that is compatible with EMMs. T…
The paper clusters hypergraphs to find diverse and experienced groups based on past experiences.
problem Finding diverse and experienced groups with respect to past experiences.
method Regularized edge-based hypergraph clustering objective with a 2-approximation algorithm.
result Demonstrates an efficient 2-approximation algorithm for clustering hypergraphs.
Paper explores how text generation quality and diversity metrics relate to distribution fitting.
problem Unclear relation between text generation quality and diversity metrics and distribution fitting.
method Theoretical approach to prove a linear combination of quality and diversity metrics can be a divergence metric.
result CR/NRR proposed as a better substitute for BLEU/Self-BLEU metrics.
Presents STRIPE model for probabilistic forecasting of non-stationary time series.
problem Probabilistic forecasting of non-stationary time series.
method STRIPE model representing structured diversity based on shape and time features, with diversification mechanism using determinantal point processes (DPP).
result STRIPE significantly outperforms baseline methods for representing diversity while maintaining forecasting accuracy.
New measures quantify how data augmentation improves model performance.
problem Understanding the effectiveness of data augmentation in deep learning.
method Introduced Affinity and Diversity measures to quantify augmentation performance.
result Augmentation performance is best achieved by optimizing both Affinity and Diversity.
How can we measure whether a natural language generation system produces both high quality and diverse outputs? Human evaluation captures quality but not diversity, as it does not catch models that simply plagiarize from the training set. On the other hand, statistical evaluation (i.e., perplexity) captures diversity b…
Proposes a method to generate diverse translations by conditioning on target domain.
problem NMT models lack diversity in translations, even with search algorithms.
method Condition the decoder on a latent variable representing target domain, generated by a target encoder.
result Generated diverse translations without affecting performance or training time.
DMNL bandits optimize assortment choices balancing relevance and diversity.
problem Balancing relevance-driven choice with within-assortment diversity.
method Augments MNL choice probabilities with a submodular diversity function, proposing a white-box UCB-based algorithm.
result Achieves at least a (1−e+11)-approximate regret bound of $ ilde{O}\left(d \sqrt{T/K}
ight)$. New method improves text classification without labeled target data.
problem Improving text classification under domain shift without labeled target data.
method Diversity-based generalization using multi-head attention with diversity constraints.
result Method matches state-of-the-art performance without labeled target data.
EDU method finds diverse optimal solutions for expensive simulators.
problem Optimizing expensive black-box simulators for diverse solutions.
method EDU method searches for diverse locally-optimal solutions within a tolerance level.
result EDU yields a closed-form acquisition function facilitating efficient sequential queries.
A new LLM-based method enhances diversity in oversampling for imbalanced classification.
problem Limited diversity in synthetic minority samples generated by current LLM-based approaches reduces robustness and generalizability.
method Condition synthetic sample generation on minority labels and features, use permutation strategy for fine-tuning, fine-tune on minority and interpolated samples.
result Significantly outperforms eight SOTA baselines in diverse synthetic sample generation and downstream classification tasks.
This work tackles semi-supervised federated learning by reducing model gradient diversity.
problem Improving test accuracy in semi-supervised federated learning with limited labeled data.
method Investigates and compares various design choices including consistency regularization loss, Batch Normalization, and Group Normalization.
result Grouping-based model averaging combined with Group Normalization and consistency regularization loss improves test accuracy.
Ensembles, as a widely used and effective technique in the machine learning community, succeed within a key element -- "diversity." The relationship between diversity and generalization, unfortunately, is not entirely understood and remains an open research issue. To reveal the effect of diversity on the generalization…
Kernel sparsity ("dying ReLUs") and lack of diversity are commonly observed in CNN kernels, which decreases model capacity. Drawing inspiration from information theory and wireless communications, we demonstrate the intersection of coding theory and deep learning through the Grassmannian subspace packing problem in CNN…
A financial market is called "diverse" if no single stock is ever allowed to dominate the entire market in terms of relative capitalization. In the context of the standard Ito-process model initiated by Samuelson (1965) we formulate this property (and the allied, successively weaker notions of "weak diversity" and "asy…
Introduce a variance-weighted batch distribution for diverse sampling in diffusion models.
problem Independent sampling in diffusion models.
method Introduce a variance-weighted batch distribution.
result Sampler with a transparent probabilistic target.
D-CBRS manages memory for continual learning by accounting for intra-class diversity.
problem Forgetting in continual learning, especially with class-imbalanced data.
method D-CBRS introduces a novel approach to store instances in memory, considering intra-class diversity.
result D-CBRS outperforms existing methods on data sets with intra-class diversity.
We introduce a new sampling method for large language models that balances diversity and parallelism.
problem Balancing diversity and parallelism in decoding for large language models.
method Arithmetic sampling framework compatible with various sampling variations.
result Improves estimation of expected BLEU score reward and reduces the gap with beam search.
We propose a new model for making generalizable and diverse retrosynthetic reaction predictions. Given a target compound, the task is to predict the likely chemical reactants to produce the target. This generative task can be framed as a sequence-to-sequence problem by using the SMILES representations of the molecules.…
Paper presents a deterministic method for diverse subset selection.
problem Diverse subset selection problems in recommendation, summarization, and search.
method Greedy deterministic adaptation of k-DPP for low-rank approximations and image search.
result The method yields low-rank approximations of kernel matrices and demonstrates effectiveness in image search.
Though deep neural networks have achieved significant progress on various tasks, often enhanced by model ensemble, existing high-performance models can be vulnerable to adversarial attacks. Many efforts have been devoted to enhancing the robustness of individual networks and then constructing a straightforward ensemble…
This paper improves ensemble learning for vision tasks by encouraging diversity in predictions.
problem Generating effective ensembles of neural networks for multi-modal data.
method Explicitly optimize a diversity inducing adversarial loss for learning stochastic latent variables.
result Significant improvements in classification accuracy and out-of-distribution detection compared to baselines.
New measures quantify diversity of latent representations using metric space magnitude.
problem Evaluating the diversity of latent representations in machine learning models.
method Developed magnitude-based measures for latent representations, stable under data perturbations.
result Demonstrated superior performance across various domains and tasks.
Improved UCB algorithm for diversity in bandits with lower bounds.
problem Improving diversity in bandit-based recommendations.
method Designing a UCB algorithm exploiting the specific structure of the problem.
result The UCB algorithm achieves bounded distribution-dependent regret under certain conditions.
Ensemble of diverse CNNs detects and mitigates adversarial attacks.
problem Detecting and defending against adversarial attacks.
method An ensemble of specialized CNNs with a voting mechanism.
result Significant reduction in adversarial attack risk rate.
Proposes a method to increase diversity without sacrificing meritocracy.
problem Systemic bias in datasets affecting diversity and meritocracy.
method Optimally flipping outcome labels and training classification models simultaneously.
result The price of diversity is low and sometimes negative, enhancing diversity without significantly affecting meritocracy.
Self-distillation improves model performance by increasing teacher diversity and smoothing predictions.
problem Improving model generalization and performance through self-distillation.
method Interpreting self-distillation as MAP estimation and proposing instance-specific label smoothing.
result Self-distillation enhances model performance by increasing teacher diversity and smoothing predictions.
We design a new algorithm for batch active learning with deep neural network models. Our algorithm, Batch Active learning by Diverse Gradient Embeddings (BADGE), samples groups of points that are disparate and high-magnitude when represented in a hallucinated gradient space, a strategy designed to incorporate both pred…
Proposes Ada-Sit method for mortality prediction of rare diseases.
problem Data insufficiency and clinical diversity of rare diseases make mortality prediction hard.
method Initialization-sharing multi-task learning method (Ada-Sit) for fast adaptation to similar tasks.
result Experimental results show the proposed model is effective for mortality prediction of diverse rare diseases.
New metric evaluates generative models across domains, diagnosing fidelity, diversity, and generalization.
problem Evaluating generative models in diverse domains with limited metrics.
method Introduces a 3D evaluation metric (α-Precision, β-Recall, Authenticity) for domain-agnostic diagnostics. result Unified metric characterizes fidelity, diversity, and generalization, diagnosing model performance.
Thanks to the recent availability of comprehensive and detailed online databases of startup companies, it has become possible to more directly investigate startup ecosystems i.e. startup populations in specific regions. In this paper, we analyze the emergence of 20+ such ecosystems in Europe and the USA, with a specifi…
LoRA-MCL improves language models by generating diverse sentence continuations.
problem Language models struggle with generating diverse, plausible sentence continuations.
method Low-Rank Adaptation combined with Multiple Choice Learning (MCL) to handle ambiguity.
result LoRA-MCL generates high-diversity and relevant outputs in various tasks.
RL enhances LLM planning but introduces spurious solutions and diversity collapse.
problem Theoretical understanding of RL's benefits and limitations in LLM planning.
method Graph-based abstraction, policy gradient, Q-learning, supervised fine-tuning.
result RL's exploration is crucial for generalization, but PG suffers from diversity collapse.