Proposes method to discover diverse near-optimal policies in reinforcement learning.
problem Finding different solutions to the same problem in reinforcement learning.
method Formalizes problem as CMDP, uses Successor Features, proposes new diversity rewards.
result Proposed method discovers diverse near-optimal policies that are robust and distinct.
Standard practice in training neural networks involves initializing the weights in an independent fashion. The results of recent work suggest that feature "diversity" at initialization plays an important role in training the network. However, other initialization schemes with reduced feature diversity have also been sh…
This work explores how XAI methods can visualize the diversity of feature representations in Bayesian Neural Networks.
problem Explaining the diversity of feature representations learned by Bayesian Neural Networks.
method Application of global XAI methods to visualize and quantify the diversity of feature representations.
result The diversity of learned feature representations correlates with uncertainty estimates and network width.
Online feature selection has been an active research area in recent years. We propose a novel diverse online feature selection method based on Determinantal Point Processes (DPP). Our model aims to provide diverse features which can be composed in either a supervised or unsupervised framework. The framework aims to pro…
New method shows random, diverse initializations are not essential for deep neural networks.
problem The necessity of random, diverse initializations in deep neural networks.
method Constructed a deep convolutional network with identical features by initializing weights to 0, enabling signal propagation and stable gradients.
result Random, diverse initializations are not necessary for training neural networks.
Random small feature subsets outperform FS in diverse datasets.
problem The significance of selected features in high-dimensional datasets is questionable.
method Analysis of 28 diverse datasets (microarray, RNA-Seq, etc.).
result Any arbitrary set of features performs as well as or better than selected features across datasets.
HRF enhances tree diversity in random forests to improve performance.
problem Selection bias and lack of diversity in random forests.
method Introducing heterogeneity during tree construction by assigning lower weights to features used in previous trees.
result HRF outperforms other ensemble methods in accuracy across 52 datasets.
We propose a novel diverse feature selection method based on determinantal point processes (DPPs). Our model enables one to flexibly define diversity based on the covariance of features (similar to orthogonal matching pursuit) or alternatively based on side information. We introduce our approach in the context of Bayes…
Simplifies forecast combination by using diversity of out-of-sample forecasts.
problem Estimating optimal weights for forecast combinations is challenging.
method Use out-of-sample forecasts to extract features and calculate weights for forecast combination.
result Achieves superior forecasting performance in point forecasts and prediction intervals.
Enhances model OOD detection by diversifying feature representation.
problem Limited diversity in model ensembles for OOD detection.
method Integrates diverse training tasks into model ensembles to generate distinct feature representations.
result Superior performance of Multi-Comprehension Ensemble in OOD detection.
Introduce a variance-weighted batch distribution for diverse sampling in diffusion models.
problem Independent sampling in diffusion models.
method Introduce a variance-weighted batch distribution.
result Sampler with a transparent probabilistic target.
Study shows diverse data sources improve cryptocurrency forecasting models.
problem Improving cryptocurrency market forecasting accuracy.
method Integrating various data types, including on-chain metrics, traditional indices, and macroeconomic indicators.
result Data source diversity significantly enhances forecasting model performance.
The two key players in Generative Adversarial Networks (GANs), the discriminator and generator, are usually parameterized as deep neural networks (DNNs). On many generative tasks, GANs achieve state-of-the-art performance but are often unstable to train and sometimes miss modes. A typical failure mode is the collapse o…
New algorithms for model selection in linear contextual bandits without feature diversity conditions.
problem Model selection in linear contextual bandits without feature diversity conditions.
method Data-adaptive algorithms that provide model selection guarantees without feature diversity conditions.
result O(d^α T^{1-α}) model selection guarantees with no feature diversity conditions.
BOP-Elites uses Bayesian Optimisation for QD search, improving efficiency and insight.
problem Finding diverse high-performing points from an objective function.
method Bayesian Optimisation and Gaussian Processes to model quality and diversity.
result Significantly more sample efficient and better at identifying niche solutions.
Supporting evidence for adaptive feature program across diverse models.
problem Analyzing feature learning in neural networks.
method Over-parameterized sequence models and feature error measure (FEM).
result FEM is decreasing during training of adaptive feature models.
MxPool learns graph features from diverse graphs using a hierarchical structure.
problem Learning graph features from diverse graphs with varying properties and sizes.
method MxPool uses a multiplex structure with multiple graph convolution/pooling networks in a hierarchical learning structure.
result MxPool outperforms state-of-the-art methods on graph classification benchmarks.
Prediction tasks over nodes and edges in networks require careful effort in engineering features used by learning algorithms. Recent research in the broader field of representation learning has led to significant progress in automating prediction by learning the features themselves. However, present feature learning ap…
Jigsaw-VAE tackles feature imbalance in VAE latent variables, improving generalization across environments.
problem Feature imbalance in VAE latent variables leads to poor generalization and biased sample generation.
method Proposes a regularization scheme to balance features in VAE latent variables and introduces a metric to measure balance.
result The regularization scheme substantially addresses feature imbalance, leading to improved generalization and diverse sample generation.
Presents STRIPE model for probabilistic forecasting of non-stationary time series.
problem Probabilistic forecasting of non-stationary time series.
method STRIPE model representing structured diversity based on shape and time features, with diversification mechanism using determinantal point processes (DPP).
result STRIPE significantly outperforms baseline methods for representing diversity while maintaining forecasting accuracy.
There are many problems in machine learning and data mining which are equivalent to selecting a non-redundant, high "quality" set of objects. Recommender systems, feature selection, and data summarization are among many applications of this. In this paper, we consider this problem as an optimization problem that seeks …
GANs mode collapse solved with Bures distance.
problem GANs mode collapse or mode dropping.
method Use Bures distance to match real and fake batch diversity in feature space.
result Diversity matching reduces mode collapse and improves sample quality.
MAP-Elites generates diverse trading strategies for improved execution performance.
problem Optimizing trading execution schedules in volatile market conditions.
method Quality-diversity algorithm (MAP-Elites) generating a portfolio of specialized strategies.
result Diverse strategies achieve 8-10% performance improvements, validating quality-diversity methods.
Kernel sparsity ("dying ReLUs") and lack of diversity are commonly observed in CNN kernels, which decreases model capacity. Drawing inspiration from information theory and wireless communications, we demonstrate the intersection of coding theory and deep learning through the Grassmannian subspace packing problem in CNN…
New statistical guarantees for transfer learning with diverse tasks.
problem Statistical guarantees for transfer learning across different tasks.
method Formal analysis of t+1 tasks with shared representation. result Sample complexity reduction for new tasks using shared representation.
AutoFS combines trainers to improve feature selection efficiency and effectiveness.
problem Balancing feature selection efficiency and effectiveness.
method Interactive Reinforced Feature Selection (IRFS) framework with diverse trainers.
result Improved feature selection efficiency and effectiveness compared to existing methods.
The explosion of time series data in recent years has brought a flourish of new time series analysis methods, for forecasting, clustering, classification and other tasks. The evaluation of these new methods requires either collecting or simulating a diverse set of time series benchmarking data to enable reliable compar…
DivDis learns diverse hypotheses from underspecified data to improve robustness.
problem Learning from underspecified datasets leads to multiple equally viable solutions, causing out-of-distribution issues.
method DivDis framework: 1) learns diverse hypotheses using unlabeled test data, 2) selects one hypothesis with minimal additional supervision.
result DivDis finds robust features in image and natural language processing problems.
Gradient descent can efficiently learn a target function with diverse and near-orthogonal features.
problem Learning a target function with additive structure and diverse features.
method Gradient descent training of a two-layer neural network.
result A large subset of polynomial target functions can be efficiently learned.
New framework learns sufficient invariant features robustly across distribution shifts.
problem Learning robust models under distribution shifts between training and test datasets.
method Sufficient Invariant Learning (SIL) framework and Adaptive Sharpness-aware Group Distributionally Robust Optimization (ASGDRO) algorithm.
result Empirical evaluations confirm ASGDRO's robustness against distribution shifts.
Our work proves CSF can recover ground-truth features in RL, improving understanding of feature learning.
problem Understanding the role of representation and mutual information in reinforcement learning.
method Investigates Contrastive Successor Features (CSF) method for identifiable representation learning in reinforcement learning.
result Proves CSF can recover ground-truth features up to a linear transformation.
Emotion recognition from speech is one of the key steps towards emotional intelligence in advanced human-machine interaction. Identifying emotions in human speech requires learning features that are robust and discriminative across diverse domains that differ in terms of language, spontaneity of speech, recording condi…
We propose MAD-GAN, an intuitive generalization to the Generative Adversarial Networks (GANs) and its conditional variants to address the well known problem of mode collapse. First, MAD-GAN is a multi-agent GAN architecture incorporating multiple generators and one discriminator. Second, to enforce that different gener…
Proposes optimizing neural network ensemble diversity in feature space.
problem Improving diversity in neural network ensembles while maintaining performance.
method Optimizes particles in the feature space of a specific intermediate layer.
result Significantly outperforms Deep Ensembles on various metrics.
DVERGE diversifies adversarial vulnerabilities to enhance robust ensemble models.
problem Diverse adversarial vulnerabilities for robust ensemble models.
method Isolates and diversifies adversarial vulnerabilities through distillation and training.
result Achieves higher robustness against transfer attacks compared to previous methods.
Quality-Diversity algorithms explore multiple high-performing solutions in a search space.
problem Finding multiple high-performing solutions in complex optimization problems.
method Evolutionary computation approach focusing on behavioral space and holistic solution distribution.
result Quality-Diversity algorithms provide a comprehensive view of high-performing solutions in a search space.
A new method finds diverse near-optimal portfolios using quality-diversity.
problem Optimizing financial portfolios with robustness to input parameter uncertainties.
method Quality-Diversity (QD) optimization using CVT-MAP-Elites algorithm.
result Diverse set of near-optimal portfolios identified.
NegToMe uses images to guide text-based models away from unwanted visual elements.
problem Insufficient text-based adversarial guidance for complex visual concepts.
method Negative token merging (NegToMe) using visual features from reference images.
result Significantly enhances output diversity and reduces visual similarity to copyrighted content.
We propose a single neural probabilistic model based on variational autoencoder that can be conditioned on an arbitrary subset of observed features and then sample the remaining features in "one shot". The features may be both real-valued and categorical. Training of the model is performed by stochastic variational Bay…
The major challenge of learning from multi-label data has arisen from the overwhelming size of label space which makes this problem NP-hard. This problem can be alleviated by gradually involving easy to hard tags into the learning process. Besides, the utilization of a diversity maintenance approach avoids overfitting …
In an ordinary feature selection procedure, a set of important features is obtained by solving an optimization problem such as the Lasso regression problem, and we expect that the obtained features explain the data well. In this study, instead of the single optimal solution, we consider finding a set of diverse yet nea…
Optimizes forecast accuracy and diversity using multi-task deep learning.
problem Forecasting combinations of time series data.
method Multi-task deep learning architecture that selects and combines forecasting models.
result Enhances point forecast accuracy compared to state-of-the-art methods.
We introduce a method called multi-scale local shape analysis, or MLSA, for extracting features that describe the local structure of points within a dataset. The method uses both geometric and topological features at multiple levels of granularity to capture diverse types of local information for subsequent machine lea…
Proposes MOC method for better counterfactual explanations in ML models.
problem Difficulties in balancing multiple objectives for counterfactual explanations.
method Translates counterfactual search into a multi-objective optimization problem.
result Returns diverse counterfactuals with different trade-offs and maintains feature diversity.
Enhances student diversity in collaborative learning.
problem Student homogenization in large groups.
method Random routing, diverse feature sets, and random subgroup imitation.
result Significantly outperforms state-of-the-art approaches.
Framework reduces simplicity bias in NNs, improving OOD generalization and robustness.
problem Simplicity bias in deep learning models leads to biased predictions and poor OOD generalization.
method Proposes a framework that regularizes conditional mutual information to encourage use of diverse features.
result Demonstrates effectiveness in various settings, enhancing OOD generalization and robustness.
Study breaks down graphs into structural and featural components for task-agnostic data valuation.
problem Lack of methods to assess the value of graphs in data marketplaces.
method Introduces blind message passing framework to evaluate graphs without specific task metrics.
result Demonstrates effectiveness in capturing structural disparities, relevance, and diversity of seller data for buyers.
New distributions allow greedy arm selection in sparse bandit problems.
problem Sparse contextual bandit problem with sparse parameters and feature distributions.
method Introduced new distribution classes and demonstrated that mixtures of these distributions are also greedy-applicable.
result Greedy algorithm applicable to a wider range of arm feature distributions, including those with origin-asymmetric support.