Proposes optimizing neural network ensemble diversity in feature space.
problem Improving diversity in neural network ensembles while maintaining performance.
method Optimizes particles in the feature space of a specific intermediate layer.
result Significantly outperforms Deep Ensembles on various metrics.
Improved neural population modeling using shared features and ensemble detection.
problem Missing shared coding properties in neural latent variable models.
method Feature sharing across tuning curves and soft clustering of neurons.
result More interpretable and better-performing neural population models.
Popular interpretability methods often produce inaccurate feature importance estimates.
problem Inaccurate feature importance estimates in deep neural networks.
method Empirical measure of feature importance accuracy across large-scale image classification datasets.
result Only certain ensemble-based methods (VarGrad and SmoothGrad-Squared) outperform random assignment of feature importance.
First place solution for cross-device user matching in online advertising.
problem Identifying same users across multiple devices from browsing logs.
method Pairwise classification using unsupervised neural feature ensemble and supervised classifiers.
result Improved accuracy in cross-device user matching compared to traditional methods.
DRE combines DNN with random feature regression for efficient neural network design.
problem Designing and training deep neural networks (DNN) efficiently and effectively.
method DRE architecture with two-layer neural networks, randomly drawn input and output weights trained with linear ridge regression.
result DRE outperforms state-of-the-art DNN in many data sets with lower computational cost.
Overparameterized ensembles don't offer generalization benefits over single large models.
problem Theoretical limitations of ensembles in overparameterized settings.
method Using ensembles of random feature (RF) regressors, the paper clarifies how modern ensembles differ from underparameterized counterparts.
result Infinite ensembles of overparameterized RF regressors become pointwise equivalent to single infinite-width RF regressors, and finite width ensembles converge to single models with the same parameter budget.
Tree ensemble kernels improve Bayesian optimization for mixed features and constraints.
problem Optimizing over mixed-feature spaces with known constraints.
method Kernel interpretation of tree ensembles as Gaussian Process prior, compatible optimization formulation for acquisition function, integration of known constraints.
result Framework outperforms state-of-the-art methods for mixed-feature spaces and constraints.
IEA improves CNN models by averaging multiple convolutional layers.
problem Improving CNN model accuracy through ensemble learning.
method Replacing single convolutional layers with Inner Average Ensembles (IEA) of multiple convolutional layers.
result CNN models using IEA outperform those with regular convolutional layers.
Paper uses stacking with neural networks to predict cryptocurrency price direction.
problem Predicting the direction of cryptocurrency prices.
method Generative and discriminative classifiers stacked over a one-layer neural network, using technical indicators and sentiment analysis.
result Stacking method outperformed individual models in accuracy.
DVE uses GPs on DNN outputs to provide UQ without retraining.
problem Feature collapse in DNNs affects UQ methods.
method Deep Vecchia ensemble (DVE) of GPs on DNN hidden layers.
result Deterministic UQ possible in feature-collapsed DNNs.
LIFE framework improves model accuracy and interpretability.
problem Achieving high prediction accuracy and interpretability in neural networks.
method Three-step process: subset definition, feature creation, and linear model combination.
result LIFE consistently outperforms other models in prediction accuracy and interpretability.
Paper proposes an ensemble algorithm for neural networks to improve test accuracy.
problem Overfitting and lack of interpretability in deep neural networks.
method Analyzes paths between clusters in hidden spaces to extract features, then presents an ensemble algorithm with test accuracy guarantees.
result State-of-the-art results for Wide-ResNets on CIFAR-10 and improved test accuracy for all models.
Predicting MRI coil failures using image features and ensemble learning.
problem Ensuring trouble-free operation of MRI systems by predicting hardware failures.
method Two-level data analysis with neural networks and Random Forest ensemble learning.
result Improved prediction results with an F-score of 94.14% and an accuracy of 99.09%.
Spatial smoothing improves BNNs' accuracy, uncertainty, and robustness without increasing computational cost.
problem Large ensembles in BNNs increase computational cost and reduce performance.
method Spatial smoothing adds blur layers to convolutional neural networks to ensemble neighboring feature map points.
result Spatial smoothing improves BNNs' performance with fewer ensembles and enhances robustness.
Ensemble++ uses shared-factor ensembles to scale Thompson Sampling for linear and nonlinear bandits.
problem Computational challenges in Thompson Sampling for large-scale or non-conjugate settings.
method Ensemble++ with shared-factor architecture and random linear combinations.
result Ensemble++ achieves comparable regret to exact Thompson Sampling with significantly smaller ensemble sizes.
Efficient facial feature learning with shared representations reduces redundancy and improves accuracy.
problem Redundancy and high computational load in training deep ensemble models.
method Wide Ensemble-based Convolutional Neural Networks (ESRs) with varying branching levels.
result ESRs reduce residual generalization error and outperform state-of-the-art methods on facial expression recognition.
Proposes a deep neural network for early disk drive failure prediction.
problem Early prediction of disk drive failure using multivariate time series sensor data.
method Enriched features derived from sensor data through transformations, combined with ensemble learning and deep neural network architecture.
result Significantly improved classification accuracy in predicting disk drive failure.
Ensemble methods improve model performance by averaging over subsampled predictors.
problem Understanding the effect of feature subsampling in ensemble methods.
method Fit linear predictors using ordinary least squares on random submatrices of the data matrix.
result The asymptotic risk of an ensemble is equal to the ridge regression risk, optimal for linear predictors.
FoRDE uses input gradients to improve neural network ensembles.
problem Improving neural network ensembles for robustness and accuracy.
method Proposes FoRDE, an ensemble learning method based on ParVI, which repels function space by input gradients.
result FoRDE significantly outperforms DEs and other ensemble methods in accuracy and calibration.
Wind speed prediction improved using a novel deep ensemble learning model inspired by jet aerodynamics.
problem Accurate wind speed forecasting for renewable energy production.
method Proposes a novel Deep Ensemble Learning using Jet-like Architecture (DEL-Jet) to enhance robustness and generalization of a learning system.
result The DEL-Jet technique improves the robustness and generalization of the learning system, as shown by performance evaluations.
Deep CNNs improve technical forecasting accuracy.
problem Improving financial forecasting accuracy using machine learning.
method Reframed technical analysis as feature-extractive layer in CNNs, optimizing over different resolutions.
result Ensemble of shallow, thresholded CNNs outperforms technical methods.
A fast method estimates stability of ensemble feature selectors.
problem Improving stability of ensemble feature selectors for better prediction.
method Simulator of a feature selector to estimate stability.
result Reduces computation time for estimating stability.
Unified framework for ensemble sampling in nonlinear contextual bandits with provable regret bounds.
problem Efficient exploration in nonlinear contextual bandits with unknown feature dimensions.
method Developed GLM-ES and Neural-ES for generalized linear and neural contextual bandits, respectively, using maximum likelihood estimation on randomly perturbed data.
result Unified high-probability frequentist regret bounds for GLM-ES and Neural-ES, matching state-of-the-art results.
A method interprets black-box models using an ensemble of gradient boosting machines.
problem Local and global interpretation of black-box models.
method An ensemble of gradient boosting machines (GBMs) to form a generalized additive model.
result Efficiency and properties demonstrated on synthetic and real datasets.
Hydra distills ensemble models into a single model while preserving diversity and uncertainty.
problem Loss of ensemble diversity and uncertainty in distilled models.
method Single multi-headed neural network with shared body network.
result Hydra improves distillation performance and preserves ensemble diversity and uncertainty.
LEC prevents deep nets from memorizing noisy examples.
problem Deep nets overfit to noisy data.
method LEC removes noisy examples based on an ensemble of perturbed networks.
result LTEC outperforms state-of-the-art on noisy MNIST, CIFAR-10, and CIFAR-100.
An ensemble of randomized NNs improves time series forecasting accuracy.
problem Forecasting time series with multiple seasonality and nonstationarity.
method Randomized neural networks with pattern-based time series representation and diversity control strategies.
result Outperforms statistical and machine learning models in forecasting accuracy.
Theory and method for reducing prediction variance in noisy feature-subsampled ridge ensembles.
problem Reduction of prediction variance in noisy data with feature bagging.
method Developed analytical learning curves for noisy ridge ensembles, introduced heterogeneous feature ensembling.
result Subsampling shifts the double-descent peak, leading to improved performance over a single linear predictor.
Neural networks favor simple features over complex ones, even when complex features are available.
problem Neural networks exhibit a bias towards simple features over complex ones, even when complex features are present.
method Rigorously defined simplicity bias, theoretical and empirical demonstrations, ensemble approach to improve robustness.
result One hidden layer neural networks favor simple features over complex ones, even in the presence of more robust features.
Improved anti-cancer drug sensitivity prediction using REFINED CNN ensemble learning.
problem Challenges in predicting anti-cancer drug sensitivity for individual cell lines.
method Using REFINED CNN, which represents high-dimensional vectors as compact 2D images with spatial correlations, and building ensembles of these models.
result Ensemble approaches significantly improve drug sensitivity prediction performance compared to single models.
Deep learning ensemble improves Alzheimer's disease classification accuracy.
problem Improving diagnostic accuracy for Alzheimer's disease.
method Proposes a deep ensemble learning framework integrating multisource data and expert wisdom.
result 4% improvement in classification accuracy compared to existing methods.
Deep Super Learner combines traditional machine learning algorithms in a hierarchical structure for improved performance.
problem Improving performance of traditional machine learning algorithms using ensemble methods.
method Deep Super Learner combines traditional machine learning algorithms in a hierarchical structure.
result Deep Super Learner achieves log loss and accuracy results competitive to deep neural networks.
Radar-based classification improves accuracy of autonomous driving by identifying new classes.
problem Challenges in classifying sparse radar data for autonomous driving.
method Ensemble of classifiers using one-vs-one and one-vs-all strategies, with feature selection.
result Improved classification performance and identification of novel classes.
Paper proposes an ensemble approach to improve fairness in classifier decisions.
problem Improving fairness in classifier decisions to prevent bias.
method Inspired by dropout techniques, feature drop-out is used to reduce classifier dependence on sensitive features while maintaining accuracy.
result An ensemble of classifiers with reduced sensitivity to sensitive features and improved accuracy.
Study shows how feature weighting affects neural network regularization.
problem Understanding how feature weighting influences neural network regularization.
method Derived equivalence paths connecting different weighting matrices and ridge regularization levels.
result Ridge estimators trained on weighted features are asymptotically equivalent when evaluated against test vectors.
DropIn method makes Reservoir Computing robust to missing inputs.
problem Robustness to missing input features in Reservoir Computing models.
method Ensembling Dropout regularization to train a committee of subnetworks.
result Maintains comparable predictive performance with missing inputs.
New CMCL method reduces errors in deep neural network tasks.
problem Overconfidence in ensemble methods for deep neural networks.
method Confident Multiple Choice Learning (CMCL) with new loss, architecture, and training method.
result CMCL reduces top-1 error rates by 14.05% on CIFAR and 6.60% on SVHN.
A new method selects multiple activation functions at each layer of a neural network.
problem Selecting an adequate activation function requires trial and error.
method Activation Ensembles: introduces additional variables α to allow for multiple activation functions at each neuron. result Achieves superior results compared to traditional techniques.
A deep learning subsampling technique improves modulation classification accuracy.
problem Improving modulation classification accuracy in wireless communication systems.
method Proposes a data-driven subsampling strategy using deep neural networks to simulate signal removal.
result Improves classification accuracy to higher levels than traditional methods.
For2For combines forecasts to improve time series forecasting.
problem Improving time series forecasting accuracy.
method Combines standard forecasting methods and machine learning models using forecasts as features.
result Outperforms all submissions in the M4 competition for quarterly series and most monthly series.
Hybrid ML ensemble predicts market risk and generates alpha.
problem Forecasting short-horizon market risk and generating alpha.
method Neural networks and tree-based voting models integrated for trading signal.
result Sharpe ratio of 2.51 and annualized CAPM alpha of +0.28.
Optimizes decision tree ensembles with feature cost constraints.
problem Learning decision rules with a budget on feature usage.
method 0-1 integer program formulation for pruning ensembles, considering feature-sharing and accuracy/cost trade-offs.
result Significant reduction in feature cost with high prediction accuracy.
This work uses decision trees to encode relevant features and their interactions into neural networks, improving model performance.
problem Overfitting in neural networks with many irrelevant variables.
method Defines a mapping to encode decision tree extracted relationships into a neural network.
result The approach outperforms fully connected neural networks and tree-based methods.
forgeNet uses a tree-based ensemble to learn feature graphs for deep learning in omics data.
problem Small sample size vs. large feature space in omics data.
method forgeNet integrates a forest feature graph extractor with a GEDFN architecture.
result ForgeNet achieves high classification accuracy on synthetic and real datasets.
Study loop corrections in random feature models affecting training and test errors.
problem Analyzing loop corrections in random feature models to understand training and test errors.
method Statistical physics and effective field theory approach to study loop corrections.
result Derived loop corrections to training error, test error, and generalization gap.
The paper classifies market states to predict trading strategies, outperforming traditional methods.
problem Directly predicting prices or returns is unreliable; classifying market states is a better approach.
method Classify market states using various labels and features, then combine probabilities from neural networks.
result Trading strategy ensembles outperform traditional methods in returns and risk-adjusted returns.
New ensemble methods improve time series forecasting accuracy.
problem Global Forecasting Models (GFM) lack localisation for heterogeneous datasets.
method Ensemble techniques with clustering and varied GFM models.
result Significantly higher accuracy achieved compared to baseline models.
Representation learning, especially which by using deep learning, has been widely applied in classification. However, how to use limited size of labeled data to achieve good classification performance with deep neural network, and how can the learned features further improve classification remain indefinite. In this pa…