A fast method estimates stability of ensemble feature selectors.
problem Improving stability of ensemble feature selectors for better prediction.
method Simulator of a feature selector to estimate stability.
result Reduces computation time for estimating stability.
Theory and method for reducing prediction variance in noisy feature-subsampled ridge ensembles.
problem Reduction of prediction variance in noisy data with feature bagging.
method Developed analytical learning curves for noisy ridge ensembles, introduced heterogeneous feature ensembling.
result Subsampling shifts the double-descent peak, leading to improved performance over a single linear predictor.
ControlBurn selects few features from tree ensembles for better model interpretability.
problem Improving model interpretability in machine learning models.
method Sparse tree ensembles with lasso optimization.
result ControlBurn selects feature-sparse subsets for better model interpretability.
Ensembles of random-feature models can't outperform a single large model.
problem Finding the optimal balance between model size and ensemble size.
method Deterministic equivalent risk estimates and scaling laws analysis.
result Ensembles of random-feature models achieve near-optimal performance only under specific conditions.
Enhances model OOD detection by diversifying feature representation.
problem Limited diversity in model ensembles for OOD detection.
method Integrates diverse training tasks into model ensembles to generate distinct feature representations.
result Superior performance of Multi-Comprehension Ensemble in OOD detection.
We consider the problem of learning decision rules for prediction with feature budget constraint. In particular, we are interested in pruning an ensemble of decision trees to reduce expected feature cost while maintaining high prediction accuracy for any test example. We propose a novel 0-1 integer program formulation …
Proposes optimizing neural network ensemble diversity in feature space.
problem Improving diversity in neural network ensembles while maintaining performance.
method Optimizes particles in the feature space of a specific intermediate layer.
result Significantly outperforms Deep Ensembles on various metrics.
Study ridge ensembles in proportional feature-to-sample size regime, proving risk equivalence and GCV consistency.
problem Characterizing and optimizing ridge ensembles in proportional feature-to-sample size regimes.
method Proportional asymptotics analysis, GCV for tuning, proving risk equivalence.
result Risk of optimal full ridgeless ensemble matches optimal ridge predictor's risk.
Optimizes feature shifts for tree ensemble reclassification.
problem Improving tree ensemble reclassification accuracy through feature perturbation.
method Mathematical optimization of feature shifts to maximize reclassification probability.
result Effective feature ranking and optimization for tree ensemble reclassification.
Study compares single vs ensemble feature selection for cancer diagnosis.
problem Identifying relevant variables for cancer diagnosis and prognosis.
method Comparison of single feature selection algorithms and ensemble of diverse algorithms.
result Ensemble approach did not improve predictive performance over individual algorithms.
Ensembles dynamic models using random feature approximations.
problem Online scalable Bayesian learning with dynamic models and ensembling.
method Random feature approximations and dynamic models using random walks.
result Better performance with alternative basis expansions like Hilbert space Gaussian processes.
Tree ensemble kernels improve Bayesian optimization for mixed features and constraints.
problem Optimizing over mixed-feature spaces with known constraints.
method Kernel interpretation of tree ensembles as Gaussian Process prior, compatible optimization formulation for acquisition function, integration of known constraints.
result Framework outperforms state-of-the-art methods for mixed-feature spaces and constraints.
Proposes simplified SHAP for faster black-box model explanations.
problem Computational expense of SHAP for models with many features.
method Ensemble of random SHAPs with feature selection and point generation.
result Efficiency and properties demonstrated through numerical experiments.
Ensembles of decision trees perform well on many problems, but are not interpretable. In contrast to existing approaches in interpretability that focus on explaining relationships between features and predictions, we propose an alternative approach to interpret tree ensemble classifiers by surfacing representative poin…
Overparameterized ensembles don't offer generalization benefits over single large models.
problem Theoretical limitations of ensembles in overparameterized settings.
method Using ensembles of random feature (RF) regressors, the paper clarifies how modern ensembles differ from underparameterized counterparts.
result Infinite ensembles of overparameterized RF regressors become pointwise equivalent to single infinite-width RF regressors, and finite width ensembles converge to single models with the same parameter budget.
Proposes an ensemble framework for outlier detection with embedded feature selection.
problem Improving performance of outlier detection in noisy data.
method Unified feature selection and outlier detection, thresholded self-paced learning, alternate algorithm.
result The proposed framework ODEFS outperforms existing methods on real-world datasets.
Combining feature importance estimates improves reliability of machine learning predictions.
problem Lack of consensus on feature importance quantification makes explanations unreliable.
method Proposes a feature importance fusion framework combining multiple quantifiers.
result Feature importance ensembles reduce prediction error by 15%.
Ensemble models provide more accurate feature importance estimates than single models.
problem Inaccurate variable importance estimates due to model instability and stochasticity.
method Theoretical analysis and validation on benchmarks and real data.
result Ensembling at the model level reduces excess risk and provides more accurate variable-importance estimates.
Ensemble learning that can be used to combine the predictions from multiple learners has been widely applied in pattern recognition, and has been reported to be more robust and accurate than the individual learners. This ensemble logic has recently also been more applied in feature selection. There are basically two st…
This work proposes a robust ensemble method for decision trees that resists adversarial attacks.
problem Adversarial attacks on machine learning models, especially decision trees.
method Feature partitioning to train robust ensembles and approximate certification methods.
result The proposed ensemble method can resist evasion attacks by a majority of its models.
A method for clustering using transfer learning from similar labeled data.
problem Clustering with datasets having different features and labeled data.
method Constructing meta-features to describe structural characteristics of data and transferring them between source and target domains.
result The method is efficient and works under arbitrary feature descriptions of source and target domains with smaller complexity.
Improved neural population modeling using shared features and ensemble detection.
problem Missing shared coding properties in neural latent variable models.
method Feature sharing across tuning curves and soft clustering of neurons.
result More interpretable and better-performing neural population models.
HRF enhances tree diversity in random forests to improve performance.
problem Selection bias and lack of diversity in random forests.
method Introducing heterogeneity during tree construction by assigning lower weights to features used in previous trees.
result HRF outperforms other ensemble methods in accuracy across 52 datasets.
Ensemble methods that average over a collection of independent predictors that are each limited to a subsampling of both the examples and features of the training data command a significant presence in machine learning, such as the ever-popular random forest, yet the nature of the subsampling effect, particularly of th…
In this paper, we propose a hierarchical feature-aware tracking framework for efficient visual tracking. Recent years, ensembled trackers which combine multiple component trackers have achieved impressive performance. In ensembled trackers, the decision of results is usually a post-event process, i.e., tracking result …
A new method for feature importance inference without data splitting.
problem Feature importance inference for machine learning models.
method Minipatch ensembles for model-agnostic, distribution-free inference.
result Asymptotic validity of confidence intervals without data splitting.
Very few K-nearest-neighbor (KNN) ensembles exist, despite the efficacy of this approach in regression, classification, and outlier detection. Those that do exist focus on bagging features, rather than varying k or bagging observations; it is unknown whether varying k or bagging observations can improve prediction. Giv…
Two proteins are homologous if they have a common evolutionary origin, and the binary classification problem is to identify proteins in a candidate set that are homologous to a particular native protein. The feature (explanatory) variables available for classification are various measures of similarity of proteins. The…
We describe the 1st place winning approach for the CIKM Cup 2016 Challenge. In this paper, we provide an approach to reasonably identify same users across multiple devices based on browsing logs. Our approach regards a candidate ranking problem as pairwise classification and utilizes an unsupervised neural feature ense…
Calibrated ensembles improve both ID and OOD accuracy in distribution shift.
problem Desired balance between in-distribution and out-of-distribution accuracy.
method Ensemble standard and robust models, calibrating on ID data only.
result ID-calibrated ensembles outperform state-of-the-art methods on multiple datasets.
The paper uses machine learning to detect malicious executable files.
problem Detecting malicious executable files using static analysis.
method Pre-processing, cleaning, encoding, feature selection, and ensemble training of classifiers.
result An ensemble of classifiers effectively detects malicious executable files.
Paper presents a machine learning framework for corn yield forecasting.
problem Accurate and timely prediction of corn yields in the US Corn Belt.
method Machine learning ensembles considering complete and partial in-season weather data.
result Ensemble models outperform individual models, achieving best prediction accuracy.
Paper proposes an ensemble approach to improve fairness in classifier decisions.
problem Improving fairness in classifier decisions to prevent bias.
method Inspired by dropout techniques, feature drop-out is used to reduce classifier dependence on sensitive features while maintaining accuracy.
result An ensemble of classifiers with reduced sensitivity to sensitive features and improved accuracy.
Wind speed prediction improved using a novel deep ensemble learning model inspired by jet aerodynamics.
problem Accurate wind speed forecasting for renewable energy production.
method Proposes a novel Deep Ensemble Learning using Jet-like Architecture (DEL-Jet) to enhance robustness and generalization of a learning system.
result The DEL-Jet technique improves the robustness and generalization of the learning system, as shown by performance evaluations.
New framework detects model weaknesses in decision tree ensembles.
problem Detecting feature sensitivity in decision tree ensembles.
method Data-aware sensitivity framework using MILP and SMT.
result Realistic and interpretable examples of model weaknesses.
Efficient facial feature learning with shared representations reduces redundancy and improves accuracy.
problem Redundancy and high computational load in training deep ensemble models.
method Wide Ensemble-based Convolutional Neural Networks (ESRs) with varying branching levels.
result ESRs reduce residual generalization error and outperform state-of-the-art methods on facial expression recognition.
Paper improves DNS typo-squatting detection with ensemble model.
problem Detecting and preventing DNS typo-squatting attacks.
method Ensemble-based feature selection and bagging classification model.
result The proposed framework achieves high accuracy and precision in identifying typo-squatting domains.
Ensemble learning is a mainstay in modern data science practice. Conventional ensemble algorithms assigns to base models a set of deterministic, constant model weights that (1) do not fully account for variations in base model accuracy across subgroups, nor (2) provide uncertainty estimates for the ensemble prediction,…
DRE combines DNN with random feature regression for efficient neural network design.
problem Designing and training deep neural networks (DNN) efficiently and effectively.
method DRE architecture with two-layer neural networks, randomly drawn input and output weights trained with linear ridge regression.
result DRE outperforms state-of-the-art DNN in many data sets with lower computational cost.
No feature ranking can be faithful, stable, and complete when features are collinear.
problem The impossibility of creating a feature ranking that is simultaneously faithful, stable, and complete when features are collinear.
method Proving the impossibility, quantifying it for four model classes, resolving it via ensemble averaging (DASH), and machine-verifying it with Lean 4 theorems.
result No method lies outside the dichotomy of faithful-complete methods (unstable, with rankings that flip up to 50% of the time) and ensemble methods (stable, reporting ties for symmetric features).
New research shows deep models learn sparse features, limiting transfer learning; ensembling improves performance.
problem Sparse feature learning in deep models limits transfer learning performance.
method Developed a theoretical framework and proposed an ensembling strategy to aggregate multiple models.
result Ensembling yields a 9% improvement in transfer accuracy without extra pretraining cost.
DVERGE diversifies adversarial vulnerabilities to enhance robust ensemble models.
problem Diverse adversarial vulnerabilities for robust ensemble models.
method Isolates and diversifies adversarial vulnerabilities through distillation and training.
result Achieves higher robustness against transfer attacks compared to previous methods.
D2KLab's approach predicts tweet engagement using two stages.
problem Predicting user engagement with tweets.
method Two-stage approach: feature learning and ensemble XGBoost.
result Ranked 22 in the 2020 RecSys Challenge leaderboard.
Predicting MRI coil failures using image features and ensemble learning.
problem Ensuring trouble-free operation of MRI systems by predicting hardware failures.
method Two-level data analysis with neural networks and Random Forest ensemble learning.
result Improved prediction results with an F-score of 94.14% and an accuracy of 99.09%.
AGM improves model accuracy through adaptive generation and feature augmentation.
problem Improving model accuracy in machine learning competitions.
method Adaptive Generation Model (AGM) using stacked ensemble learning with horizontal and vertical model expansion and feature augmentation.
result AGM outperforms previous models in 7 data sets.
Classification Ensemble, which uses the weighed polling of outputs, is the art of combining a set of basic classifiers for generating high-performance, robust and more stable results. This study aims to improve the results of identifying the Persian handwritten letters using Error Correcting Output Coding (ECOC) ensemb…
Multi-objective optimization for hyperparameters and features.
problem Optimizing hyperparameters and selecting features for machine learning models.
method Two approaches: model-based optimization and NSGA-II-based wrapper approach using filter ensembles.
result NSGA-II approach is computationally less expensive but may require more evaluations.
This paper tackles robustness of ensemble stumps and trees under general ℓ_p norm perturbations.
problem The vulnerability of ensemble stumps and trees to small input perturbations under the ℓ_∞ norm.
method Developed dynamic programming algorithms for robustness verification and certified defense under general ℓ_p norm perturbations.
result First certified defense method for ensemble stumps and trees under ℓ_p norm perturbations.