New framework learns interpretable rule ensembles without sacrificing accuracy.
problem Trade-off between accuracy and interpretability in rule ensembles.
method Introduces local interpretability and a regularizer to promote it, using coordinate descent with local search.
result Learns rule ensembles with fewer rules to explain individual predictions, maintaining comparable accuracy.
In this article supervised learning problems are solved using soft rule ensembles. We first review the importance sampling learning ensembles (ISLE) approach that is useful for generating hard rules. The soft rules are then obtained with logistic regression from the corresponding hard rules. In order to deal with the p…
Ensemble methods for supervised machine learning have become popular due to their ability to accurately predict class labels with groups of simple, lightweight "base learners." While ensembles offer computationally efficient models that have good predictive capability they tend to be large and offer little insight into…
FIRE extracts interpretable rules from tree ensembles.
problem Building sparse and interpretable rule sets from tree ensembles.
method Optimization-based framework with fusion regularization and sparsity-inducing penalties.
result FIRE outperforms state-of-the-art rule ensemble algorithms.
In this article, the logic rule ensembles approach to supervised learning is applied to the unsupervised or semi-supervised clustering. Logic rules which were obtained by combining simple conjunctive rules are used to partition the input space and an ensemble of these rules is used to define a similarity matrix. Simila…
LIBRE learns interpretable Boolean rules from data.
problem Creating interpretable classifiers in imbalanced settings.
method Ensemble of weak learners on random feature subsets, combined with a simple union.
result Efficiently balances prediction accuracy and interpretability.
This paper improves prediction rule ensembles using model-based data generation.
problem Improving the sparsity and predictive accuracy of prediction rule ensembles.
method The authors use surrogate models to train Lasso regression with data generated by a boosted decision tree ensemble, improving PRE performance.
result The use of surrogacy models can substantially improve the sparsity of PRE while retaining predictive accuracy.
A new ensemble method using random projections for kNN classification.
problem Improving kNN classification accuracy through ensemble methods.
method Random projection of bootstrap samples into lower dimensions, using extended neighbourhood rule for base learners.
result Enhanced classification accuracy compared to traditional kNN and other ensembles.
We propose a method to extract interpretable rules from tree ensembles.
problem Tree ensembles are accurate but hard to interpret.
method Propose an estimator to extract compact sets of decision rules from tree ensembles.
result Our estimator improves accuracy and reveals useful relationships in the data.
The paper proposes a method to optimize rule-based models for better accuracy and interpretability.
problem Developing rule-based models for regression and classification with better accuracy and interpretability.
method Column generation to optimize over an exponentially large space of rules, using integer programming or a heuristic.
result The proposed methods achieve better accuracy-complexity trade-offs than existing rule ensemble algorithms.
Fed-FEARE model extracts rules from multiple agencies' data securely.
problem Data privacy and secure rule extraction across multiple agencies.
method Federated F-score based ensemble tree model.
result Model performance measures significantly improved with federated learning.
A new ensemble method improves kNN performance by extending the neighborhood rule.
problem Traditional kNN's limitations when test points are outside the spherical region and ensemble's high errors.
method Determines neighbors in k steps, using bootstrap samples and optimal models selection.
result The proposed ensemble method outperforms state-of-the-art methods on 17 benchmark datasets.
A new stopping rule based on E-values helps efficiently use sampling in Bayesian Deep Ensembles.
problem How long should sampling continue in Bayesian Deep Ensembles to yield significant improvements?
method Formulated as a sequential anytime-valid hypothesis test, using E-values to decide when to stop sampling.
result Only a fraction of the full-chain budget is often required for significant improvements.
Tree ensembles such as random forests and boosted trees are accurate but difficult to understand, debug and deploy. In this work, we provide the inTrees (interpretable trees) framework that extracts, measures, prunes and selects rules from a tree ensemble, and calculates frequent variable interactions. An rule-based le…
A new stochastic method handles ensemble creation with cost constraints.
problem Creating ensembles under cost limitations in decision-making.
method Introducing a novel stochastic approach to solve the knapsack problem.
result The approach efficiently incorporates ensemble accuracy and cost constraints.
SigD2 reduces noisy rules in rule-based classifiers for better accuracy and readability.
problem Redundant and noisy rules in rule-based classifiers reduce model accuracy and readability.
method Two-stage pruning strategy and ensemble methods (bagging and boosting) to reduce noise and improve model performance.
result SigD2 and ACboost ensemble models outperform state-of-the-art classifiers in terms of accuracy and rule count.
Design rule check is a critical step in the physical design of integrated circuits to ensure manufacturability. However, it can be done only after a time-consuming detailed routing procedure, which adds drastically to the time of design iterations. With advanced technology nodes, the outcomes of global routing and deta…
New method improves model explainability and accuracy with low computational cost.
problem Improving model explainability and accuracy in classification models.
method Distributionally robust optimization to learn sparse ensembles of rule sets.
result Improves model performance on various metrics compared to competing methods.
CRE discovers interpretable subgroups with heterogeneous treatment effects.
problem Identifying subgroups with notable treatment effect heterogeneity.
method Causal Rule Ensemble (CRE) using an ensemble-of-trees approach.
result CRE offers interpretable decision rules and high stability in subgroup discovery.
Improving Bayesian filtering with strictly proper scoring rules
problem Bayesian filtering of partially and noisily observed dynamical systems
method Proper scoring ensemble filter (PSEF)
result Accurate approximation of challenging filtering distributions
In this article, we have proposed several approaches for post processing a large ensemble of prediction models or rules. The results from our simulations show that the post processing methods we have considered here are promising. We have used the techniques developed here for estimation of quantitative traits from mar…
Machine learning ensemble improves accuracy by considering minority answers as more likely true.
problem Ensemble methods often rely on majority voting, which can fail when the majority is wrong.
method Proposes Bayesian Truth Serum for classification problems, detecting surprising majority answers.
result Better classification performance achieved by considering minority answers as more likely true.
The paper develops a stationary-distribution theory for Random Forest ensemble size selection.
problem Determining the optimal number of trees in Random Forests.
method Modeling the ensemble size as a birth-death Markov chain and deriving its stationary distribution.
result The stationary ensemble size B∗ scales as O(ε−2) as ε↓0. Two algorithms for interpreting and boosting tree-based models using rule covering.
problem Interpreting and boosting tree-based ensemble methods.
method Mathematical programming models constructed from decision tree rules.
result Selects a few rules that closely match the accuracy of the model.
We propose a new Bayesian model for flexible nonlinear regression and classification using tree ensembles. The model is based on the RuleFit approach in Friedman and Popescu (2008) where rules from decision trees and linear terms are used in a L1-regularized regression. We modify RuleFit by replacing the L1-regularizat…
A new gradient boosting method improves interpretability of probabilistic models.
problem Learning interpretable yet accurate probabilistic models with limited rule complexity.
method A new objective function that measures the angle between risk gradient and condition output vector projection.
result Significantly improves comprehensibility/accuracy trade-off of fitted ensemble.
The number of component classifiers chosen for an ensemble greatly impacts the prediction ability. In this paper, we use a geometric framework for a priori determining the ensemble size, which is applicable to most of existing batch and online ensemble classifiers. There are only a limited number of studies on the ense…
Proposes an interpretable machine learning framework for multi-arm HTE estimation.
problem Challenges in estimating heterogeneous treatment effects in multi-arm settings.
method Rule-based ensemble approach for HTE estimation in multi-arm trials.
result Achieved lower bias and higher estimation accuracy compared to existing methods.
Oblique BART improves tree-based predictions.
problem Axis-aligned decision rules in BART can be suboptimal.
method Developed an oblique version of BART using data-adaptive hyperplane partitions.
result Oblique BART outperformed axis-aligned BART and other tree methods on benchmarks.
Improved Bayesian inference for neuronal ensemble inference reduces computational cost.
problem Efficient inference of neuronal ensembles from activity data.
method Modified MCMC algorithm with simulated annealing for hyperparameter control.
result Our method reduces computational cost while maintaining or improving inference accuracy.
A new method predicts precipitation distributions from ensemble forecasts.
problem Improving accuracy and calibration of precipitation forecasts.
method Distributional regression U-Nets for postprocessing ensemble precipitation forecasts.
result Competitive performance in continuous ranked probability score, especially for heavy precipitation.
The paper explores dynamic ensembles for multi-step forecasting.
problem Lack of research on dynamic ensembles for multi-step forecasting.
method Extensive experiments with 3568 time series and an ensemble of 30 multi-output models.
result Dynamic ensembles based on arbitrating and windowing perform best.
Binary classification rules based on covariates typically depend on simple loss functions such as zero-one misclassification. Some cases may require more complex loss functions. For example, individual-level monitoring of HIV-infected individuals on antiretroviral therapy (ART) requires periodic assessment of treatment…
MOSS optimizes decision rules for accuracy and stability.
problem Constructing stable sets of decision rules.
method Multi-objective optimization framework incorporating sparsity, accuracy, and stability.
result MOSS outperforms state-of-the-art rule ensembles in predictive performance and stability.
We consider using an ensemble of binary classifiers for transductive prediction, when unlabeled test data are known in advance. We derive minimax optimal rules for confidence-rated prediction in this setting. By using PAC-Bayes analysis on these rules, we obtain data-dependent performance guarantees without distributio…
Several real problems ranging from text classification to computational biology are characterized by hierarchical multi-label classification tasks. Most of the methods presented in literature focused on tree-structured taxonomies, but only few on taxonomies structured according to a Directed Acyclic Graph (DAG). In thi…
New NIDS uses hypergraphs for real-time detection of evolving port scans.
problem Challenges in offline NIDS for auto-generated port scans.
method Hypergraph-based metrics for real-time ML ensemble NIDS.
result High accuracy, precision, and recall in detecting port scans and attacks.
Repulsive deep ensembles improve diversity and Bayesian inference.
problem Challenges in maintaining diversity among ensemble members trained independently.
method Introducing a repulsive term in the update rule of deep ensembles.
result Training dynamics of repulsive ensembles follow a Wasserstein gradient flow of KL divergence with the true posterior.
A new method creates simpler, more interpretable decision trees from complex ensembles.
problem Complex tree ensembles reduce interpretability and control over machine learning models.
method Dynamic-programming based algorithm for finding a minimum-size decision tree.
result Optimal born-again trees are simpler and more interpretable than original ensembles.
We consider the problem of learning decision rules for prediction with feature budget constraint. In particular, we are interested in pruning an ensemble of decision trees to reduce expected feature cost while maintaining high prediction accuracy for any test example. We propose a novel 0-1 integer program formulation …
The motivation of this work is to improve the performance of standard stacking approaches or ensembles, which are composed of simple, heterogeneous base models, through the integration of the generation and selection stages for regression problems. We propose two extensions to the standard stacking approach. In the fir…
AEA dynamically aggregates ensemble targets for actor-critic learning.
problem Static ensemble aggregation methods struggle with overestimation bias and variance.
method Adaptive Ensemble Aggregation (AEA) dynamically constructs ensemble-based targets.
result AEA converges to optimal variance reduction and maximal Fisher information.
This paper introduces TNTK to study infinite soft tree ensembles.
problem Understanding the behavior of infinite soft tree ensembles.
method Introduced Tree Neural Tangent Kernel (TNTK) to analyze infinite soft tree ensembles.
result Identified several non-trivial properties of infinite soft tree ensembles.
We address the problem of aggregating an ensemble of predictors with known loss bounds in a semi-supervised binary classification setting, to minimize prediction loss incurred on the unlabeled data. We find the minimax optimal predictions for a very general class of loss functions including all convex and many non-conv…
This study examines ensembling of diffusion models for improved generative quality.
problem Improving generative quality with ensembling of score-based diffusion models.
method Investigated ensembling of scores from multiple diffusion models on image and tabular data.
result Ensembling scores generally improves model likelihood and score-matching loss but not perceptual quality metrics.
High predictive performance and ease of use and interpretability are important requirements for the applicability of a computer-aided diagnosis (CAD) to human reading studies. We propose a CAD system specifically designed to be more comprehensible to the radiologist reviewing screening breast MRI studies. Multiparametr…
This study compares multivariate vs univariate machine learning for multi-output regression.
problem When to use multivariate ensemble techniques over separate univariate models.
method Comparative analysis of different multivariate approaches for multi-output regression.
result Multivariate ensemble techniques outperform separate univariate models in simulations.
A new ensemble of Gaussian processes improves active learning efficiency.
problem Efficiently labeling data in high-cost domains like medical imaging.
method Adaptive weighted ensemble of Gaussian processes (EGP) for active learning.
result EGP-based approaches outperform single GP-based active learning methods.