Two algorithms for interpreting and boosting tree-based models using rule covering.
problem Interpreting and boosting tree-based ensemble methods.
method Mathematical programming models constructed from decision tree rules.
result Selects a few rules that closely match the accuracy of the model.
New decision-theoretic characterization separates belief and decision posteriors.
problem Understanding the conditions under which loss-based updating coincides with Bayesian updating.
method Decision-theoretic approach to distinguish belief and decision posteriors.
result Generalized Bayes coincides with ordinary Bayesian updating only if the loss is proportional to negative log-likelihood.
SIRUS creates interpretable rules from random forests for regression.
problem Lack of interpretability in complex machine learning models.
method Random forest with rule extraction for stability and simplicity.
result SIRUS produces stable and interpretable rule sets.
From doctors diagnosing patients to judges setting bail, experts often base their decisions on experience and intuition rather than on statistical models. While understandable, relying on intuition over models has often been found to result in inferior outcomes. Here we present a new method, select-regress-and-round, f…
A new method for decision tree selection in recommendation systems.
problem Feature-based selection of a single tree from an ensemble for dynamic interpretation.
method A multi-armed contextual bandit recommendation framework that trains a system on top of Random Forests to identify the most relevant tree.
result The dynamic method outperforms an independent CART tree and is comparable to Random Forest in predictive performance.
Proposes sparse local and regional counterfactual rules for robust recourses.
problem Challenges in counterfactual explanations, especially stability, synthesis, and implementation.
method Probabilistic framework using Random Forest to derive sparse local and regional counterfactual rules.
result Effective recourses derived from high-density regions, providing sparse and robust counterfactual rules.
ExMatrix visualizes RF models, aiding in decision understanding.
problem Lack of interpretability in complex models like RF.
method ExMatrix, a novel visualization method for RF interpretability.
result ExMatrix handles large, complex RF models and promotes interpretability.
The paper develops deep learning models for personalized treatment rules in survival analysis.
problem Deriving optimal treatment rules for bivariate survival outcomes in randomized trials.
method Adaptive prediction-powered learning using deep neural networks and stochastic policies.
result Maximizes joint survival probability beyond fixed time points (t1,t2). State-of-the-art learning algorithms, such as random forests or neural networks, are often qualified as "black-boxes" because of the high number and complexity of operations involved in their prediction mechanism. This lack of interpretability is a strong limitation for applications involving critical decisions, typica…
Recent exploration of optimal individualized decision rules (IDRs) for patients in precision medicine has attracted a lot of attention due to the heterogeneous responses of patients to different treatments. In the existing literature of precision medicine, an optimal IDR is defined as a decision function mapping from t…
Rule-based classifiers quantify uncertainty using Bernoulli random variables.
problem Quantifying the uncertainty of precision estimates for rule-based text classifiers.
method Treat partitions of sub-strings as Bernoulli random variables, compare means using statistical tests, and combine classifiers using Dempster-Shafer theory.
result The approach can be used to combine binary classifiers into a multi-label classifier.
Coordinate descent methods employ random partial updates of decision variables in order to solve huge-scale convex optimization problems. In this work, we introduce new adaptive rules for the random selection of their updates. By adaptive, we mean that our selection rules are based on the dual residual or the primal-du…
Oblique BART improves tree-based predictions.
problem Axis-aligned decision rules in BART can be suboptimal.
method Developed an oblique version of BART using data-adaptive hyperplane partitions.
result Oblique BART outperformed axis-aligned BART and other tree methods on benchmarks.
New GAN formulation addresses mode collapse issue.
problem Mode collapse in GANs.
method Randomized decision rules, empirical Bayes, stochastic gradient MCMC.
result Proposed method converges to Nash equilibrium.
Paper extends transfer learning for decision rules, improving treatment rule estimation.
problem Estimating optimal individualized treatment rules under changing conditions.
method Bayes decision rules and low-dimensional empirical risk minimization.
result Consistent estimators and risk bounds established under mild conditions.
Recent decades have seen an interest in prediction problems for which Bayesian methodology has been used ubiquitously. Sampling from or approximating the posterior predictive distribution in a Bayesian model allows one to make inferential statements about potentially observable random quantities given observed data. Th…
DTE uses tree leaf means to embed data, balancing accuracy and speed.
problem High variance in decision tree splits and computational inefficiency of ensembles.
method DTE constructs an interpretable feature representation using leaf means of a trained tree.
result DTE strikes a balance between accuracy and computational efficiency, outperforming ensembles.
Both neural networks and decision trees are popular machine learning methods and are widely used to solve problems from diverse domains. These two classifiers are commonly used base classifiers in an ensemble framework. In this paper, we first present a new variant of oblique decision tree based on a linear classifier,…
Hidden cost: Smoothing shrinks decision boundaries, affecting class-wise accuracy.
problem The fragility of machine learning models and the need for robustness verification.
method Randomized smoothing approach to achieve statistical robustness.
result Smoothed classifiers' decision boundaries shrink, leading to class-wise accuracy disparity.
A novel gradient-based method optimizes decision trees for complex tasks.
problem Training decision trees with arbitrary differentiable loss functions.
method Gradient-based optimization using first and second derivatives of loss functions.
result Improves accuracy and flexibility in decision tree optimization.
A new method learns interpretable decision rules using submodular optimization.
problem Learning interpretable decision rules from data.
method Submodular optimization approach for selecting rules from a large set.
result The method effectively learns interpretable rule sets from real datasets.
New algorithms optimize decision rules in strategic scenarios, minimizing prediction risk and incentivizing better outcomes.
problem Strategic agents manipulate features to improve outcomes, complicating decision-making models.
method Efficient algorithms for learning decision rules that minimize prediction risk, incentivize better outcomes, and estimate true model coefficients.
result Optimal decision rules can be learned through testing and observing agent responses, circumventing hardness results.
VisRuler simplifies decision extraction from bagged and boosted trees.
problem Complexity and lack of interpretability in ensemble models.
method Visual analytics tool for selecting robust models, important features, and essential decisions.
result Users successfully extracted and explained decisions from ensemble models.
Improves decision tree performance by correcting split selection errors.
problem Invalid statistical guarantees in split selection for decision trees.
method Introduces anytime-valid inference to provide valid statistical guarantees.
result Provides anytime-valid control of false splits under arbitrary data streams.
LI-ITR combines flexible ML with interpretable approximations for personalized treatment rules.
problem Combining flexibility and interpretability in personalized treatment rules.
method Uses variational autoencoders and a mixture of interpretable experts.
result Accurately recovers true local coefficients and optimal treatment strategies.
Paper introduces exact credible sets for classification problems.
problem No general way to construct exact credible sets for classification.
method Generalized credible set with connection to Neyman--Pearson lemma and randomized decision rule.
result Achieves any preassigned credible level for classification problems.
This work proposes optimal decision rules for hierarchical classifiers to better align with evaluation metrics.
problem Heuristic decision rules in hierarchical classification do not align with evaluation metrics.
method Derives optimal decision rules for various prediction settings, focusing on hierarchical hFβ scores. result Optimal decision rules enhance the performance and reliability of hierarchical classifiers.
This paper develops a new method to model treatment effects that are heterogeneous across different quantiles.
problem Modeling treatment effects that vary across different quantiles of the outcome distribution.
method The paper combines quantile classification with local polynomial estimation to build a decision tree and forest.
result The proposed QLPRT and QLPRF methods provide a new way to estimate and infer heterogeneous treatment effects.
Sparse oblique decision tree improves security rules for renewable power systems.
problem Identifying secure operating conditions in power systems with high renewable energy.
method Sparse weighted oblique decision tree to learn and embed linear security rules.
result The method significantly increases secure states and reduces solution time.
R2N learns interpretable rules and literals from numerical features.
problem Lack of expressive vocabulary in rule-based decision models.
method Relational Rule Network (R2N) learns literals and rules end-to-end.
result Learned literals improve prediction accuracy and rule conciseness.
Autonomous driving decision-making is a great challenge due to the complexity and uncertainty of the traffic environment. Combined with the rule-based constraints, a Deep Q-Network (DQN) based method is applied for autonomous driving lane change decision-making task in this study. Through the combination of high-level …
P-SE explains model decisions with minimal feature subsets and fast estimators.
problem Explain model decisions in regression and classification.
method Probabilistic Sufficient Explanations (P-SE) with random Forests for conditional probability estimation.
result Consistent and efficient explanations for regression and classification models.
Advances rule-based multi-label classification using conformal prediction.
problem Improving accuracy and decision making in multi-label classification.
method Combines conformal prediction with rule-based learning to provide natural conformity scores and calibrate rule assessments.
result Calibrated conformity scores enhance prediction accuracy and decision making.
S-SIRUS explains RF for spatial data, improving accuracy and interpretability.
problem Non-interpretable nature of Random Forest in spatially dependent data.
method Proposes S-SIRUS, a spatial extension of SIRUS for extracting interpretable rules.
result S-SIRUS outperforms SIRUS in spatially dependent data, offering higher predictive accuracy and shorter rule lists.
We propose a new Bayesian model for flexible nonlinear regression and classification using tree ensembles. The model is based on the RuleFit approach in Friedman and Popescu (2008) where rules from decision trees and linear terms are used in a L1-regularized regression. We modify RuleFit by replacing the L1-regularizat…
Transform ANNs into interpretable decision trees.
problem Lack of interpretability in ANNs.
method Developed two MDT algorithms: EC-DT and Extended C-Net.
result Extended C-Net generates the most compact and effective trees.
MOSS optimizes decision rules for accuracy and stability.
problem Constructing stable sets of decision rules.
method Multi-objective optimization framework incorporating sparsity, accuracy, and stability.
result MOSS outperforms state-of-the-art rule ensembles in predictive performance and stability.
Meta decision trees explain user ratings in recommendation systems.
problem Building explainable recommendation systems with clear user explanations.
method Learned regression functions and sparse decision rules based on user embeddings.
result The method provides accurate and interpretable ratings.
Bayesian method infers local rules for collective animal movement.
problem Learn local rules governing long-term group behaviors.
method Bayesian Inverse Reinforcement Learning with Linearly-Solvable Markov Decision Process.
result Recover true costs and find value of collective movement.
SOAR generates rules for both positive and negative classes in binary classification.
problem Lack of interpretability in machine learning models for binary classification.
method Extends or-of-and classification technique to both positive and negative classes.
result Competitive classification performance with simulated-annealing optimization.
There has been significant recent work on the theory and application of randomized coordinate descent algorithms, beginning with the work of Nesterov [SIAM J. Optim., 22(2), 2012], who showed that a random-coordinate selection rule achieves the same convergence rate as the Gauss-Southwell selection rule. This result su…
SBAMDT uses adaptive soft splits to model complex decision boundaries.
problem Limited ability of standard decision trees to capture complex decision boundaries.
method Probabilistic additive decision tree model with adaptive soft multivariate splits.
result Demonstrated improved predictive performance on synthetic and real datasets.
We seek decision rules for prediction-time cost reduction, where complete data is available for training, but during prediction-time, each feature can only be acquired for an additional cost. We propose a novel random forest algorithm to minimize prediction error for a user-specified {\it average} feature acquisition b…
An online decision-making algorithm using stochastic gradient descent for big data.
problem Efficiently updating decision rules in online decision making with big data.
method Stochastic gradient descent for online updates, asymptotic normality of estimators.
result Asymptotic normality of parameter and value estimators, enabling statistical inference.
Paper proves LCVB method's consistency in Bayesian posteriors and decision rules.
problem Approximating Bayesian posteriors and decision rules.
method Loss-calibrated variational Bayes (LCVB) method.
result LCVB method's consistency in both approximate posterior and decision rules.
This paper improves prediction rule ensembles using model-based data generation.
problem Improving the sparsity and predictive accuracy of prediction rule ensembles.
method The authors use surrogate models to train Lasso regression with data generated by a boosted decision tree ensemble, improving PRE performance.
result The use of surrogacy models can substantially improve the sparsity of PRE while retaining predictive accuracy.
Improves generative models for cost-sensitive decisions.
problem Generative models lack awareness of decision costs.
method Integrates a decision loss into the training objective.
result Improves cost-sensitive forecast accuracy.
In the artificial intelligence field, learning often corresponds to changing the parameters of a parameterized function. A learning rule is an algorithm or mathematical expression that specifies precisely how the parameters should be changed. When creating an artificial intelligence system, we must make two decisions: …