New framework optimizes classification trees with logistic loss and ℓ1 regularization.
problem Improving interpretability and generalization of classification trees.
method Developed a generalized framework for CTs, incorporating logistic loss and ℓ1 regularization. result Optimal Logistic Tree model outperforms state-of-the-art MIP-based approaches in terms of interpretability and generalization.
This paper analyzes stability of decision trees and logistic regression.
problem Stability of decision trees and logistic regression is analyzed to understand their performance and sensitivity.
method Two stability notions (hypothesis and pointwise hypothesis stability) are derived for decision trees and logistic regression. The stability of decision trees depends on the number of leaves, while for logistic regression, it depends on the smallest eigenvalue of the Hessian matrix. Upper bounds on generalization error are constructed.
result Logistic regression is not a stable learning algorithm.
We propose a new algorithm called PLUTO for building logistic regression trees to binary response data. PLUTO can capture the nonlinear and interaction patterns in messy data by recursively partitioning the sample space. It fits a simple or a multiple linear logistic regression model in each partition. PLUTO employs th…
In this work, we study the use of logistic regression in manufacturing failures detection. As a data set for the analysis, we used the data from Kaggle competition Bosch Production Line Performance. We considered the use of machine learning, linear and Bayesian models. For machine learning approach, we analyzed XGBoost…
Boosted decision trees typically yield good accuracy, precision, and ROC area. However, because the outputs from boosting are not well calibrated posterior probabilities, boosting yields poor squared error and cross-entropy. We empirically demonstrate why AdaBoost predicts distorted probabilities and examine three cali…
A hybrid model improves bankcard response classification accuracy.
problem Bankcard response classification accuracy improvement.
method CHAID for interaction detection in logistic regression.
result The hybrid model outperforms logistic regression in classification accuracy.
New model identifies microbial subcommunities robustly, accounting for cross-sample heterogeneity.
problem Inference in LDA is sensitive to the number of subcommunities and often creates artificial ones.
method Incorporates logistic-tree normal (LTN) model into LDA to account for cross-sample heterogeneity.
result Restores robustness of inference and identifies meaningful subcommunities.
A new gradient tree boosting framework reduces variance and accelerates performance.
problem High variance in stochastic gradient boosting.
method Combining gradient tree boosting with importance sampling and a regularizer.
result Achieves a linear convergence rate on logistic loss and 2.5x--18x acceleration on LogitBoost and LambdaMART.
Approximates large Random Forest models to save space.
problem Large model size limits deployment on small devices.
method Approximate each tree using multinomial logistic regression.
result Reduced model size without significant loss of accuracy.
Study builds machine learning models to predict ICU patients' one-year mortality.
problem Improving mortality prediction accuracy in ICU patients with heart disease.
method Developed and evaluated various machine learning models using MIMIC-III database.
result Best models achieved 85.12% prediction accuracy and AUC of 0.901.
Obtaining accurate and well calibrated probability estimates from classifiers is useful in many applications, for example, when minimising the expected cost of classifications. Existing methods of calibrating probability estimates are applied globally, ignoring the potential for improvements by applying a more fine-gra…
DAC provides interpretable feature importance curves for tree ensembles.
problem Challenges in understanding feature importance and prediction in tree ensembles.
method Disentangled Attribution Curves (DAC) method to visualize feature importance.
result DAC improves interpretability of tree ensembles while maintaining accuracy.
Domestic violence (DV) is a global social and public health issue that is highly gendered. Being able to accurately predict DV recidivism, i.e., re-offending of a previously convicted offender, can speed up and improve risk assessment procedures for police and front-line agencies, better protect victims of DV, and pote…
Decision trees are consistent for regression and classification tasks even with many predictors.
problem Consistency of decision trees with many predictors.
method CART and C4.5 methodology, oracle inequality, sparsity constraints.
result Decision trees and random forests are consistent for various types of data.
TREX explains tree ensembles by identifying key training examples.
problem Identifying which training examples most influence tree ensemble predictions.
method TREX builds a surrogate model using a kernel that captures tree ensemble structure, approximating the original model.
result TREX provides accurate and effective explanations for tree ensembles.
This paper uses supervised learning to predict optimal chunk-size for parallel linear algebra operations.
problem Finding the optimal chunk-size for parallel linear algebra operations.
method The paper uses supervised learning models (logistic regression, neural networks, decision trees) to predict the optimal chunk-size for multiple linear algebra operations.
result The custom decision tree model outperforms classical decision trees and other models in predicting optimal chunk-size for linear algebra operations.
FILTER model uses fusion penalized logistic threshold regression for high-dimensional data with unknown cut points.
problem Modeling high-dimensional data with unknown cut points and binary responses.
method Fusion penalized logistic threshold regression (FILTER) model with fused lasso penalty for variable selection.
result Established non-asymptotic error bounds for coefficient estimation and model selection consistency.
Study develops a robust federated logistic regression for financial data analysis.
problem Analyzing financial data in a federated setting while protecting privacy and interpretability.
method Proposes a robust federated logistic regression framework balancing privacy, interpretability, and robustness.
result Demonstrates comparable performance to classical centralized algorithms on both IID and non-IID data, including outliers.
Extends CRR model with q-binomial random walks for asset pricing.
problem Asset pricing with time-varying probabilities and trend parameters.
method Introduces a q-binomial extension of the CRR model with non-self-similar binomial trees.
result Convergence to Black-Scholes formula with rate O(N^(-1/2)).
Enhances neural networks in hyperbolic space for better data structure capture.
problem Capturing hierarchical data structures efficiently.
method Unified hyperbolic model for neural network components.
result Superior parameter efficiency and outperformance over Euclidean methods.
YASENN interprets neural networks by partitioning activation sequences.
problem Interpreting complex neural network decisions.
method YASENN uses layer-wise gradient boosting decision trees to distill and partition neuron activation sequences.
result YASENN provides interpretable partitions of the input space, revealing neural network decision artifacts.
The paper uses XAI to predict RFQ fulfillment accuracy.
problem Improving accuracy in predicting RFQ fulfillment for less liquid asset classes.
method Advanced algorithms like Logistic Regression, Random Forest, XGBoost, and Bayesian Neural Tree.
result Improved accuracy in RFQ fill rate predictions.
Study compares ML algorithms for predicting stock market directional bias.
problem Predicting the direction of stock market movements.
method Examined and contrasted logistic regression, decision tree, random forest, and a deep neural network.
result All models consistently reach above 50% in directional bias forecasting.
Bayesian logistic regression improves clinical risk prediction models over time.
problem Improving clinical risk prediction models after deployment to adapt to temporal shifts.
method Bayesian logistic regression (BLR) and Markov variant (MarBLR) for online recalibration and revision of prediction models.
result BLR and MarBLR consistently outperform static models and other online revision methods, improving average AUC and calibration index.
Data mining and machine learning techniques such as classification and regression trees (CART) represent a promising alternative to conventional logistic regression for propensity score estimation. Whereas incomplete data preclude the fitting of a logistic regression on all subjects, CART is appealing in part because s…
Hybrid machine learning improves gallstone risk prediction.
problem Complex gallstone disease risk factors and interactions.
method Adaptive LASSO for variable selection, BART for interactions, differential equations for interpretation.
result Enhanced prediction accuracy and actionable insights.
Gradient boosting adapted for vector inputs.
problem Applying gradient boosting to multi-class classification problems.
method Extended gradient boosting framework to vector inputs using histogram-based decision trees.
result Efficient algorithm for vector-valued objectives.
Enhances explainability of AI models without sacrificing accuracy.
problem Lack of interpretability in black-box models like Deep Neural Networks and Gradient Boosting.
method Co-supervised Local Model Synthesis (SynthTree) using Mixture of Linear Models (MLM).
result Statistical models significantly enhance explainability of AI models.
We explore a novel approach to semi-supervised learning. This approach is contrary to the common approach in that the unlabeled examples serve to "muffle," rather than enhance, the guidance provided by the labeled examples. We provide several variants of the basic algorithm and show experimentally that they can achieve…
Research creates a machine learning model for predicting TAVI patient mortality.
problem Lack of robust risk scores for TAVI patients.
method Gradient boosting on decision trees, feature analysis and selection, model validation.
result Model outperforms existing risk scores with AUC of 0.83.
This paper compares ML algorithms for PD prediction, finding XGBoost to be the most effective.
problem Predicting the probability of default in loan portfolios.
method Comparison of five ML algorithms (Random Forests, Decision Trees, XGBoost, Gradient Boosting, AdaBoost) with logistic regression.
result XGBoost outperforms other ML algorithms for PD prediction.
Study compares Android malware and benign apps using permissions.
problem Classify Android malware from benign apps.
method Used machine learning techniques like decision trees, random forests, SVM, etc.
result Permissions are a strong feature for malware detection.
New loss function improves classification accuracy.
problem Negative expected accuracy as an alternative to cross entropy for classification.
method Empirical investigation using modified leaky version of expected accuracy.
result Comparable or better accuracy compared to cross entropy, robust to label noise.
The paper introduces new measures to quantify variability in decision tree models due to observational multiplicity.
problem The variability in decision tree models due to observational multiplicity.
method Introduces leaf regret and structural regret to decompose observational multiplicity.
result Structural regret is the primary driver of observational multiplicity, accounting for over 15 times the variability of leaf regret in some datasets.
Paper tackles transparency and auditability of machine learning in credit scoring.
problem Missed potential in using modern machine learning for credit scoring due to lack of transparency.
method Develops a framework for making black box machine learning models transparent, auditable, and explainable.
result Comparable interpretability can be achieved with machine learning while maintaining predictive power.
We aim to predict whether an employee of a company will leave or not, using the k-Nearest Neighbors algorithm. We use evaluation of employee performance, average monthly hours at work and number of years spent in the company, among others, as our features. Other approaches to this problem include the use of ANNs, decis…
Package implements ABC-Boost for multi-class classification.
problem Improving accuracy in multi-class classification.
method Combines histogram-based binning and explicit gain formula for efficient tree splitting.
result ABC-Boost significantly improves multi-class classification accuracy.
New insights into how randomization affects greedy model selection.
problem Understanding the impact of feature subsampling on greedy model selection.
method Investigated greedy forward selection with feature subsampling, proving effects on bias and variance.
result Ensembling with feature subsampling reduces both bias and variance, unlike convex base learners.
A Longitudinal Attribute-Conditioned Neural Network (LANTERN) framework for modeling health-state transition probabilities in irregular longitudinal data.
problem Estimating long-term care transition probabilities in irregular longitudinal health data.
method A neural network that learns from individual health history, incorporates time elapsed, and conditions on demographic and socioeconomic attributes.
result Improves severe disability discrimination and maintains strong calibration.
DeepBark improves tree bark re-identification accuracy.
problem Challenging illuminations make tree bark hard to re-identify.
method Used a large dataset of 2,400 bark images to train DeepBark and SqueezeBark.
result DeepBark achieves 87.2% mAP in retrieving relevant bark images.
The recently proposed "generalized min-max" (GMM) kernel can be efficiently linearized, with direct applications in large-scale statistical learning and fast near neighbor search. The linearized GMM kernel was extensively compared in with linearized radial basis function (RBF) kernel. On a large number of classificatio…
Decision trees and shallow neural networks have different geometric complexities, impacting their interpretability and accuracy.
problem The geometric simplicity of decision boundaries in decision trees conflicts with the approximation capabilities of shallow neural networks.
method Analysis of the Radon total variation (RTV) seminorm to compare geometric complexity of decision regions and neural network approximations.
result Smooth barrier scores can approximate decision regions with finite RTV, but their performance depends on the tube-mass condition near the decision boundary.
Generalized additive models (GAMs) are favored in many regression and binary classification problems because they are able to fit complex, nonlinear functions while still remaining interpretable. In the first part of this paper, we generalize a state-of-the-art GAM learning algorithm based on boosted trees to the multi…
A new method selects representative subdata without assuming underlying models.
problem Selecting representative subdata for fast and efficient analysis.
method Uses decision trees to partition data and selects samples from each partition.
result PED subdata results in higher classification accuracy than other methods.
Maximum likelihood estimator performance in logistic regression analyzed.
problem Performance of maximum likelihood estimator in logistic regression.
method Sharp non-asymptotic guarantees for existence and excess logistic risk.
result Sharp guarantees for the existence and excess risk of MLE in logistic regression.
Paper compares AI models for credit scoring and explains them.
problem Lack of interpretability in advanced AI models hinders credit risk management.
method Comparison of logistic regression, AI algorithms, and techniques to interpret AI models.
result Advanced tree-based models provide the best prediction of client default.
Lasso proves consistent model selection for high-dimensional Ising models.
problem Model selection consistency of Lasso for high-dimensional Ising models.
method Theoretical analysis of Lasso with and without post-thresholding for Ising models.
result Lasso without post-thresholding is model selection consistent in the whole paramagnetic phase with n=Ω(d3logp). Logitron combines Perceptron and logistic loss for improved classification.
problem Non-convex and non-smooth zero-one loss function in classification models.
method Introduces a Perceptron-augmented convex classification framework with an extended logistic loss function.
result Hinge-Logitron outperforms logistic regression and SVM in classification accuracy.