The study examines how altering impurity functions influences optimal splits in binary classification trees.
problem Understanding how altering impurity functions affects optimal splits in binary classification trees.
method Investigates how skewing impurity functions biases optimal splits towards isolating points of a particular class.
result A necessary and sufficient condition for skewing an impurity function to bias optimal splits towards isolating points of a particular class is provided.
Bregman perspective on CART provides a unified framework for impurity measures.
problem Unifying impurity measures in CART
method Bregman divergence approach
result Unified framework for impurity measures
A new method estimates uncertainty without explicit prediction models.
problem Costly data acquisition in machine learning.
method Distance-weighted Class Impurity method for uncertainty estimation.
result Distance-weighted Class Impurity effectively estimates uncertainty without prediction models.
New method classifies spinor orbits in dimensions up to 14.
problem Classify orbits of the spin group in semi-spinor spaces.
method Use pure spinors and combinatorial constraints to classify orbits.
result Classification of spinor orbits in dimensions up to 14.
Machine learning methods are applied to finding the Green's function of the Anderson impurity model, a basic model system of quantum many-body condensed-matter physics. Different methods of parametrizing the Green's function are investigated; a representation in terms of Legendre polynomials is found to be superior due…
We consider supersymmetric gauge theories with impurities in various dimensions. These systems arise in the study of intersecting branes. Unlike conventional gauge theories, the Higgs branch of an impurity theory can have compact directions. For models with eight supercharges, the Higgs branch is a hyperKahler manifold…
The paper studies bias and adaptivity of CART regression trees.
problem Bias and adaptivity of CART regression trees.
method Derives an interesting connection between bias and MDI measure of variable importance.
result Decision trees with CART have small bias and are adaptive to signal strength and direction.
The paper analyzes the convergence of CART under a SID condition, improving previous results.
problem Investigating the convergence rate of CART under a sufficient impurity decrease condition.
method Established an upper bound on prediction error under SID condition, introduced easily verifiable conditions.
result Improved convergence rate of CART under SID condition, demonstrated examples of error bound limitations.
A hybrid impurity measure balances theoretical soundness and computational efficiency.
problem Developing a robust impurity measure for decision trees.
method Integrates Tsallis entropy with an exponential polarization component.
result Simple parametric measures outperform ITC, but ITC variants are competitive with strong theoretical guarantees.
Novel loss functions improve decision tree learning from noisy data.
problem Training decision trees with noisy labels.
method Introducing distribution losses and a new negative exponential loss.
result The negative exponential loss leads to efficient and robust decision tree learning.
A persistent challenge in practical classification tasks is that labeled training sets are not always available. In particle physics, this challenge is surmounted by the use of simulations. These simulations accurately reproduce most features of data, but cannot be trusted to capture all of the complex correlations exp…
This paper analyzes Mean Decrease Impurity (MDI) variable importance in random forests.
problem Lack of interpretability in random forest variable importances.
method Analysis of Mean Decrease Impurity (MDI) in random forests.
result MDI provides a variance decomposition of the output when variables are independent and there are no interactions.
AquaSight detects water impurity using deep learning.
problem Water pollution and lack of affordable water quality assessment.
method Convolutional Neural Networks for automated water impurity detection.
result Deep learning model achieved 96% accuracy in detecting water contamination.
This paper optimizes high-dimensional oblique splits for decision trees, enhancing performance and computational efficiency.
problem Enhancing decision tree performance and computational efficiency in high-dimensional data.
method Established Sufficient Impurity Decrease (SID) convergence for s0-sparse oblique splits, proposing progressive trees for iterative refinement. result Demonstrated that SID function class expands with s0-sparsity, enabling capture of complex data-generating processes. New model-independent compact representations of imaginary-time data are presented in terms of the intermediate representation (IR) of analytical continuation. This is motivated by a recent numerical finding by the authors [J. Otsuki et al., arXiv:1702.03056]. We demonstrate the efficiency of the IR through continuous-…
Optimizes harvesting in biopharmaceutical fermentation with limited data.
problem High variability in fermentation outputs due to model risk with small data.
method Stochastic model, Bayesian approach, Markov decision process.
result Improves fermentation output and reduces variability.
Develops RF-GLS for binary geospatial data.
problem Challenges in extending RF to binary geospatial data.
method Proposes RF-GLS for binary data, embedding it in generalized mixed effects models.
result Establishes consistency of RF-GP for mean function and covariate effect estimation.
Global stability bounds for matrix frames in phase retrieval problems.
problem Phase retrieval for matrix frames in various applications.
method Computable global stability bounds for the quasi-linear analysis map β, using Whitney stratification of positive semidefinite matrices of low rank.
result Novel conditions for a frame to be generalized phase retrievable.
New local MDI variable importances derived from global scores match Shapley values.
problem Local feature relevance in tree-based models.
method Deriving local MDI importance measure from global scores and linking it to Shapley values.
result Local MDI importances have a natural connection with Shapley values.
Rapidly growing product lines and services require a finer-granularity forecast that considers geographic locales. However the open question remains, how to assess the quality of a spatio-temporal forecast? In this manuscript we introduce a metric to evaluate spatio-temporal forecasts. This metric is based on an Opti- …
Paper explores how unsupervised learning can be understood through linear algebra concepts.
problem Understanding unsupervised learning through linear algebra concepts.
method Introducing the concept of linearly independent populations and using them to solve for prevalence values.
result Unsupervised learning can be realized as a generalization of supervised learning.
We present ADHM-Nahm data for instantons on the Taub-NUT space and encode these data in terms of Bow Diagrams. We study the moduli spaces of the instantons and present these spaces as finite hyperkahler quotients. As an example, we find an explicit expression for the metric on the moduli space of one SU(2) instanton. W…
Self-poisoning in adaptive OOD detectors is explained with a sharp threshold theory and certified calibration.
problem Self-poisoning in adaptive OOD detectors.
method Modeling bank impurity as a generalized Pólya urn, proving almost-sure convergence to a mean-field equilibrium.
result A certified admission gate removes the transition at every contamination rate, controlling false positives label-free.
Improved decision tree learning guarantees for complex functions.
problem Achieving provable guarantees for decision tree induction with complex target functions.
method Introduces a new splitting criterion that considers correlations between target function and subsets of attributes.
result Proves provable guarantees for all target functions with respect to the uniform distribution, circumventing previous impossibility results.
Recommender systems play an essential role in the modern business world. They recommend favorable items like books, movies, and search queries to users based on their past preferences. Applying similar ideas and techniques to Monte Carlo simulations of physical systems boosts their efficiency without sacrificing accura…
Study automates feature selection and clustering for HFT stock price forecasting.
problem Manual feature selection and clustering for high-frequency trading (HFT) stock price forecasting.
method Dual competitive feature importance mechanism and clustering via shallow neural network topology.
result Enhanced forecasting ability of the RBFNN regressor through automated feature selection and clustering.
ALPE improves mid-price forecasting in HFT with real-time data.
problem Real-time mid-price forecasting in high-frequency trading.
method Adaptive Learning Policy Engine (ALPE) using RL and adaptive epsilon decay.
result ALPE outperforms other models in mid-price forecasting.
Study uses random forest to detect unlawful insider trading in financial data.
problem Detecting and identifying unlawful insider trading in complex financial data.
method Integrates PCA-RF and standalone RF models with semi-manually labeled transactions.
result 96.43% accurate classification of transactions, 95.47% lawful, 98.00% unlawful.
Novel approach for creating interpretable classifiers using bilevel optimization of split-rules in NLDTs.
problem Creating highly accurate and easily interpretable classifiers for practical applications.
method Representing classifiers as assemblies of simple mathematical rules using NLDTs with evolutionary bilevel optimization.
result The approach ensures interpretability while achieving high accuracy on various classification problems.
The paper analyzes ESS metrics and their connections to entropy families.
problem Improving the accuracy of effective sample size approximations.
method Examined Rényi and Tsallis entropy families and their relationship to ESS.
result Proved that ESS functions in the Huggins-Roy family meet theoretical conditions.
We seek decision rules for prediction-time cost reduction, where complete data is available for training, but during prediction-time, each feature can only be acquired for an additional cost. We propose a novel random forest algorithm to minimize prediction error for a user-specified {\it average} feature acquisition b…
A novel hybrid data-driven approach is developed for forecasting power system parameters with the goal of increasing the efficiency of short-term forecasting studies for non-stationary time-series. The proposed approach is based on mode decomposition and a feature analysis of initial retrospective data using the Hilber…
Plasma tomography consists in reconstructing the 2D radiation profile in a poloidal cross-section of a fusion device, based on line-integrated measurements along several lines of sight. The reconstruction process is computationally intensive and, in practice, only a few reconstructions are usually computed per pulse. I…
Regularizes decision trees to reduce inference time by up to 4x with minimal accuracy loss.
problem Optimizing decision tree execution time on resource-constrained devices.
method Regularizes impurity computation during CART algorithm training to favor highly asymmetric distributions.
result Reduces inference time by up to 4x with minimal accuracy loss.
Proposes MRF for consistency and privacy in RF.
problem Insufficient theoretical understanding of RF's consistency and privacy.
method Introduces MRF with multinomial distributions for feature and value selection.
result Proves MRF's consistency and analyzes its privacy within differential privacy.
We study both the continuous model and the discrete model of the integer quantum Hall effect on the hyperbolic plane in the presence of disorder, extending the results of an earlier paper [CHMM]. Here we model impurities, that is we consider the effect of a random or almost periodic potential as opposed to just periodi…
New algorithm reduces discrimination in predictions.
problem Tackles potential discrimination in AI predictions.
method Integrates fairness adjustments into tree-building process.
result Reduces discriminatory predictions without significant loss in accuracy.
RPSF improves shapelet-based random forest accuracy and speed.
problem Limitations of random shapelet forest, including high training cost and limited information per node.
method Combines pairs of shapelets, omits threshold searching, and uses DMDI for interpretability.
result Improves accuracy and training speed of shapelet-based random forest.
Semi-automatic data annotation helps experts label unlabeled samples based on feature space projection.
problem Laborious manual data annotation for machine learning.
method Interactive semi-automatic approach using feature space projection and semi-supervised learning.
result Reduces user annotation effort and improves classification accuracy.
FedForest adapts RF for federated learning, improving performance and efficiency.
problem Adapting RF for federated learning with heterogeneous data.
method FedForest uses a novel splitting procedure to aggregate client statistics, allowing non-parametric personalization.
result FedForest's federated RF achieves performance close to centralized models while being communication-efficient.
The paper studies statistical properties of CART regression trees.
problem Understanding the statistical properties of CART regression trees.
method The paper constructs a prior distribution on split points and solves a nonlinear optimization problem to bound the Pearson correlation between the optimal decision stump and response data.
result CART with cost-complexity pruning achieves an optimal complexity/goodness-of-fit tradeoff when the depth scales with the logarithm of the sample size.
Supervised learning algorithms are nowadays successfully scaling up to datasets that are very large in volume, leveraging the potential of in-memory cluster-computing Big Data frameworks. Still, massive datasets with a number of large-domain categorical features are a difficult challenge for any classifier. Most off-th…
A new tree-based model for multivariate responses interprets piecewise linear regimes.
problem Recovering piecewise multivariate linear regimes in complex data.
method Twoblock clustering trees with coskewness-based dimension reduction.
result Recovery of piecewise linear regimes in data.
Proposes ReDT for interpretable, compressed, and robust decision trees.
problem Improving interpretability and performance of decision trees.
method Knowledge distillation with soft labels and multiple cross-validation.
result ReDT achieves fewer nodes than classical decision trees while maintaining good performance and interpretability.
Kernel methods are popular in clustering due to their generality and discriminating power. However, we show that many kernel clustering criteria have density biases theoretically explaining some practically significant artifacts empirically observed in the past. For example, we provide conditions and formally prove the…
Two algorithms for interpreting and boosting tree-based models using rule covering.
problem Interpreting and boosting tree-based ensemble methods.
method Mathematical programming models constructed from decision tree rules.
result Selects a few rules that closely match the accuracy of the model.
Combines RFs and GLMs for better accuracy and interpretable feature importance.
problem Inability to interpret random forests (RFs) due to black box nature and unstable feature importance methods.
method Reinterprets decision trees and MDI as linear regression and R² values, combining RFs and GLMs in RF+.
result MDI+ outperforms existing feature importance measures in identifying signal features and stability.
Collaborative Trees model analyzes feature interactions and additive effects.
problem Analyzing complex statistical associations between features and response variables.
method Proposes a novel tree model and its bagging version to decompose mean decrease in impurity and visualize feature contributions.
result Demonstrates the superior capability of the tree model in estimating additive effects and interaction effects.