Optimal classification rules control error rates in multiclass mixture models.
problem Classifying observations in multiclass mixture models while controlling error rates.
method Finding optimal classification rules by searching an optimal region in the observation space, using Maximum A Posteriori (MAP) rule and heuristic computation.
result The FDR-like optimal rule can be significantly less conservative than thresholded MAP rules.
Paper extends transfer learning for decision rules, improving treatment rule estimation.
problem Estimating optimal individualized treatment rules under changing conditions.
method Bayes decision rules and low-dimensional empirical risk minimization.
result Consistent estimators and risk bounds established under mild conditions.
Paper proves fair classification can be done via simple thresholding.
problem Achieving fair binary classification subject to group fairness constraints.
method Proves Bayes optimal fair learning rule is a group-wise thresholding rule over the Bayes regressor with randomization.
result Proposes an efficient unconstrained optimization algorithm for post-processing fair classification.
New rule-based method for classification with scalability, interpretability, and fairness.
problem Developing a scalable and fair classification method.
method Column generation for linear programming, decision tree-based heuristic, and rule-based optimization.
result The method returns interpretable rules with optimal weights and addresses fairness constraints.
Conventional techniques for supervised classification constrain the classification rules considered and use surrogate losses for classification 0-1 loss. Favored families of classification rules are those that enjoy parametric representations suitable for surrogate loss minimization, and low complexity properties suita…
A new bias score method optimizes fairness in classification.
problem Ensuring fairness in binary classification under group constraints.
method Introducing bias scores and developing a post-hoc approach to adapt to fairness constraints.
result The method maintains high accuracy while ensuring fairness constraints.
Novel approach for creating interpretable classifiers using bilevel optimization of split-rules in NLDTs.
problem Creating highly accurate and easily interpretable classifiers for practical applications.
method Representing classifiers as assemblies of simple mathematical rules using NLDTs with evolutionary bilevel optimization.
result The approach ensures interpretability while achieving high accuracy on various classification problems.
SOAR generates rules for both positive and negative classes in binary classification.
problem Lack of interpretability in machine learning models for binary classification.
method Extends or-of-and classification technique to both positive and negative classes.
result Competitive classification performance with simulated-annealing optimization.
This work proposes optimal decision rules for hierarchical classifiers to better align with evaluation metrics.
problem Heuristic decision rules in hierarchical classification do not align with evaluation metrics.
method Derives optimal decision rules for various prediction settings, focusing on hierarchical hFβ scores. result Optimal decision rules enhance the performance and reliability of hierarchical classifiers.
New scoring rules improve probabilistic classification model evaluation.
problem Traditional scoring rules misalign with the preference for correct classifications.
method Introduces Penalized Brier Score (PBS) and Penalized Logarithmic Loss (PLL) to modify proper scoring rules.
result PBS and PLL better identify optimal checkpoints and early stopping points, leading to superior F1 scores.
New learning rules achieve optimal sample complexity for weakly supervised classification.
problem Learning from label proportions in weakly supervised settings.
method Debiased proportional square loss and EasyLLP learning rule.
result Achieves optimal sample complexity in both realizable and agnostic settings.
As a contribution to interpretable machine learning research, we develop a novel optimization framework for learning accurate and sparse two-level Boolean rules. We consider rules in both conjunctive normal form (AND-of-ORs) and disjunctive normal form (OR-of-ANDs). A principled objective function is proposed to trade …
Many leading classification algorithms output a classifier that is a weighted average of kernel evaluations. Optimizing these weights is a nontrivial problem that still attracts much research effort. Furthermore, explaining these methods to the uninitiated is a difficult task. Letting all the weights be equal leads to …
New method improves model explainability and accuracy with low computational cost.
problem Improving model explainability and accuracy in classification models.
method Distributionally robust optimization to learn sparse ensembles of rule sets.
result Improves model performance on various metrics compared to competing methods.
This paper considers generalized linear models using rule-based features, also referred to as rule ensembles, for regression and probabilistic classification. Rules facilitate model interpretation while also capturing nonlinear dependences and interactions. Our problem formulation accordingly trades off rule set comple…
The paper optimizes k-NN for distributed learning with minimax optimal performance.
problem Minimizing error rates in classification, regression, and density estimation.
method Optimal aggregation of fixed-k nearest neighbors from multiple subsets of data.
result Achieves minimax optimal error rates up to a logarithmic factor.
Interpretable classifiers have recently witnessed an increase in attention from the data mining community because they are inherently easier to understand and explain than their more complex counterparts. Examples of interpretable classification models include decision trees, rule sets, and rule lists. Learning such mo…
Recently, several authors have advocated the use of rule learning algorithms to model multi-label data, as rules are interpretable and can be comprehended, analyzed, or qualitatively evaluated by domain experts. Many rule learning algorithms employ a heuristic-guided search for rules that model regularities contained i…
Advances rule-based multi-label classification using conformal prediction.
problem Improving accuracy and decision making in multi-label classification.
method Combines conformal prediction with rule-based learning to provide natural conformity scores and calibrate rule assessments.
result Calibrated conformity scores enhance prediction accuracy and decision making.
For high-dimensional classification, it is well known that naively performing the Fisher discriminant rule leads to poor results due to diverging spectra and noise accumulation. Therefore, researchers proposed independence rules to circumvent the diverse spectra, and sparse independence rules to mitigate the issue of n…
In the framework of supervised classification (discrimination) for functional data, it is shown that the optimal classification rule can be explicitly obtained for a class of Gaussian processes with "triangular" covariance functions. This explicit knowledge has two practical consequences. First, the consistency of the …
We consider the problem of learning a sparse rule model, a prediction model in the form of a sparse linear combination of rules, where a rule is an indicator function defined over a hyper-rectangle in the input space. Since the number of all possible such rules is extremely large, it has been computationally intractabl…
Binary classification rules based on covariates typically depend on simple loss functions such as zero-one misclassification. Some cases may require more complex loss functions. For example, individual-level monitoring of HIV-infected individuals on antiretroviral therapy (ART) requires periodic assessment of treatment…
The paper explores how to select data points for optimal learning performance.
problem Optimizing data selection for empirical risk minimizers.
method Fixing a learning rule and focusing on optimizing the training data selection.
result Achieving performance comparable to training on the entire population with a small subset of data points.
SigD2 reduces noisy rules in rule-based classifiers for better accuracy and readability.
problem Redundant and noisy rules in rule-based classifiers reduce model accuracy and readability.
method Two-stage pruning strategy and ensemble methods (bagging and boosting) to reduce noise and improve model performance.
result SigD2 and ACboost ensemble models outperform state-of-the-art classifiers in terms of accuracy and rule count.
The paper tackles fair classification with multiple sensitive features.
problem Existing fair classification methods often consider a single sensitive feature, but in practice, individuals are defined by multiple sensitive features.
method Characterizes Bayes-optimal fair classifiers for multiple sensitive features under various fairness measures, proposing in-processing and post-processing algorithms.
result Bayes-optimal fair classifiers for multiple sensitive features are instance-dependent thresholding rules that rely on a weighted sum of group membership probabilities.
Defines a similarity measure for classification distributions.
problem Measuring similarity between classification distributions.
method Proposes task similarity, a novel measure quantifying performance of source distributions on target distributions.
result Empirical task similarity correlates with transfer efficiency and semantic similarity of source distributions.
Paper classifies ruled surfaces in Lorentz-Minkowski space for a specific flow.
problem Classifying ruled surfaces in Lorentz-Minkowski space.
method Examining homothetic self-similar solutions of the inverse mean curvature flow.
result Existence of two classes of non-cylindrical homothetic solitons.
Multi-label classification (MLC) is a supervised learning problem in which, contrary to standard multiclass classification, an instance can be associated with several class labels simultaneously. In this chapter, we advocate a rule-based approach to multi-label classification. Rule learning algorithms are often employe…
Interpretability has always been a major concern for fuzzy rule-based classifiers. The usage of human-readable models allows them to explain the reasoning behind their predictions and decisions. However, when it comes to Big Data classification problems, fuzzy rule-based classifiers have not been able to maintain the g…
New reformulations for multiclass classification problems using optimal transport.
problem Adversarial multiclass classification problems.
method Multimarginal optimal transport formulation.
result Reveals geometric structure and extends binary classification results.
This paper gives, in generic situations, a complete classification of ruled minimal surfaces in pseudo-Euclidean space with arbitrary index. In addition, we discuss the condition for ruled minimal surfaces to exist, and give a counter-example on the problem of Bernstein type.
Given an n-sample of random vectors (Xi,Yi)1≤i≤n whose joint law is unknown, the long-standing problem of supervised classification aims to \textit{optimally} predict the label Y of a given a new observation X. In this context, the nearest neighbor rule is a popular flexible and intuitive method …
OptiNet achieves near-minimax error rates with compression in Euclidean space.
problem Error and compression rates in non-parametric multiclass classification.
method Compression-based learning rule OptiNet and a novel general compression scheme.
result OptiNet achieves non-trivial compression rates with near-minimax error rates in Euclidean space.
A new classification rule for FDA improves classification performance by accounting for unequal covariance matrices.
problem Unequal covariance matrices in practical situations affect the performance of FDA and its variants.
method Proposes a novel classification rule for FDA that accounts for unequal covariance matrices, applicable to many FDA variants.
result The new classification rule improves classification performance compared to original FDA and variants.
A new decision tree induction method using MIP for faster optimization.
problem Optimizing decision tree split rules for better classification performance.
method Mixed-integer programming (MIP) for Gini reduction maximization, efficient search algorithm.
result bsnsing trees outperform other tree models in new case discrimination.
Optimal biomarker combinations for treatment-selection can be derived by minimizing total burden to the population caused by the targeted disease and its treatment. However, when multiple biomarkers are present, including all in the model can be expensive and hurt model performance. To remedy this, we consider feature …
Develops methods for fair classification under linear disparity constraints.
problem Disparate impacts of machine learning algorithms on protected groups.
method Bayes-optimal fair classification methods via pre-, in-, and post-processing.
result Explicit forms of Bayes-optimal fair classifiers under linear disparity measures.
A novel gradient-based method optimizes decision trees for complex tasks.
problem Training decision trees with arbitrary differentiable loss functions.
method Gradient-based optimization using first and second derivatives of loss functions.
result Improves accuracy and flexibility in decision tree optimization.
Paper presents unsupervised calibration for split conformal classification.
problem Inconvenient requirement of labeled calibration samples.
method Uses unsupervised calibration samples alongside supervised training samples.
result Achieves comparable performance to supervised calibration methods.
We discuss theoretical aspects of the product rule for classification problems in supervised machine learning for the case of combining classifiers. We show that (1) the product rule arises from the MAP classifier supposing equivalent priors and conditional independence given a class; (2) under some conditions, the pro…
A new ensemble method improves kNN performance by extending the neighborhood rule.
problem Traditional kNN's limitations when test points are outside the spherical region and ensemble's high errors.
method Determines neighbors in k steps, using bootstrap samples and optimal models selection.
result The proposed ensemble method outperforms state-of-the-art methods on 17 benchmark datasets.
Differentially-private Bayes consistency rule for binary classification and density estimation.
problem Privacy constraints limit private learning in the distribution-free PAC model.
method Constructs a universally Bayes consistent learning rule that satisfies differential privacy.
result Private learning is possible for arbitrary distributions, even with a single algorithm.
Tree-based algorithm for functional data analysis reduces generalization error.
problem Classification and regression problems with functional data.
method Constrained convex optimization for weighted functional L2 space, multiple splitting rules, and weighted integral features. result Reduces generalization error while maintaining interpretability.
Paper introduces MRCs that minimize worst-case 0-1 loss, providing tight performance guarantees.
problem Minimizing worst-case 0-1 loss in classification.
method MRCs that minimize worst-case 0-1 loss with uncertainty sets of distributions.
result MRCs provide tight performance guarantees and are strongly universally consistent.
Binary classification is a common statistical learning problem in which a model is estimated on a set of covariates for some outcome indicating the membership of one of two classes. In the literature, there exists a distinction between hard and soft classification. In soft classification, the conditional class probabil…
Study classifies ruled surfaces critical to Dirichlet energy.
problem Identifying ruled surfaces critical to Dirichlet energy.
method Explicit parametrization of ruled surfaces.
result Classification of ruled surfaces as critical points of Dirichlet energy.
New scoring rules compare probabilistic top lists in classification.
problem Evaluation of probabilistic top lists in classification.
method Elicitability through symmetric proper scoring rules.
result Brier score provides a well-suited metric for comparison.