Classy learns interpretable probabilistic rule lists for multiclass classification.
problem Creating interpretable multiclass classifiers that are both accurate and understandable.
method Probabilistic rule lists and minimum description length (MDL) principle for model selection.
result Classy selects small probabilistic rule lists that outperform state-of-the-art classifiers in terms of predictive performance and interpretability.
New scoring rules compare probabilistic top lists in classification.
problem Evaluation of probabilistic top lists in classification.
method Elicitability through symmetric proper scoring rules.
result Brier score provides a well-suited metric for comparison.
NeuRules learns interpretable rule lists from data without pre-discretization.
problem Interpretable rule lists for high-stakes decisions in machine learning.
method Neuro-symbolic continuous optimization with temperature annealing.
result NeuRules outperforms existing methods in learning interpretable rule lists.
We present the design and implementation of a custom discrete optimization technique for building rule lists over a categorical feature space. Our algorithm produces rule lists with optimal training performance, according to the regularized empirical risk, with a certificate of optimality. By leveraging algorithmic bou…
Proposes FairCORELS for fair, interpretable rule lists.
problem Fairness in interpretable models for high-stakes decisions.
method Multi-objective variant of CORELS, supporting fairness measures.
result Outperforms state-of-the-art fair classification techniques.
A new method explains RNNs by decision lists over skipgrams, improving explanation fidelity and interpretability.
problem Lack of understanding how input segments combine to form patterns in neural network outputs.
method Proposes a pipeline to explain RNNs using decision lists over skipgrams, creating synthetic and real-world datasets for evaluation.
result Persistently achieves high explanation fidelity and interpretable rules.
We aim to produce predictive models that are not only accurate, but are also interpretable to human experts. Our models are decision lists, which consist of a series of if...then... statements (e.g., if high blood pressure, then stroke) that discretize a high-dimensional, multivariate feature space into a series of sim…
In this paper we introduce a novel family of decision lists consisting of highly interpretable models which can be learned efficiently in a greedy manner. The defining property is that all rules are oriented in the same direction. Particular examples of this family are decision lists with monotonically decreasing (or i…
pRSL combines probabilistic rules to improve multi-label classification.
problem Modeling the structure between multi-label classes for better performance.
method Uses probabilistic propositional logic rules and belief propagation to combine predictions from multiple classifiers.
result pRSL achieves state-of-the-art performance on various benchmark datasets.
Study shows registration regime improves IPO pricing efficiency in China.
problem Improving IPO pricing efficiency in China's entrepreneurial boards.
method Examined the impacts of regime changes and rule implementations on IPO pricing efficiency.
result The registration regime is most efficient in Chinese IPO pricing.
Generative networks minimize predictive scoring rules for probabilistic forecasting.
problem Evaluating and improving probabilistic forecasts using generative models.
method Training generative networks to minimize predictive-sequential scoring rules on temporal sequences.
result Our method outperforms adversarial approaches in probabilistic calibration.
Black-box explanation is the problem of explaining how a machine learning model -- whose internal logic is hidden to the auditor and generally complex -- produces its outcomes. Current approaches for solving this problem include model explanation, outcome explanation as well as model inspection. While these techniques …
Mining association rules is an important technique for discovering meaningful patterns in transaction databases. Many different measures of interestingness have been proposed for association rules. However, these measures fail to take the probabilistic properties of the mined data into account. In this paper, we start …
New scoring rules improve probabilistic classification model evaluation.
problem Traditional scoring rules misalign with the preference for correct classifications.
method Introduces Penalized Brier Score (PBS) and Penalized Logarithmic Loss (PLL) to modify proper scoring rules.
result PBS and PLL better identify optimal checkpoints and early stopping points, leading to superior F1 scores.
S-SIRUS explains RF for spatial data, improving accuracy and interpretability.
problem Non-interpretable nature of Random Forest in spatially dependent data.
method Proposes S-SIRUS, a spatial extension of SIRUS for extracting interpretable rules.
result S-SIRUS outperforms SIRUS in spatially dependent data, offering higher predictive accuracy and shorter rule lists.
Kernel Bayesian inference is a principled approach to nonparametric inference in probabilistic graphical models, where probabilistic relationships between variables are learned from data in a nonparametric manner. Various algorithms of kernel Bayesian inference have been developed by combining kernelized basic probabil…
We provide self-contained proof of a theorem relating probabilistic coherence of forecasts to their non-domination by rival forecasts with respect to any proper scoring rule. The theorem appears to be new but is closely related to results achieved by other investigators.
This paper introduces probabilistic SNNs for efficient neural processing.
problem Training algorithms for SNNs lag behind hardware implementations.
method Discrete-time probabilistic models and variational inference.
result Derivation of learning rules for SNNs from first principles.
ProSMIN improves representation quality through probabilistic self-supervised learning.
problem Improving representation quality in self-supervised learning.
method ProSMIN uses two neural networks, online and target, to learn diverse representations through knowledge distillation and a modified scoring rule loss function.
result ProSMIN achieves superior accuracy and calibration on various downstream tasks.
Given a system of equations in a "random" finitely generated subgroup of the braid group, we show how to find a small ordered list of elements in the subgroup, which contains a solution to the equations with a significant probability. Moreover, with a significant probability, the solution will be the first in the list.…
We present sparse tree-based and list-based density estimation methods for binary/categorical data. Our density estimation models are higher dimensional analogies to variable bin width histograms. In each leaf of the tree (or list), the density is constant, similar to the flat density within the bin of a histogram. His…
Signature kernel scoring rule improves weather forecasting by capturing temporal and spatial dependencies.
problem Lack of suitable scoring rules for probabilistic weather forecasting.
method Reframe weather variables as continuous paths using iterated integrals (signature kernels) to capture temporal and spatial dependencies.
result Signature kernel scoring rule outperforms conventional methods in weather forecasting, especially for long-term forecasts.
New tree splitting criteria improve probabilistic predictions.
problem Improving tree-based nonparametric predictive distributions.
method Using proper scoring rules for tree splitting criteria.
result Trees with new splitting criteria produce better predictive distributions.
Unsupervised ranking faces one critical challenge in evaluation applications, that is, no ground truth is available. When PageRank and its variants show a good solution in related subjects, they are applicable only for ranking from link-structure data. In this work, we focus on unsupervised ranking from multi-attribute…
Proposes a method to reconcile count time series forecasts.
problem No formal framework for probabilistic reconciliation of count time series.
method Generalizes Bayes' rule for reconciling real-valued and count variables.
result Improves forecast accuracy for count variables compared to Gaussian reconciliation.
Investigates principles of generalization in list learning, refutes sample compression conjecture.
problem Determining applicability of classical principles in list PAC learning.
method Examines uniform convergence and sample compression in list PAC learning.
result Sample compression fails in list PAC learning, refutes conjecture.
Study explores reinforcement learning in a complex game environment, analyzing rule inference and policy learning.
problem Learning optimal policies in environments with hidden rules.
method Investigated using the Game Of Hidden Rules (GOHR) environment, employing Feature-Centric and Object-Centric state representations with a Transformer-based A2C algorithm.
result Transformer-based A2C models outperform traditional methods in GOHR, demonstrating the effectiveness of representation strategies.
A method for concept-based learning using probabilistic inference and expert rules.
problem Concept-based learning with limited training data.
method Divide images into patches, transform into embeddings, cluster, and use frequentist inference to find concepts.
result FI-CBL outperforms concept bottleneck model in small data scenarios.
Study identifies regions where scoring rules reliably detect forecast errors.
problem Insufficient reliability of scoring rules in evaluating multivariate probabilistic forecasts.
method Systematic finite-sample analysis of proper scoring rules on synthetic and real-world data.
result Identified regions of reliability for scoring rules in time-series forecasting.
Abstraction and realization are bilateral processes that are key in deriving intelligence and creativity. In many domains, the two processes are approached through rules: high-level principles that reveal invariances within similar yet diverse examples. Under a probabilistic setting for discrete input spaces, we focus …
A new probabilistic model for semi-supervised learning unifies various methods.
problem Combining different aspects of data distribution for semi-supervised learning.
method A probabilistic model that interprets and improves upon existing SSL methods.
result The model unifies various SSL methods and extends to neuro-symbolic learning.
The study finds infinitely many Lefschetz pencils on ruled surfaces with negative Euler characteristic.
problem Finding Lefschetz pencils on ruled surfaces with negative Euler characteristic.
method Building Lefschetz fibrations on a blow-up of the surface via partial conjugation, then showing compatibility with a symplectic form.
result Ruled surfaces with negative Euler characteristic admit infinitely many inequivalent Lefschetz pencils and fibrations.
RIPE is a novel deterministic and easily understandable prediction algorithm developed for continuous and discrete ordered data. It infers a model, from a sample, to predict and to explain a real variable Y given an input variable X∈X (features). The algorithm extracts a sparse set of hyperrectangles $…
A new gradient boosting method improves interpretability of probabilistic models.
problem Learning interpretable yet accurate probabilistic models with limited rule complexity.
method A new objective function that measures the angle between risk gradient and condition output vector projection.
result Significantly improves comprehensibility/accuracy trade-off of fitted ensemble.
Improves generative models for cost-sensitive decisions.
problem Generative models lack awareness of decision costs.
method Integrates a decision loss into the training objective.
result Improves cost-sensitive forecast accuracy.
Proposes a cost-sensitive method to generate probabilistic SVM outputs.
problem Generating probabilistic SVM outputs efficiently and cost-effectively.
method Cost-sensitive ensemble SVM with bootstrap probability estimation.
result Improves performance on imbalanced datasets and outperforms benchmarks.
Spiking Neural Networks (SNNs) are distributed trainable systems whose computing elements, or neurons, are characterized by internal analog dynamics and by digital and sparse synaptic communications. The sparsity of the synaptic spiking inputs and the corresponding event-driven nature of neural processing can be levera…
Unified theory for neural scaling laws in hierarchically compositional data.
problem Understanding neural scaling laws in hierarchically compositional data.
method Probabilistic context-free grammars and power-law distributed production rules.
result Unified learning curve behavior for classification and next-token prediction tasks.
The article reviews scoring rules for estimating and evaluating forecasts.
problem Evaluating probabilistic forecasts and estimating probability distributions.
method Mathematical foundations and characterization of scoring rules.
result Important families of scoring rules and their applications in statistics and machine learning.
Conventional techniques for supervised classification constrain the classification rules considered and use surrogate losses for classification 0-1 loss. Favored families of classification rules are those that enjoy parametric representations suitable for surrogate loss minimization, and low complexity properties suita…
This research focuses on invariant probabilistic predictions, showing they are not robust under distribution shifts.
problem The challenge of creating robust probabilistic predictions that remain consistent under distribution shifts.
method A causality-inspired framework to investigate invariance and robustness of probabilistic predictions with respect to proper scoring rules.
result Arbitrary distribution shifts do not admit invariant and robust probabilistic predictions, unlike point predictions.
SIRUS creates interpretable rules from random forests for regression.
problem Lack of interpretability in complex machine learning models.
method Random forest with rule extraction for stability and simplicity.
result SIRUS produces stable and interpretable rule sets.
Designs a Cellular Automata rule for forming touching loop patterns.
problem Forming stable touching loop patterns in a 2D grid.
method Developed a Cellular Automata rule that uses templates to cover the space and match patterns.
result The rule successfully evolves stable touching loop patterns in a 2D grid.
In this paper we formulate a probabilistic model for class-specific discriminant subspace learning. The proposed model can naturally incorporate the multi-modal structure of the negative class, which is neglected by existing class-specific methods. Moreover, it can be directly used to define a class-specific probabilis…
Statistical relational models provide compact encodings of probabilistic dependencies in relational domains, but result in highly intractable graphical models. The goal of lifted inference is to carry out probabilistic inference without needing to reason about each individual separately, by instead treating exchangeabl…
The paper proposes a method to optimize rule-based models for better accuracy and interpretability.
problem Developing rule-based models for regression and classification with better accuracy and interpretability.
method Column generation to optimize over an exponentially large space of rules, using integer programming or a heuristic.
result The proposed methods achieve better accuracy-complexity trade-offs than existing rule ensemble algorithms.
Decision makers, such as doctors and judges, make crucial decisions such as recommending treatments to patients, and granting bails to defendants on a daily basis. Such decisions typically involve weighting the potential benefits of taking an action against the costs involved. In this work, we aim to automate this task…
SIRUS creates interpretable rules from random forests for manufacturing.
problem Lack of interpretability in complex models for critical decisions.
method SIRUS is a classification algorithm based on random forests that produces a simple list of rules.
result SIRUS achieves stability and accuracy comparable to random forests.