This paper considers generalized linear models using rule-based features, also referred to as rule ensembles, for regression and probabilistic classification. Rules facilitate model interpretation while also capturing nonlinear dependences and interactions. Our problem formulation accordingly trades off rule set comple…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “linear prediction rules”
New bounds prevent degradation in high-dimensional signal estimation.
problem Statistical learning bounds degradation with increasing dimensionality.
method Investigates linear prediction rules under structural assumptions.
result Derives upper and lower bounds on generalization error.
In many predictive decision-making scenarios, such as credit scoring and academic testing, a decision-maker must construct a model that accounts for agents' propensity to "game" the decision rule by changing their features so as to receive better decisions. Whereas the strategic classification literature has previously…
R2N learns interpretable rules and literals from numerical features.
problem Lack of expressive vocabulary in rule-based decision models.
method Relational Rule Network (R2N) learns literals and rules end-to-end.
result Learned literals improve prediction accuracy and rule conciseness.
New framework learns interpretable rule ensembles without sacrificing accuracy.
problem Trade-off between accuracy and interpretability in rule ensembles.
method Introduces local interpretability and a regularizer to promote it, using coordinate descent with local search.
result Learns rule ensembles with fewer rules to explain individual predictions, maintaining comparable accuracy.
Extracting Interpretable Models from Tree Ensembles: Computational and Statistical Perspectivesstat.ML
We propose a method to extract interpretable rules from tree ensembles.
problem Tree ensembles are accurate but hard to interpret.
method Propose an estimator to extract compact sets of decision rules from tree ensembles.
result Our estimator improves accuracy and reveals useful relationships in the data.
SIRUS creates interpretable rules from random forests for regression.
problem Lack of interpretability in complex machine learning models.
method Random forest with rule extraction for stability and simplicity.
result SIRUS produces stable and interpretable rule sets.
Efficiently combines probabilistic predictions using kernel embeddings.
problem Combining multiple probabilistic predictions to improve forecast accuracy.
method Embed predictions into RKHS, optimizes kernel-based scoring rules, finds efficient implementation.
result Generalised linear pool outperforms traditional linear pool in operational wind speed forecasts.
TransINT embeds KGs by preserving implication rules, outperforming existing methods.
problem Embedding KGs while preserving relation implications for better access and analysis.
method Isomorphic intersections of linear subspaces with shared parameters for missing facts.
result Significant performance improvement in link prediction and triple classification.
This paper explores how to choose scoring rules for estimating properties with parametric assumptions.
problem Indirect elicitation of properties with parametric assumptions.
method Developed a framework for choosing proper scoring rules for indirect elicitation, considering constraints and optimal solutions.
result The optimal estimation of the target property changes monotonically with the increase of each weight, and often setting some weights as zero yields the best configuration.
The phenomenon of benign overfitting is one of the key mysteries uncovered by deep learning methodology: deep neural networks seem to predict well, even with a perfect fit to noisy training data. Motivated by this phenomenon, we consider when a perfect fit to training data in linear regression is compatible with accura…
Oblique BART improves tree-based predictions.
problem Axis-aligned decision rules in BART can be suboptimal.
method Developed an oblique version of BART using data-adaptive hyperplane partitions.
result Oblique BART outperformed axis-aligned BART and other tree methods on benchmarks.
New adaptive models improve prediction accuracy with missing data.
problem Improving prediction accuracy with missing data entries.
method Adaptive optimization approach, learning imputation and regression simultaneously.
result 2-10% improvement in out-of-sample accuracy in strongly non-random missing data settings.
Advances rule-based multi-label classification using conformal prediction.
problem Improving accuracy and decision making in multi-label classification.
method Combines conformal prediction with rule-based learning to provide natural conformity scores and calibrate rule assessments.
result Calibrated conformity scores enhance prediction accuracy and decision making.
Optimal allocation of human effort to correct AI assessments in decision-making.
problem How to allocate costly human effort to correct noisy or biased AI-generated assessments.
method Decision-theoretic framework treating AI assessments as signals and human judgments as costly information. Developed estimation procedures under nonparametric and linear models.
result Our approach substantially outperforms LLM-only predictions and achieves performance comparable to full human review while using only 20-30% of the human information.
Paper presents a framework to automatically discover constraints from data.
problem Discovering constraints from data for structured output prediction.
method Formulates structured output prediction as ILP, mines constraints by estimating polytopes of feasible set.
result Successfully identifies feasible sets and constraints for various tasks.
We consider the problem of learning a sparse rule model, a prediction model in the form of a sparse linear combination of rules, where a rule is an indicator function defined over a hyper-rectangle in the input space. Since the number of all possible such rules is extremely large, it has been computationally intractabl…
A new type of distributional regression tree uses soft split rules for better predictive performance.
problem Estimating complete conditional distributions in regression.
method Distributional adaptive soft regression trees using multivariate soft split rules.
result The method outperforms various benchmark methods, especially in complex non-linear interactions.
We consider the following conditional linear regression problem: the task is to identify both (i) a -DNF condition and (ii) a linear rule such that the probability of is (approximately) at least some given bound , and minimizes the loss of predicting the target in the distribution of …
We describe an application of machine learning to the problem of predicting preterm birth. We conduct a secondary analysis on a clinical trial dataset collected by the National In- stitute of Child Health and Human Development (NICHD) while focusing our attention on predicting different classes of preterm birth. We com…
This paper investigates the effects of the "uptick rule" (a short selling regulation formally known as rule 10a-1) by means of a simple stock market model, based on the ARED (adaptive rational equilibrium dynamics) modeling framework, where heterogeneous and adaptive beliefs on the future prices of a risky asset were f…
Improves comparison of F-measures for imbalanced datasets.
problem Comparing F-measures for classification algorithms on imbalanced data.
method Two improvements to existing F-measure comparison methods.
result Enhanced accuracy in comparing F-measures for classification.
Paper tackles linear models with missing values, achieving minimax optimal results.
problem Missing values in real-world data complicate linear model learning.
method Proposes a rigorous setting and a new algorithm leveraging missing data distribution.
result Derives minimax optimal adaptive risk bounds for predictions with missing values.
HRTPP improves TPP interpretability and accuracy in medical event modeling.
problem Lack of interpretability in TPPs for medical event sequences.
method Hybrid-Rule Temporal Point Processes (HRTPP) integrating temporal logic rules and numerical features.
result HRTPP outperforms state-of-the-art interpretable TPPs in predictive performance and clinical interpretability.
This research focuses on invariant probabilistic predictions, showing they are not robust under distribution shifts.
problem The challenge of creating robust probabilistic predictions that remain consistent under distribution shifts.
method A causality-inspired framework to investigate invariance and robustness of probabilistic predictions with respect to proper scoring rules.
result Arbitrary distribution shifts do not admit invariant and robust probabilistic predictions, unlike point predictions.
Proposes a score to compare rule-based algorithms' interpretability.
problem Lack of consensus on interpretability for predictive models.
method Defines a score with three terms: predictivity, stability, and simplicity, each quantified by simple formulas.
result Compares interpretability of rule-based and tree-based algorithms for regression and classification.
In this paper, we study the problem of learning probabilistic logical rules for inductive and interpretable link prediction. Despite the importance of inductive link prediction, most previous works focused on transductive link prediction and cannot manage previously unseen entities. Moreover, they are black-box models …
As we rely more and more on machine learning models for real-life decision-making, being able to understand and trust the predictions becomes ever more important. Local explainer models have recently been introduced to explain the predictions of complex machine learning models at the instance level. In this paper, we p…
This paper improves prediction rule ensembles using model-based data generation.
problem Improving the sparsity and predictive accuracy of prediction rule ensembles.
method The authors use surrogate models to train Lasso regression with data generated by a boosted decision tree ensemble, improving PRE performance.
result The use of surrogacy models can substantially improve the sparsity of PRE while retaining predictive accuracy.
A new gradient boosting method improves interpretability of probabilistic models.
problem Learning interpretable yet accurate probabilistic models with limited rule complexity.
method A new objective function that measures the angle between risk gradient and condition output vector projection.
result Significantly improves comprehensibility/accuracy trade-off of fitted ensemble.
We consider the problem in regression analysis of identifying subpopulations that exhibit different patterns of response, where each subpopulation requires a different underlying model. Unlike statistical cohorts, these subpopulations are not known a priori; thus, we refer to them as cadres. When the cadres and their a…
Paper presents unsupervised calibration for split conformal classification.
problem Inconvenient requirement of labeled calibration samples.
method Uses unsupervised calibration samples alongside supervised training samples.
result Achieves comparable performance to supervised calibration methods.
PureTS uses simple linear models to improve long-term time series forecasting.
problem Improving long-term time series forecasting with complex models.
method Developed PureTS with three pure linear layers.
result PureTS achieves state-of-the-art performance in long sequence prediction tasks.
Prediction Rule Reshapingstat.ML
Two methods are proposed for high-dimensional shape-constrained regression and classification. These methods reshape pre-trained prediction rules to satisfy shape constraints like monotonicity and convexity. The first method can be applied to any pre-trained prediction rule, while the second method deals specifically w…
New tree splitting criteria improve probabilistic predictions.
problem Improving tree-based nonparametric predictive distributions.
method Using proper scoring rules for tree splitting criteria.
result Trees with new splitting criteria produce better predictive distributions.
Work in machine learning and statistics commonly focuses on building models that capture the vast majority of data, possibly ignoring a segment of the population as outliers. However, there does not often exist a good model on the whole dataset, so we seek to find a small subset where there exists a useful model. We ar…
This research adapts scoring rules for training survival models, improving predictive performance.
problem Training survival models with traditional methods struggles with censoring.
method Adapting scoring rules for survival analysis, creating a flexible framework for model training.
result Scoring rules can be successfully incorporated into model training, yielding competitive performance.
Analysis of deep neural networks under various learning rules reveals dynamics of feature and prediction learning.
problem Understanding how different learning rules affect feature and prediction dynamics in deep neural networks.
method Analysis of infinite-width deep networks trained with gradient descent and various learning rules.
result The evolution of the output function is governed by an effective neural tangent kernel (eNTK), which varies depending on the learning rule and training regime.
Cannon, Floyd and Parry have studied the modulus of finite subdivision rules extensively. We investigate the properties of the modulus of subdivision rules with linear and exponential growth at every vertex, using barycentric subdivision and a subdivision rule for the Borromean rings as examples. We show that the subdi…
New scoring rules compare probabilistic top lists in classification.
problem Evaluation of probabilistic top lists in classification.
method Elicitability through symmetric proper scoring rules.
result Brier score provides a well-suited metric for comparison.
Introduces a rule-based Bayesian regression for better uncertainty quantification and expert knowledge integration.
problem Handling regression problems with uncertainty quantification and expert intuition.
method Combines Bayesian inference and rule-based systems for better model performance.
result Improves model performance with better uncertainty quantification and point predictions.
Ensemble methods for supervised machine learning have become popular due to their ability to accurately predict class labels with groups of simple, lightweight "base learners." While ensembles offer computationally efficient models that have good predictive capability they tend to be large and offer little insight into…
Novel approach for creating interpretable classifiers using bilevel optimization of split-rules in NLDTs.
problem Creating highly accurate and easily interpretable classifiers for practical applications.
method Representing classifiers as assemblies of simple mathematical rules using NLDTs with evolutionary bilevel optimization.
result The approach ensures interpretability while achieving high accuracy on various classification problems.
Proposes PEMI for online selective conformal prediction with asymmetric rules.
problem Challenges of handling asymmetric selection mechanisms in online selective conformal prediction.
method PEMI: permutation-based framework for selective conformal prediction with arbitrary asymmetric selection rules.
result Achieves exact selection-conditional coverage for any asymmetric selection mechanism and any prediction model.
New method improves model explainability and accuracy with low computational cost.
problem Improving model explainability and accuracy in classification models.
method Distributionally robust optimization to learn sparse ensembles of rule sets.
result Improves model performance on various metrics compared to competing methods.
The paper develops deep learning models for personalized treatment rules in survival analysis.
problem Deriving optimal treatment rules for bivariate survival outcomes in randomized trials.
method Adaptive prediction-powered learning using deep neural networks and stochastic policies.
result Maximizes joint survival probability beyond fixed time points .
Nonlinear Dimensionality Reduction for Data Visualization: An Unsupervised Fuzzy Rule-based Approachcs.LG
Proposes a fuzzy rule-based method for data visualization.
problem Preserving neighborhood relationships and handling non-linear manifolds in data visualization.
method Uses a first-order Takagi-Sugeno model with clusters and Geodesic c-means clustering for rule generation and parameter estimation.
result Behaves desirably and performs better than or comparable to other methods.
Proposes measures for uncertainty quantification using proper scoring rules.
problem Uncertainty quantification for prediction tasks.
method Decomposes proper scoring rules into divergence and entropy components, tailoring uncertainty quantification to specific tasks.
result Flexibility in uncertainty quantification improves performance in selective prediction and active learning.