Two new feature selection algorithms improve on RFE.
problem Optimal feature selection for faster and more accurate models.
method Fibonacci and k-Subsecting Recursive Feature Elimination.
result Faster feature selection with comparable predictive performance.
A new feature selection method using random forest and Kolmogorov filter.
problem Ultra-high dimensional data feature selection.
method Fused Kolmogorov filter with random forest based recursive feature elimination.
result Selection and L2 consistency under weak conditions. Dimensionality reduction is one of the key issues in the design of effective machine learning methods for automatic induction. In this work, we introduce recursive maxima hunting (RMH) for variable selection in classification problems with functional data. In this context, variable selection techniques are especially a…
Enhances detection of adverse drug events using diverse healthcare record data.
problem Detecting adverse drug events from mixed data types in electronic health records.
method Aggregate diagnosis codes, drug codes, and lab measurements; use recursive feature selection.
result Significant improvement in AUC using additional features, statistically significant.
Machine learning finds new natural laws from noisy data.
problem Discovery of natural laws relies on human inspiration.
method Recursive-LASSO-based symbolic regression (RLS) method.
result Data-driven formulation of natural laws from noisy data.
RocketStack integrates predictions from multiple base learners using a recursive stacking architecture up to ten levels.
problem Feature redundancy, complexity, and computational burden in deep stacking.
method Level-aware recursive stacking with pruning and compression techniques.
result Increasing accuracy with depth and outperforming standalone ensembles at later levels.
A new feature selection method for semi-supervised learning with imperfect labels.
problem Feature selection for semi-supervised learning with imperfectly labeled data.
method Genetic algorithm for proposing feature subsets, probabilistic error model for mislabeling, multi-class C-bound selection criterion.
result Empirical results show the effectiveness of the proposed framework compared to state-of-the-art approaches.
This paper uses MIO to select features for kernel SVM classification.
problem Feature selection for kernel SVM classification.
method Mixed-integer optimization (MIO) for feature subset selection.
result The MIO approach can often outperform linear-SVM-based methods in prediction performance.
ERM uses energy-based selection to improve recursive reasoning.
problem Lack of principled inference mechanism in recursive models.
method Energy-guided Recursive Model (ERM) introduces Hopfield energies for trajectory selection.
result ERM achieves optimal solutions on various puzzles.
Due to its linear complexity, naive Bayes classification remains an attractive supervised learning method, especially in very large-scale settings. We propose a sparse version of naive Bayes, which can be used for feature selection. This leads to a combinatorial maximum-likelihood problem, for which we provide an exact…
In recent years, state-of-the-art methods for supervised learning have exploited increasingly gradient boosting techniques, with mainstream efficient implementations such as xgboost or lightgbm. One of the key points in generating proficient methods is Feature Selection (FS). It consists in selecting the right valuable…
Sparse linear regression, which entails finding a sparse solution to an underdetermined system of linear equations, can formally be expressed as an l0-constrained least-squares problem. The Orthogonal Least-Squares (OLS) algorithm sequentially selects the features (i.e., columns of the coefficient matrix) to greedil…
Paper identifies how neural networks learn features.
problem Understanding how neural networks automatically select features.
method Deep Neural Feature Ansatz, implementing average gradient outer product.
result Recursive Feature Machines achieve state-of-the-art performance.
A new variable importance measure for DRFs detects broader impacts on output distributions.
problem Estimating full conditional distributions of multivariate outputs given inputs.
method Based on the drop and relearn principle and MMD distance.
result Consistent and high-performing variable importance measure for DRFs.
In machine learning, Feature Selection (FS) is a major part of efficient algorithm. It fuels the algorithm and is the starting block for our prediction. In this paper, we present a new method, called Optimal Coordinate Ascent (OCA) that allows us selecting features among block and individual features. OCA relies on coo…
A new method uses Shapley values to select important features for classification.
problem Feature selection for improving classification models.
method Classification game with Shapley value apportioning of hinge loss.
result Threshold 0 on SVEA value identifies significant features.
Proposes a two-stage sector rotation method using machine learning and deep learning.
problem Identifying sectors with high investment attractiveness based on market conditions.
method Two-stage methodology: 1) Predict ETF prices using market indicators and feature selection, 2) Rank sectors based on predicted returns and select top sectors.
result The proposed methodology outperforms equally weighted portfolios and Echo State Networks show outstanding performance.
Tangle machines are topologically inspired diagrammatic models. Their novel feature is their natural notion of equivalence. Equivalent tangle machines may differ locally, but globally they are considered to share the same information content. The goal of tangle machine equivalence is to provide a context-independent me…
Small sample size hinders accurate long-term COVID-19 case predictions.
problem Difficulty in predicting medium and long-term COVID-19 case trends.
method Analysis of machine learning models' performance; feature selection; comparison of different models.
result Simple linear regression models provide reliable 2-week predictions but not beyond.
As the Portable Document Format (PDF) file format increases in popularity, research in analysing its structure for text extraction and analysis is necessary. Detecting headings can be a crucial component of classifying and extracting meaningful data. This research involves training a supervised learning model to detect…
mNARX+ creates accurate surrogate models for complex systems without requiring domain expertise.
problem Creating accurate surrogate models for complex dynamical systems without extensive domain knowledge.
method Data-driven, recursive algorithm that automatically selects temporal features and their causal ordering.
result Automatically identifies critical auxiliary quantities and their order for accurate modeling.
RAMPART ranks top-k features more accurately than existing methods.
problem Accurate ranking of important features in machine learning.
method Adaptive sequential halving strategy combined with ensembling techniques.
result RAMPART achieves the correct top-k ranking with high probability.
We propose a novel approach to model selection for simulator-based statistical models. The proposed approach defines a mixture of candidate models, and then iteratively updates the weight coefficients for those models as well as the parameters in each model simultaneously; this is done by recursively applying Bayes' ru…
New algorithm improves latent variable model estimation.
problem Estimating parameters in latent variable models.
method Jarzynski-adjusted Langevin algorithm (JALA) for SMC methods.
result JALA-EM provides maximum marginal likelihood estimate.
xDeepInt learns both vector-wise and bit-wise feature interactions.
problem Learning feature interactions for CTR prediction and recommendation.
method Polynomial Interaction Network (PIN) architecture with subspace-crossing mechanism.
result xDeepInt outperforms state-of-the-art models in CTR prediction and recommendation.
SRRM improves recursive transport surrogates in the small-discrepancy regime.
problem Insufficient understanding of recursive partitioning methods' statistical behavior and resolution in the small-discrepancy regime.
method Introduced Selective Recursive Rank Matching (SRRM) to improve the resolution of Recursive Rank Matching (RRM).
result SRRM yields a higher-fidelity practical surrogate for the Wasserstein distance at moderate additional computational cost.
GADGET framework decomposes global feature effects using recursive partitioning.
problem Misleading global feature effects when feature interactions are present.
method Generalized additive decomposition of global effects (GADGET) based on recursive partitioning.
result Minimizes interaction-related heterogeneity of local feature effects.
A new probability distribution on full rooted trees helps in model selection.
problem Model selection for full rooted trees is problematic due to their hierarchical structure.
method Assume a prior distribution on full rooted trees, using Bayes decision theory.
result The proposed distribution enables optimal model selection and prevents overfitting.
The paper challenges the use of decision trees for pointwise inference due to slow convergence rates.
problem The slow convergence rates of decision trees in uniform norm, especially with non-vanishing probability.
method Demonstrates the limitations of adaptive recursive partitioning and shows how random forests can improve performance.
result Decision trees can fail to achieve polynomial rates of convergence in uniform norm, even with pruning.
Tab-TRM uses recursive model for insurance pricing on tabular data.
problem Insurance pricing on tabular data.
method Adapts recursive latent reasoning to insurance modeling using a compact, parameter-efficient network.
result Improves insurance pricing accuracy using iterative refinement of latent tokens.
GRANITE unifies feature-based explanation methods to reduce disagreement.
problem Disagreement among feature-based explanation methods.
method GRANITE partitions feature space into regions minimizing interaction and distribution influences.
result Unified and consistent feature explanations.
Recurrent neural networks (RNNs) process input text sequentially and model the conditional transition between word tokens. In contrast, the advantages of recursive networks include that they explicitly model the compositionality and the recursive structure of natural language. However, the current recursive architectur…
This paper studies recursive ensembles driven by Fibonacci updates, improving learning dynamics.
problem Improving learning dynamics in recursive ensemble learning.
method Develops second-order recursive architectures with Fibonacci-type update flows.
result Establishes global convergence conditions and generalization bounds for recursive ensembles.
The paper proposes a new probability distribution for rooted trees.
problem Overfitting in tree selection for statistical models.
method Bayesian approach with a generalized probability distribution for rooted trees.
result Recursive methods to evaluate the probability distribution without approximations.
Various approaches to gene selection for cancer classification based on microarray data can be found in the literature and they may be grouped into two categories: univariate methods and multivariate methods. Univariate methods look at each gene in the data in isolation from others. They measure the contribution of a p…
Self-training in linear models shows a U-shaped test-risk curve due to signal forgetting and denoising.
problem Understanding the dynamics of iterative self-training in high-dimensional linear regression.
method Derivation of deterministic-equivalent recursions for prediction risk and effective noise, analysis of signal forgetting and denoising effects.
result An optimal early-stopping time is determined, and a U-shaped test-risk curve is observed.
Breaking symmetry in training data is key for generalization in feature learning kernels.
problem Grokking in algebraic tasks, where models perform well on training but fail on unseen data.
method Used Recursive Feature Machine (RFM) with AGOP to learn task-relevant features, breaking symmetry in training data.
result Generalization occurs only when symmetry in the training set is broken, and RFM generalizes by recovering underlying invariance group action.
Study adds investment gains and losses to recursive utility model, proving existence and uniqueness of utility process.
problem Existence and uniqueness of utility process in a recursive utility model with investment gains and losses.
method Generalized recursive utility model with constant elasticity of intertemporal substitution and relative risk aversion degree. Proved existence and uniqueness in a specific, finite-state Markovian setting.
result Utility process exists and is unique when agent derives nonnegative gain-loss utility, and non-existent or non-unique otherwise.
Oblique BART improves tree-based predictions.
problem Axis-aligned decision rules in BART can be suboptimal.
method Developed an oblique version of BART using data-adaptive hyperplane partitions.
result Oblique BART outperformed axis-aligned BART and other tree methods on benchmarks.
Recursive Feature Machines show grokking in modular arithmetic without neural networks.
problem Grokking in modular arithmetic tasks.
method Recursive Feature Machines (RFM) with Average Gradient Outer Product (AGOP).
result RFM and neural networks learn block-circulant features to solve modular arithmetic.
Stock selection improved with a novel neural model capturing continuous stock dynamics.
problem Lack of continuous stock dynamics prediction and implicit cross-domain dependencies.
method StockODE, a latent variable model with NRODEs and hierarchical hypergraph for continuous stock volatility and inter-domain dependencies.
result Significantly outperforms baselines, improving Sharpe Ratio by up to 18.57%.
The paper analyzes distances and volumes in lens spaces using recursion and formulas.
problem The problem of moments for distances between points on lens spaces.
method Derivation of recursion relations, formulas for moments and moment generating function, explicit formula for ball volumes.
result Explicit formulas for the volume of balls of all radii in lens spaces.
The current study proposes a dimension reduction method, stepwise support vector machine (SVM), to reduce the dimensions of large p small n datasets. The proposed method is compared with other dimension reduction methods, namely, the Pearson product difference correlation coefficient (PCCs), recursive feature eliminati…
New random forest algorithms for PU learning minimize risk directly.
problem Learning from positive and unlabeled data.
method Recursive greedy risk minimization for decision trees.
result Efficient PU random forest algorithm with robustness and low hyperparameter tuning.
We develop an approach for feature elimination in statistical learning with kernel machines, based on recursive elimination of features.We present theoretical properties of this method and show that it is uniformly consistent in finding the correct feature space under certain generalized assumptions.We present four cas…
Proposes an active RBI framework using Rényi information measures for more informed decision-making.
problem Optimal latent variable estimates in real-time settings with streaming noisy observations.
method Unified inference and query selection steps through Rényi entropy and α-divergence; new objective called Momentum for exploration.
result Analytically demonstrates superior performance compared to conventional methods like mutual information.
Improved action recognition in live videos with hybrid FR-DL method.
problem High computational costs and lack of temporal information in conventional action recognition.
method Automated selection of representative frames, feature extraction, background subtraction, HOG, deep neural network, LSTM, Softmax-KNN classifier.
result Significant improvement in accuracy and speed compared to state-of-the-art methods.
Paper develops efficient recursive learning for multi-channel systems with heterogeneous dynamics.
problem Accurately learning system dynamics in complex, multi-channel systems with nonlinear and noisy data.
method Formulates system as Gaussian process state-space models (GPSSMs), introduces heterogeneous multi-output kernel, and develops recursive inference framework.
result Matches SOTA offline GPSSMs in accuracy with 1/100 runtime, and outperforms SOTA online GPSSMs by 70% in accuracy under noise with 1/20 runtime.