Study proposes Local Linear Encoding for better feature discretization.
problem Improving feature discretization for numeric data.
method Theoretical analysis and Local Linear Encoding (LLE) method.
result LLE outperforms conventional methods with fewer parameters.
A novel graph spectral method for mixed categorical and numerical data.
problem Feature learning for mixed data types (numerical and categorical).
method Graph spectral decomposition of the graph Laplacian to model probabilistic dependence structure.
result Increased separability and clusterability of observations in the transformed feature space.
Conventional mutual information (MI) based feature selection (FS) methods are unable to handle heterogeneous feature subset selection properly because of data format differences or estimation methods of MI between feature subset and class label. A way to solve this problem is feature transformation (FT). In this study,…
New method evaluates feature interactions using orthogonal variance decomposition.
problem Feature selection fails to account for interactions between features.
method Orthogonal variance decomposition to evaluate feature subsets considering interactions.
result Our method accurately identifies relevant features and improves model accuracy.
NGRC shows numerical instabilities with short lags and high-degree polynomials.
problem Numerical instabilities in NGRC feature matrix.
method Combining numerical linear algebra and dynamical systems theory, we study feature matrix conditioning. We evaluate different numerical algorithms for solving the regularized least-squares problem.
result SVD-based training achieves accurate forecasts without regularization, preferable for short lags and high-degree polynomials.
New method recovers sparse vectors from random sinusoidal features.
problem Recovering sparse vectors from random sinusoidal features.
method Proposes a numerically stable algorithm for sparse vector reconstruction.
result Sparse vectors can be reliably recovered from random sinusoidal features.
Improved BIRCH clustering method to avoid numeric issues.
problem Numeric instability in BIRCH clustering.
method Introduced a new cluster feature to replace the sum of squares, avoiding catastrophic cancellation.
result The new method is more numerically stable and efficient.
Enhances FM models for numerical features using function basis encoding.
problem Challenges in incorporating numerical features into FM variants.
method Encoding numerical features into a vector of function values for learning segmentized functions.
result Improves model accuracy by learning segmentized functions of numerical features.
New method improves kernel approximation for larger datasets.
problem Efficiently approximate kernel functions for large datasets.
method Monte Carlo integration for numerical approximation of kernel functions.
result Improved convergence behavior and empirical support for better kernel estimates.
Visualizes deep network feature contributions in images.
problem Understanding information flow in deep networks.
method Forward-Backward approach for feature visualization.
result Numerical results show benefits over existing methods.
Optimal decision trees are constructed via integer programming for better accuracy and interpretability.
problem Overfitting and loss of interpretability in decision trees.
method Mixed integer programming formulation to construct optimal decision trees of a prespecified size, considering categorical and numerical features.
result Very good accuracy can be achieved with small trees using moderately-sized training sets.
Bayesian Topic Regression models causal inference with text and numerical data.
problem Causal inference using observational text data with both text and numerical confounders.
method Combines supervised Bayesian topic model with Bayesian regression framework, respecting the Frisch-Waugh-Lovell theorem.
result Joint approach recovers ground truth with lower bias than benchmarks, superior prediction results compared to separate approaches.
Researchers quantify the relationship between feature depth and performance in deep neural networks.
problem Understanding how depth affects feature extraction and generalization in deep neural networks.
method Adaptive analysis of feature-depth trade-offs in deep nets, proving optimal generalization performance.
result Optimal generalization performance achieved through empirical risk minimization on deep nets.
Study examines neural networks for feature extraction and their impact on machine learning models.
problem Improving feature extraction for better machine learning model performance.
method Used neural networks to extract features from images and numeric data, then compared these features with SVMs and KNNs.
result Neural network-extracted features significantly enhance SVM and KNN performance in many cases.
The paper analyzes how deep models memorize spurious features.
problem Understanding how deep models memorize spurious features in training data.
method Characterizes spurious feature memorization via model stability and feature alignment.
result Memorization of spurious features weakens as generalization capability increases.
A new method corrects weight values to improve treatment effect estimation.
problem Estimating heterogeneous treatment effects in high-dimensional data with sample selection bias.
method Differentiable Pareto-Smoothed Weighting (DPSW) framework.
result Our method outperforms existing methods in treatment effect estimation.
Multivariate splines linked to infinitely-wide neural networks with improved numerical performance.
problem Understanding the relationship between multivariate splines and neural networks.
method Showed multivariate splines can be represented as random features in infinitely-wide neural networks with a homogeneous activation function.
result The function space of multivariate splines is a Sobolev space on a Euclidean ball with explicit norm bounds on derivatives.
This work proposes a geometric approach to equivariant message passing on Riemannian manifolds.
problem Efficiently processing data on Riemannian manifolds with equivariance.
method Geometric insight into equivariant message passing on Riemannian manifolds, using an equivariant embedding and diffusion process.
result A new class of equivariant GNNs on Riemannian manifolds.
New methods reduce extrapolation errors in feature importance.
problem Flawed feature importance methods using unrestricted permutations lead to extrapolation errors.
method Three new approaches: conditional model reliance, Knockoffs with Gaussian transformation, and restricted ALE plot designs.
result Theoretical and numerical results show our strategies reduce/eliminate extrapolation.
Feature selection is a technique to screen out less important features. Many existing supervised feature selection algorithms use redundancy and relevancy as the main criteria to select features. However, feature interaction, potentially a key characteristic in real-world problems, has not received much attention. As a…
A new method generates mixed-type features in tabular data with improved realism and accuracy.
problem Generating mixed-type features combining discrete and continuous data is challenging.
method A cascaded approach: first generates low-resolution categorical and coarse numerical features, then uses these in a high-resolution flow matching model.
result The model significantly improves detection scores, generating more realistic samples and capturing distributional details.
Proposes binarsity for linear models with one-hot encoded features.
problem Large-scale linear supervised learning with many continuous features.
method Combines one-hot encoding with binarsity penalization.
result Induces piecewise constant and eventually block sparse model weights.
New SVM feature selection methods improve wafer testing accuracy.
problem Need for improved SVM feature selection methods.
method Entropy measurement and K-medoid clustering for feature selection.
result One-class SVM's enhance wafer testing accuracy.
Regularized target encoding beats traditional methods for high cardinality features in ML.
problem Efficiently encoding high cardinality categorical variables for ML algorithms.
method Regularized target encoding compared to traditional encodings like integer and one-hot encoding.
result Regularized target encoding consistently provided the best results in a large-scale benchmark experiment.
Visual design improves financial data classification accuracy.
problem Improving financial decision-making through better data representation.
method Comparing numeric vs visual data representations in supervised classification.
result Visual transformation of numeric data leads to higher predictability.
This paper reviews methods for feature selection and extraction in pattern analysis.
problem Complex raw data require feature selection or extraction for better discrimination or representation.
method Reviews different methods of feature selection and extraction.
result Compares various methods of feature selection and extraction.
The paper solves complex swing option pricing equations with numerical methods.
problem Valuation of swing options with jumps under a mean-reverting model.
method Proposes second-order numerical methods to solve PIDEs convection-dominated and with nonlocal integral terms.
result Numerical methods confirm second-order convergence behavior.
New metrics reveal oversmoothing in GNNs more accurately than traditional methods.
problem Oversmoothing in graph neural networks reduces model performance.
method Rank-based metrics to measure oversmoothing in GNNs.
result Rank-based metrics consistently capture oversmoothing, while energy-based metrics often fail.
The goal of this paper is to assess the utility of Reduced-Order Models (ROMs) developed from 3D physics-based models for predicting transient thermal power output for an enhanced geothermal reservoir while explicitly accounting for uncertainties in the subsurface system and site-specific details. Numerical simulations…
Language models can predict numeric values as strings.
problem Regression tasks with numeric predictions.
method Causal sequence decoding models trained for next-token prediction.
result Decoder-based heads perform as well as standard heads in numeric regression tasks.
Anisotropic neural network selects relevant features from datasets.
problem Reduction of irrelevant features improves model interpretability and performance.
method General Regression Neural Network with an anisotropic Gaussian Kernel.
result The method robustly selects features from simulated and real-world datasets.
New method scales features for better clustering.
problem Irregular features disrupt classification.
method Spectral clustering with modified feature scales.
result Outperforms existing methods in experiments.
Machine learning models predict the behavior of negatively buoyant jets from wastewater.
problem Minimizing harmful effects of negatively buoyant jets during wastewater discharge.
method Training machine learning models (ANN, XGBoost, CatBoost, LightGBM) on OpenFOAM simulations and experimental data.
result Artificial Neural Network provided the best prediction with R2 0.98 and RMSE 0.28.
Slow feature analysis (SFA) is a method for extracting slowly varying features from a quickly varying multidimensional signal. An open source Matlab-implementation sfa-tk makes SFA easily useable. We show here that under certain circumstances, namely when the covariance matrix of the nonlinearly expanded data does not …
Introduces greedy feature selection for classifier-dependent feature ranking.
problem Feature selection for classification tasks.
method Greedy feature selection, identifying the most important feature at each step based on the selected classifier.
result Theoretical and numerical benefits of greedy feature selection.
A novel Bayesian computation method using importance weighting improves numerical stability and performance.
problem Bayesian computation stability and performance issues.
method Nonparametric approach via feature means, importance weighting, and kernel Bayes' rule.
result Importance weighted kernel Bayes' rule yields superior numerical stability and performance.
New method detects geologic features from seismic data more efficiently.
problem Challenges in detecting subsurface geologic features from limited seismic data.
method Data-driven approach using randomized machine learning and Nyström method.
result Significant speed-up in computational efficiency with comparable accuracy.
The paper proposes a method to analyze categorical feature interactions in large datasets using graph covariance and LLMs.
problem Analyzing complex datasets with numerous categorical features and timestamps.
method Binarization of categorical features using one-hot encoding, computation of graph covariance, identifying significant feature pairs, and using LLMs to generate explanations.
result The method identifies meaningful feature pairs and potential data stories underlying categorical feature interactions.
SSFI identifies key features affecting a single data point's prediction.
problem Understanding feature importance for a single data point.
method Interpretable algorithm using Random Forest's prediction path.
result Relative ranking of features impacting a data point's prediction.
A new method uses Legendre multiwavelets to price discrete double barrier options efficiently.
problem Pricing discrete double barrier options efficiently with reduced CPU time.
method Approximating recursive solutions of the heat equation using Legendre multiwavelets and operational matrix form.
result The method significantly reduces CPU time and is efficient for increasing monitoring dates.
A conjecturing machine uncovers feature relationships in data.
problem Discovering nonlinear and boolean relationships among features.
method Proposes a conjecturing machine that suggests feature relationships using bounds and expressions.
result Reveals true underlying relationships in datasets.
A new method for dynamic feature selection outperforms existing approaches.
problem Sequentially selecting features based on current information in machine learning.
method Greedy selection of features based on conditional mutual information, combined with a learning approach for optimization.
result The method outperforms existing feature selection methods in experiments.
SAGE quantifies feature importance in machine learning models.
problem Understanding the role of individual features in complex models.
method Formalizing predictive power through model-based and universal measures, and introducing SAGE for efficient calculation.
result SAGE assigns more accurate feature importance values than other methods.
R2N learns interpretable rules and literals from numerical features.
problem Lack of expressive vocabulary in rule-based decision models.
method Relational Rule Network (R2N) learns literals and rules end-to-end.
result Learned literals improve prediction accuracy and rule conciseness.
A new method for efficient nonlinear process monitoring using random Bernoulli features.
problem High computational demands and real-time responsiveness in online monitoring systems.
method Random Bernoulli principal component analysis to capture nonlinear patterns efficiently.
result The proposed methods offer excellent scalability and reduced computational complexity.
New framework investigates how fake features affect model generalization.
problem Model misspecification with fake features and their impact on generalization.
method Non-asymptotic high-probability bound on ridge regression generalization error.
result Trade-off between fake features and optimal ridge parameter.
In subgroup discovery, also known as supervised pattern mining, discovering high quality one-dimensional subgroups and refinements of these is a crucial task. For nominal attributes, this is relatively straightforward, as we can consider individual attribute values as binary features. For numerical attributes, the task…
We propose a tree regularization framework, which enables many tree models to perform feature selection efficiently. The key idea of the regularization framework is to penalize selecting a new feature for splitting when its gain (e.g. information gain) is similar to the features used in previous splits. The regularizat…