New methods for assessing and visualizing feature groups in machine learning models.
problem Lack of methods for interpreting feature groups in machine learning models.
method Permutation-based, refitting, and Shapley-based techniques for grouped feature importance. Introduced a sequential procedure for identifying stable feature combinations. Developed a combined features effect plot.
result Effective methods for assessing and visualizing the importance and effect of feature groups in machine learning models.
GOLFS selects features for clustering by combining global and local information.
problem Feature selection for high-dimensional clustering without labels.
method Combines global and local information via manifold learning and regularized self-representation.
result Improves feature selection and clustering accuracy.
Recommendation systems and computing advertisements have gradually entered the field of academic research from the field of commercial applications. Click-through rate prediction is one of the core research issues because the prediction accuracy affects the user experience and the revenue of merchants and platforms. Fe…
A novel weighted feature selection method using fuzzy sets improves classification accuracy and stability.
problem Improving feature selection accuracy and stability in machine learning models.
method Combination of four feature selection methods using fuzzy sets and bootstrap.
result Our method achieved significantly higher stability than individual methods.
Unified framework for calculating Shapley values with correlated features.
problem Inaccurate Shapley values due to multicollinearity.
method Unified framework with matrix formulation and mathematical proof.
result Shapley values become independent of feature correlations.
Simplifies forecast combination by using diversity of out-of-sample forecasts.
problem Estimating optimal weights for forecast combinations is challenging.
method Use out-of-sample forecasts to extract features and calculate weights for forecast combination.
result Achieves superior forecasting performance in point forecasts and prediction intervals.
Noisy Feature Mixup improves model robustness with noise-perturbed convex combinations.
problem Improving model robustness against data perturbations.
method Noise-perturbed convex combinations of pairs of data points in input and feature space.
result Improved model robustness and favorable trade-offs between accuracy and robustness.
This paper makes kernels interpretable for wide feature matrices.
problem Loss of interpretability in kernel methods for wide feature matrices.
method Re-express kernel solution as a linear combination of original features with a special metric.
result Interpretable kernel solutions for wide feature matrices are possible.
SIAN bridges simple models to neural networks by identifying necessary feature combinations.
problem The gap between simple models and powerful neural networks in performance.
method Feature interaction detection and sparse selection algorithm.
result Competitive performance across multiple tabular datasets with optimal tradeoff.
A novel approach combines feature importance scores with deep learning for forex price prediction.
problem Improving forex price prediction using deep learning models.
method Feature importance recap combined with stacking models.
result Proper feature selection significantly improves model performance.
In recent work, it was shown that combining multi-kernel based support vector machines (SVMs) can lead to near state-of-the-art performance on an action recognition dataset (HMDB-51 dataset). This was 0.4\% lower than frameworks that used hand-crafted features in addition to the deep convolutional feature extractors. I…
Various factorization-based methods have been proposed to leverage second-order, or higher-order cross features for boosting the performance of predictive models. They generally enumerate all the cross features under a predefined maximum order, and then identify useful feature interactions through model training, which…
This short report describes our submission to the ISIC 2018 Challenge in Skin Lesion Analysis Towards Melanoma Detection for Task1 and Task 3. This work has been accomplished by a team of researchers at the University of Dayton Signal and Image Processing Lab. Our proposed approach is computationally efficient are comb…
We present a novel, log-radius profile representation for convex curves and define a new operation for combining the shape features of curves. Unlike the standard, angle profile-based methods, this operation accurately combines the shape features in a visually intuitive manner. This method have implications in shape an…
This paper proposes a speech emotion recognition method based on speech features and speech transcriptions (text). Speech features such as Spectrogram and Mel-frequency Cepstral Coefficients (MFCC) help retain emotion-related low-level characteristics in speech whereas text helps capture semantic meaning, both of which…
Stock price prediction is a challenging task, but machine learning methods have recently been used successfully for this purpose. In this paper, we extract over 270 hand-crafted features (factors) inspired by technical and quantitative analysis and tested their validity on short-term mid-price movement prediction. We f…
Optimized fuzzy entropy framework improves feature selection and classification performance.
problem Improving feature selection and classification in fuzzy entropy frameworks.
method Implemented and compared combinations of ideal vectors, maximal similarity classifiers, and fuzzy entropy functions.
result Optimized combination of ideal vector, similarity classifier, and fuzzy entropy function achieved the most stable performance for all three datasets.
The article presents a method that improves the quality of classification of objects described by a combination of known and unknown features. The method is based on modernized Informational Neurobayesian Approach with consideration of unknown features. The proposed method was developed and trained on 1500 text queries…
Forecasting the movements of stock prices is one the most challenging problems in financial markets analysis. In this paper, we use Machine Learning (ML) algorithms for the prediction of future price movements using limit order book data. Two different sets of features are combined and evaluated: handcrafted features b…
IUS framework predicts EUR/USD exchange rate with improved accuracy.
problem Accurate forecasting of EUR/USD exchange rate.
method Combines large language models for sentiment analysis, deep learning for forecasting, and feature selection.
result Optuna-optimized Bi-LSTM model reduces MAE and RMSE by 10.69% and 9.56% respectively.
The group affect or emotion in an image of people can be inferred by extracting features about both the people in the picture and the overall makeup of the scene. The state-of-the-art on this problem investigates a combination of facial features, scene extraction and even audio tonality. This paper combines three addit…
TGNN combines GNN and SMM for better trading network predictions.
problem Predicting asset prices in trading networks with structural impact factors.
method Combines GNN and SMM for asset price prediction.
result TGNN outperforms existing methods in prediction accuracy.
State-of-the-art link prediction utilizes combinations of complex features derived from network panel data. We here show that computationally less expensive features can achieve the same performance in the common scenario in which the data is available as a sequence of interactions. Our features are based on social vec…
Microwave-based breast cancer detection has been proposed as a complementary approach to compensate for some drawbacks of existing breast cancer detection techniques. Among the existing microwave breast cancer detection methods, machine learning-type algorithms have recently become more popular. These focus on detectin…
Generalized linear models with nonlinear feature transformations are widely used for large-scale regression and classification problems with sparse inputs. Memorization of feature interactions through a wide set of cross-product feature transformations are effective and interpretable, while generalization requires more…
Combines datasets to improve model fitting with small sample sizes.
problem Improving model performance with limited samples from at least one dataset.
method Proposes a novel framework called Combine datasets based on Imputation (ComImp) and PCA-ComImp for combining datasets with missing data.
result Significant improvement in model accuracy, especially with small datasets and when combined with transfer learning.
CTRF combines logged data and randomized experiments for robust prediction.
problem Robust prediction models to handle distributional shifts between training and testing data.
method CTRF uses existing training data and a small amount of randomized experiment data to train a robust model.
result CTRF produces robust predictions and outperforms baseline methods in the presence of feature shifts.
A two-layer classifier improves smartphone transportation mode recognition.
problem Improving accuracy of transportation mode classification.
method Two-layer hierarchical classifier combining time and frequency domain features.
result Maximum classification accuracy of 97.02%.
Despite the success of deep learning in domains such as image, voice, and graphs, there has been little progress in deep representation learning for domains without a known structure between features. For instance, a tabular dataset of different demographic and clinical factors where the feature interactions are not gi…
The paper improves prediction and testing for signals from a linear combination of translated features with Gaussian noise.
problem Predicting and testing signals from a linear combination of translated features with varying scale parameter and Gaussian noise.
method Extends previous off-the-grid prediction results, improves minimal distance between features, proposes a goodness-of-fit test with upper bounds.
result Upper bounds on the minimax separation rate match those for the high-dimensional linear model, matching the lower bound.
Feature selection can efficiently identify the most informative features with respect to the target feature used in training. However, state-of-the-art vector-based methods are unable to encapsulate the relationships between feature samples into the feature selection process, thus leading to significant information los…
New method reconstructs missing variables in time series using autoencoders and automatic differentiation.
problem Reconstruct missing variables in time series with flexible input and output combinations.
method Train an autoencoder with all features, optimize missing variables as inputs, and use automatic differentiation.
result Flexible input and output combinations can be achieved without retraining the autoencoder.
An important step in speaker verification is extracting features that best characterize the speaker voice. This paper investigates a front-end processing that aims at improving the performance of speaker verification based on the SVMs classifier, in text independent mode. This approach combines features based on conven…
The use of features extracted using a deep convolutional neural network (CNN) combined with a writer-dependent (WD) SVM classifier resulted in significant improvement in performance of handwritten signature verification (HSV) when compared to the previous state-of-the-art methods. In this work it is investigated whethe…
Model precision in a classification task is highly dependent on the feature space that is used to train the model. Moreover, whether the features are sequential or static will dictate which classification method can be applied as most of the machine learning algorithms are designed to deal with either one or another ty…
Latent feature models (LFM)s are widely employed for extracting latent structures of data. While offering high, parameter estimation is difficult with LFMs because of the combinational nature of latent features, and non-identifiability is a particularly difficult problem when parameter estimation is not unique and ther…
SCOPE-FE improves feature engineering efficiency for high-dimensional datasets.
problem Expanding and reducing feature space in tabular learning becomes computationally expensive with increased dimensionality.
method SCOPE-FE controls the search space by regulating operator and feature-pair spaces, using OperatorProbing and FeatureClustering.
result SCOPE-FE reduces feature engineering time while maintaining competitive predictive performance.
Ensemble learning that can be used to combine the predictions from multiple learners has been widely applied in pattern recognition, and has been reported to be more robust and accurate than the individual learners. This ensemble logic has recently also been more applied in feature selection. There are basically two st…
This paper presents a framework for efficiently learning feature selection policies which use less features to reach a high classification precision on large unstructured data. It uses a Deep Convolutional Autoencoder (DCAE) for learning compact feature spaces, in combination with recently-proposed Reinforcement Learni…
ALICE combines feature selection and inter-rater agreeability for ML model insights.
problem Improving interpretability of black box machine learning models.
method Integrates feature selection and inter-rater agreeability into a user-friendly Python library.
result Initial experiments on customer churn modeling show promising insights.
AdaEnsemble learns adaptive feature interactions for CTR prediction.
problem Learning feature interactions for CTR prediction in recommender systems and Ads ranking.
method AdaEnsemble is a Sparsely-Gated Mixture-of-Experts (SparseMoE) architecture that dynamically selects feature interaction depth.
result AdaEnsemble achieves better prediction accuracy and inference efficiency compared to state-of-the-art models.
Fair MP-Boost improves fairness and interpretability in boosting methods.
problem Improving fairness and interpretability in boosting methods.
method Fair MP-Boost uses adaptive sampling of minipatches to balance accuracy and fairness.
result Fair MP-Boost enhances fairness and accuracy while providing interpretable feature importance.
New method combines spectral and sparse methods for Gaussian processes.
problem Efficiently fitting Gaussian processes to large datasets.
method Orthogonally decoupled variational Fourier features.
result Competitive performance on synthetic and real-world data.
The generalized partially linear additive model (GPLAM) is a flexible and interpretable approach to building predictive models. It combines features in an additive manner, allowing each to have either a linear or nonlinear effect on the response. However, the choice of which features to treat as linear or nonlinear is …
xDeepInt learns both vector-wise and bit-wise feature interactions.
problem Learning feature interactions for CTR prediction and recommendation.
method Polynomial Interaction Network (PIN) architecture with subspace-crossing mechanism.
result xDeepInt outperforms state-of-the-art models in CTR prediction and recommendation.
Advertising and feed ranking are essential to many Internet companies such as Facebook and Sina Weibo. Among many real-world advertising and feed ranking systems, click through rate (CTR) prediction plays a central role. There are many proposed models in this field such as logistic regression, tree based models, factor…
To date, attribute discretization is typically performed by replacing the original set of continuous features with a transposed set of discrete ones. This paper provides support for a new idea that discretized features should often be used in addition to existing features and as such, datasets should be extended, and n…
MBExplainer provides explanations for models combining graph embeddings and tabular features.
problem Explaining models using a mix of graph embeddings and tabular features.
method Model-agnostic approach using Shapley values and Monte Carlo Tree Search.
result MBExplainer efficiently finds human-readable explanations for model predictions.