Deep nets outperform shallow nets in complex feature realization.
problem Realizing complex data features with deep nets.
method Refined covering number estimates and analysis of approximation rates.
result Deep nets can improve performance without additional capacity costs for complex features.
Deep learning directly extracts features for emotion recognition, outperforming traditional methods.
problem Emotion recognition using traditional feature encoding methods.
method Used deep learning networks to directly encode features for emotion recognition.
result Highest performance in emotion recognition on EmoDB dataset.
Researchers quantify the relationship between feature depth and performance in deep neural networks.
problem Understanding how depth affects feature extraction and generalization in deep neural networks.
method Adaptive analysis of feature-depth trade-offs in deep nets, proving optimal generalization performance.
result Optimal generalization performance achieved through empirical risk minimization on deep nets.
Deep-gKnock uses DNNs to select groups of features with improved interpretability.
problem Feature selection in high-dimensional data with grouping structure.
method Combines deep neural networks with Knockoffs technique for group-feature selection.
result Improves interpretability and accurate gFDR control compared to state-of-the-art methods.
Deep neural networks are a powerful tool for feature learning and extraction given their ability to model high-level abstractions in highly complex data. One area worth exploring in feature learning and extraction using deep neural networks is efficient neural connectivity formation for faster feature learning and extr…
Deep learning models struggle with irrelevant features in survival analysis.
problem Deep learning models suffer from performance deficits when dealing with many irrelevant features in survival analysis.
method Developed novel feature selection methods for deep learning models in survival analysis.
result Substantial performance improvements are achievable with feature selection methods.
Theory explains how deep nets learn features from data.
problem Understanding how deep neural networks learn features from data.
method Developed a noise-nonlinearity phase diagram and a mechanical theory.
result Links feature learning across layers to generalization.
We propose Deep Asymmetric Multitask Feature Learning (Deep-AMTFL) which can learn deep representations shared across multiple tasks while effectively preventing negative transfer that may happen in the feature sharing process. Specifically, we introduce an asymmetric autoencoder term that allows reliable predictors fo…
Study compares random and learned features in deep Bayesian linear models.
problem Understanding how feature learning affects generalization in deep learning.
method Comparing deep random feature models to deep networks with trained layers.
result Random feature models can display double-descent behavior, while deep networks do not.
Hybrid model combines deep features with invertible transformations for accurate predictions and feature modeling.
problem Accurate prediction and feature modeling using deep and invertible transformations.
method Neural hybrid model with a linear model on features from a deep invertible transformation.
result Hybrid model achieves similar accuracy to purely predictive models while maintaining generative capabilities.
Paper uses RL and DCAE to classify large unstructured data with fewer features.
problem Classifying large unstructured data with high precision using fewer features.
method Deep Convolutional Autoencoder (DCAE) for feature learning and Double DQN/Retrace RL algorithms for policy optimization.
result The approach achieves high classification precision with fewer features than traditional methods.
Paper explores deep learning features for complex emotion recognition.
problem Improving emotion recognition accuracy in complex emotions.
method Used pretrained networks (AudioSet Net, VoxCeleb Net, Deep Speech Net) and their deep layer features for emotion recognition.
result Achieved highest F1 score of 0.85 on EmoReact dataset.
Generalized linear models with nonlinear feature transformations are widely used for large-scale regression and classification problems with sparse inputs. Memorization of feature interactions through a wide set of cross-product feature transformations are effective and interpretable, while generalization requires more…
Deep asymmetric networks prioritize feature importance.
problem Learning feature importance in neural networks.
method Nodes with smaller indices are more sensitive to variant activation functions.
result Features are learned in order of importance, and nodes can be pruned.
A deep clustering method for hyperspectral images improves clustering performance by constraining intra-class distances.
problem Clustering performance degradation in hyperspectral images due to high dimensionality.
method Intra-class distance constrained deep clustering algorithm using auto-encoder network.
result The proposed algorithm outperforms state-of-the-art methods in clustering hyperspectral images.
DeepFS uses deep neural networks to select significant features in ultra high-dimensional data.
problem Challenges in traditional feature selection methods for high-dimensional, low-sample-size data.
method Two-step nonparametric approach combining deep neural networks and feature screening.
result DeepFS effectively identifies significant features with high precision for ultra high-dimensional data.
Study shows DNNs often extract redundant features, influenced by network size and activation function.
problem Redundancy in deep neural network features.
method Hierarchical clustering of features based on cosine distances, varying network sizes and activation functions.
result Network size and activation function are key factors in DNN redundancy.
forgeNet uses a tree-based ensemble to learn feature graphs for deep learning in omics data.
problem Small sample size vs. large feature space in omics data.
method forgeNet integrates a forest feature graph extractor with a GEDFN architecture.
result ForgeNet achieves high classification accuracy on synthetic and real datasets.
New method proves neural networks can select features consistently.
problem Feature selection for deep neural networks is challenging.
method Adaptive Group Lasso selection procedure with Group Lasso as the base estimator.
result Adaptive Group Lasso is selection-consistent for a wide class of neural networks.
FiBiNET combines feature importance and bilinear interactions for CTR prediction.
problem Improving click-through rate prediction in advertising and feed ranking systems.
method FiBiNET dynamically learns feature importance via SENET and bilinear feature interactions.
result FiBiNET outperforms shallow and deep models on real-world datasets.
Proposes joint domain alignment and discriminative feature learning for deep domain adaptation.
problem Reduces domain shift and misclassification of target domain samples.
method Instance-based and center-based discriminative feature learning methods.
result Learning discriminative features in shared feature space significantly boosts deep domain adaptation performance.
Proposes a new method to prevent overfitting in deep neural networks.
problem Overfitting in deep neural networks with many trainable parameters.
method Randomly replaces elements in feature maps with specific values during training.
result Improves the testing performance of deep neural networks on benchmark datasets.
GRIP2 improves deep learning feature selection robustness in correlated and noisy data.
problem Identifying predictive features in correlated and noisy data.
method Integrates first-layer feature activity over a two-dimensional regularization surface to control sparsity and geometry, using efficient block-stochastic sampling.
result Demonstrates improved robustness and power in high correlation and low signal-to-noise ratio regimes.
Paper identifies how neural networks learn features.
problem Understanding how neural networks automatically select features.
method Deep Neural Feature Ansatz, implementing average gradient outer product.
result Recursive Feature Machines achieve state-of-the-art performance.
Study compares handcrafted and deep neural network features for OCT image classification.
problem Classifying OCT images for disease detection.
method Comparison of Histogram of Oriented Gradient (HOG), Local Binary Pattern (LBP), DenseNet-169, and ResNet50.
result Deep neural network methods outperform handcrafted features with higher accuracy and better underrepresented class performance.
Deep learning improves ECG arrhythmia classification.
problem Classifying ECG patterns accurately.
method Transfer learning from image classification to ECG spectrograms.
result 97.23% accuracy in classifying 7000 ECG instances.
Efficient skin lesion analysis combines deep CNN and handcrafted features.
problem Skin lesion analysis for melanoma detection.
method Combines deep learning and handcrafted features, introduces hybrid features.
result 0.841 score on validation dataset using SVM classifier.
Proposes a feature leveling method to improve interpretability of deep neural networks.
problem Deep neural networks are hard to interpret due to mixed feature levels.
method Introduces a feature leveling architecture to isolate low and high level features.
result Modified models achieve competitive results and improved interpretability.
Deep MF extracts hierarchical features from large data sets.
problem Mining complex, interleaved features in large data sets.
method Deep matrix factorization models and algorithms.
result Deep MF achieves outstanding performance on unsupervised tasks.
Deep learning explained through spectral filtering of hierarchical features.
problem Understanding how deep neural networks learn useful representations from data.
method Neural Low-Degree Filtering (Neural LoFi) as a stylized limit of gradient-based training.
result Predicts how representations are selected layer by layer and explains emergence of concepts.
Characterizes test error in learning with deep, structured feature maps.
problem Characterizing test error in learning with deep, structured feature maps.
method Asymptotic analysis of feature covariance and population covariance.
result Closed-form formula for feature covariance in Gaussian rainbow neural networks.
HAR-Net combines deep features with traditional hand-crafted features for better human activity recognition.
problem Challenges in traditional HAR methods, especially feature extraction.
method Combines deep learning and traditional feature engineering.
result Performance improvement of 0.9% compared to traditional SVM.
In this paper, we propose a new deep feature selection method based on deep architecture. Our method uses stacked auto-encoders for feature representation in higher-level abstraction. We developed and applied a novel feature learning approach to a specific precision medicine problem, which focuses on assessing and prio…
AGOP mechanism explains deep neural collapse in neural networks.
problem Explaining the rigid structure of data representations in deep neural networks.
method Introducing AGOP and Deep RFM to demonstrate DNC.
result AGOP mechanism causes deep neural collapse in neural networks.
DeepPINK improves reproducibility of deep neural networks by selecting features robustly.
problem Lack of interpretability and reproducibility in deep neural networks.
method Integrates feature selection with knockoffs framework in a new DNN architecture.
result Demonstrates improved reproducibility and robustness in feature selection.
Detects out-of-distribution and adversarial samples using deep feature distributions.
problem Detecting out-of-distribution and adversarial samples in deep neural networks.
method Modeling deep features with parametric probability distributions and calculating likelihoods at inference.
result Improves detection of out-of-distribution and adversarial samples, up to 12 percentage points in AUPR and AUROC metrics.
Proposes a method to select features for deep learning in noisy, high-dimensional data.
problem Feature selection for deep learning in ultra-high dimensional and highly correlated data.
method Data-adaptive multi-resolutional screening and cleaning with deep learning.
result Achieves high power while keeping false discovery rate low.
We propose semi-random features for nonlinear function approximation. The flexibility of semi-random feature lies between the fully adjustable units in deep learning and the random features used in kernel methods. For one hidden layer models with semi-random features, we prove with no unrealistic assumptions that the m…
New adversarial attack method based on deep feature distributions.
problem Adversarial attacks on CNN classifiers using output layer information.
method Modeling and exploiting class-wise and layer-wise deep feature distributions.
result Achieves state-of-the-art transfer-based attack results for undefended ImageNet models.
DSCF-Net learns deep features for clustering with robustness and locality preservation.
problem Unsupervised deep representation learning for clustering.
method Integrates robust deep concept factorization, deep self-expressive representation, and adaptive locality preserving feature learning.
result Delivers state-of-the-art performance on public databases.
Herein, we generate pseudo-features based on the multivariate probability distributions obtained from the feature maps in layers of trained deep neural networks. Further, we augment the minor-class data based on these generated pseudo-features to overcome the imbalanced data problems. The proposed method, i.e., cavity …
Paper tackles speaker verification by removing reverberation using deep LSTM networks.
problem Improving speaker verification accuracy in reverberant environments.
method Dual-label deep LSTM networks trained to map reverberant to clean speech features.
result Evaluates performance using EERs, showing improved accuracy.
This paper extends neural collapse to regression problems, revealing key features and structures.
problem Understanding the structure learned by deep neural networks in regression tasks.
method Established Neural Regression Collapse (NRC) across different models, analyzing feature and weight alignments.
result Deep neural regression models exhibit a collapsed feature space, aligning with target dimensions and covariances.
Deep random feature models are analyzed for their performance with exact asymptotic expressions.
problem Understanding the performance of deep random feature models.
method Established a novel universality result and used the convex Gaussian Min-Max theorem.
result Exact asymptotic expressions for the performance of deep random feature models are derived.
"Deep Learning" methods attempt to learn generic features in an unsupervised fashion from a large unlabelled data set. These generic features should perform as well as the best hand crafted features for any learning problem that makes use of this data. We provide a definition of generic features, characterize when it i…
Deep learning predicts risky behavior in retail investors for financial risk management.
problem Predicting profitable trading behavior in retail investors.
method Developed a deep learning model to predict trader profitability.
result Deep learning outperforms conventional machine learning methods in predicting trader behavior.
Proposes a new objective function to learn robust deep features.
problem Learning robust deep representations of noisy or unavailable features.
method Maximizes mutual information of all subsets of features relative to supervising signal.
result Surrogate objective function encourages non-redundant and conditionally independent features.
Proposes using gradients as features for efficient deep learning adaptation.
problem Efficient deep representation learning for different tasks.
method Designs a linear model incorporating gradients and activations of a pre-trained network.
result Shows strong results across various tasks and datasets.