Supporting evidence for adaptive feature program across diverse models.
problem Analyzing feature learning in neural networks.
method Over-parameterized sequence models and feature error measure (FEM).
result FEM is decreasing during training of adaptive feature models.
Feature extraction from persistence diagrams, as a tool to enrich machine learning techniques, has received increasing attention in recent years. In this paper we explore an adaptive methodology to localize features in persistent diagrams, which are then used in learning tasks. Specifically, we investigate three algori…
Unsupervised domain adaptation methods aim to alleviate performance degradation caused by domain-shift by learning domain-invariant representations. Existing deep domain adaptation methods focus on holistic feature alignment by matching source and target holistic feature distributions, without considering local feature…
Interventional domain adaptation improves feature transferability by removing spurious correlations.
problem Improper feature transferability due to spurious correlations in domain adaptation.
method Intervention strategy using unlabeled target data to generate counterfactual features and train discriminability invariance.
result Consistent performance improvements over state-of-the-art approaches in various domain adaptation tasks.
New method restores source features for SFDA without source data.
problem Domain adaptation without access to source data.
method Feature Restoration (FR) and Bottom-Up Feature Restoration (BUFR).
result BUFR outperforms existing SFDA methods in accuracy, calibration, and data efficiency.
This paper improves neural network generalization by dynamically learning kernel parameters.
problem Improving neural network generalization and adaptability.
method Diagonal adaptive kernel model that learns kernel eigenvalues and output coefficients during training.
result The diagonal adaptive kernel model significantly improves generalization over fixed-kernel methods.
Greedy PIG adapts integrated gradients for better feature attribution.
problem Interpreting deep learning model predictions.
method Unified discrete optimization framework for feature attribution and selection.
result Greedy PIG improves feature attribution on various tasks.
Distributed sensors compress and send features to a fusion center for linear regression.
problem Efficiently compress and transmit features from distributed sensors to a fusion center under varying communication constraints.
method Designs a distributed and adaptive feature compression scheme using optimal quantizers and simple adaptive strategies.
result Demonstrates improved inference performance through simulated experiments.
Various factorization-based methods have been proposed to leverage second-order, or higher-order cross features for boosting the performance of predictive models. They generally enumerate all the cross features under a predefined maximum order, and then identify useful feature interactions through model training, which…
This study addresses an issue of co-adaptation between a feature extractor and a classifier in a neural network. A naive joint optimization of a feature extractor and a classifier often brings situations in which an excessively complex feature distribution adapted to a very specific classifier degrades the test perform…
Method evaluates classification uncertainty with adaptively chosen features.
problem Finding a balance between model efficiency and fairness.
method Adaptively selects features for equalized coverage in classification.
result Valid and effective method demonstrated on simulated and real data.
Domain adaptation aims to leverage the supervision signal of source domain to obtain an accurate model for target domain, where the labels are not available. To leverage and adapt the label information from source domain, most existing methods employ a feature extracting function and match the marginal distributions of…
We introduce an unsupervised approach to efficiently discover the underlying features in a data set via crowdsourcing. Our queries ask crowd members to articulate a feature common to two out of three displayed examples. In addition we also ask the crowd to provide binary labels to the remaining examples based on the di…
This work proves convergence of adaptive resampling for random Fourier features.
problem Sampling Fourier frequencies well for high-dimensional data.
method Data adaptive resampling of Fourier frequencies, asymptotically optimal.
result Proves convergence of adaptive resampling method for regression and classification problems.
AdaEnsemble learns adaptive feature interactions for CTR prediction.
problem Learning feature interactions for CTR prediction in recommender systems and Ads ranking.
method AdaEnsemble is a Sparsely-Gated Mixture-of-Experts (SparseMoE) architecture that dynamically selects feature interaction depth.
result AdaEnsemble achieves better prediction accuracy and inference efficiency compared to state-of-the-art models.
Adaptive Group Lasso selects important features in neural networks.
problem Lack of interpretability in neural networks.
method Adaptive Group Lasso for feature selection.
result Consistent feature selection for neural networks with theoretical guarantee.
This work bridges two views of feature learning in neural networks.
problem The relationship between kernel scale changes and data-adaptive feature learning in neural networks remains unresolved.
method Using statistical mechanics, the work derives analytical expressions for network output statistics across scaling regimes.
result Kernel adaptation can be reduced to an effective kernel rescaling, but multi-scale adaptive approach provides richer insights.
Guided adaptive shrinkage uses co-data to improve feature selection in genomic studies.
problem Feature selection challenges in high-dimensional genomics data, especially in clinical settings.
method Guided adaptive shrinkage methods that use co-data to adapt shrinkage parameters.
result Improves feature selection in genomic studies, demonstrated through comparisons and examples.
Many current approaches to the design of intrusion detection systems apply feature selection in a static, non-adaptive fashion. These methods often neglect the dynamic nature of network data which requires to use adaptive feature selection techniques. In this paper, we present a simple technique based on incremental le…
MetaVRF learns adaptive kernels for fast few-shot learning.
problem Few-shot learning with limited data.
method MetaVRF with latent variable model and variational inference.
result MetaVRF produces kernels with high representational power and fast adaptation.
AFS-BM improves model accuracy by dynamically selecting features.
problem Feature selection challenges in ML, especially scalability and adaptability.
method Joint optimization for feature selection and model training with binary masking.
result AFS-BM achieves significant improvements in model accuracy and computational efficiency.
We propose a novel adversarial speaker adaptation (ASA) scheme, in which adversarial learning is applied to regularize the distribution of deep hidden features in a speaker-dependent (SD) deep neural network (DNN) acoustic model to be close to that of a fixed speaker-independent (SI) DNN acoustic model during adaptatio…
Discriminative active learning reduces data annotation costs for domain adaptation.
problem Conditional shift problem hinders domain adaptation between related but different domains.
method Three-stage active adversarial training: invariant feature space learning, uncertainty and diversity criteria, re-training with queried labels.
result Empirical comparisons show the proposed approach is more effective than existing methods.
DMFAW improves multi-view clustering with adaptive weights and feature selection.
problem Lack of effective feature selection and empirical hyperparameter selection in existing deep matrix factorization methods.
method Introduces Deep Matrix Factorization with Adaptive Weights (DMFAW) for multi-view clustering, incorporating feature selection and dynamically updating weights using Control Theory.
result DMFAW outperforms state-of-the-art methods in clustering performance.
DFIV uses deep features for IV regression, achieving optimal rates.
problem Optimal IV regression with deep features for complex target functions.
method Two-stage approach: deep feature learning followed by IV regression.
result DFIV achieves minimax optimal learning rate under certain conditions.
A framework isolates and learns approximately shared features for better domain adaptation.
problem Reducing performance degradation in unseen domains using machine learning models.
method Statistical framework distinguishing feature utilities based on correlation variance across domains. Learning approximately shared features from source tasks and fine-tuning on target tasks.
result Improved population risk compared to previous results on both source and target tasks, resolving the paradox of feature selection.
Domain adaptation is the supervised learning setting in which the training and test data are sampled from different distributions: training data is sampled from a source domain, whilst test data is sampled from a target domain. This paper proposes and studies an approach, called feature-level domain adaptation (FLDA), …
Proposes TFDF to learn transferable and discriminative features for unsupervised domain adaptation.
problem Difficult to induce supervised classifier without labeled data in unsupervised domain adaptation.
method TFDF optimizes transferability and discriminability by aligning distributions and minimizing class confusion.
result TFDF achieves better performance on real-world datasets compared to existing methods.
Adaptive regularization prevents overfitting in large-scale sparse feature models.
problem Overfitting in models with large-scale sparse categorical features.
method Adaptive regularization of embedding layers' norm budget.
result Improves model performance within a single epoch and prevents multi-epoch performance degradation.
New decentralized KRR algorithm adapts to node-specific data.
problem Consistent node-specific data in decentralized KRR.
method Data-dependent random features for adaptive RF generation.
result Average regression accuracy improved by 25.5% across six datasets.
Multi-task feature learning aims to identity the shared features among tasks to improve generalization. It has been shown that by minimizing non-convex learning models, a better solution than the convex alternatives can be obtained. Therefore, a non-convex model based on the capped-ℓ1,ℓ1 regularization wa…
AdaTrans adapts to feature and sample transfer in high-dimensional regression.
problem High-dimensional linear regression with more features than samples.
method F-AdaTrans and S-AdaTrans methods using fused-penalties and adaptive weights.
result AdaTrans achieves convergence rates close to oracle estimators and near-minimax optimal rates.
Recent unsupervised approaches to domain adaptation primarily focus on minimizing the gap between the source and the target domains through refining the feature generator, in order to learn a better alignment between the two domains. This minimization can be achieved via a domain classifier to detect target-domain feat…
In this paper, we propose a new feature selection method for unsupervised domain adaptation based on the emerging optimal transportation theory. We build upon a recent theoretical analysis of optimal transport in domain adaptation and show that it can directly suggest a feature selection procedure leveraging the shift …
New SAE algorithm proves feature recovery for LLMs with theoretical guarantees.
problem Achieving interpretable features in large language models (LLMs).
method Proposed a statistical framework and bias adaptation technique for sparse autoencoders (SAEs).
result Proved correct recovery of all monosemantic features under specific data sampling.
Recently, considerable effort has been devoted to deep domain adaptation in computer vision and machine learning communities. However, most of existing work only concentrates on learning shared feature representation by minimizing the distribution discrepancy across different domains. Due to the fact that all the domai…
Domain adaptation aims to exploit the knowledge in source domain to promote the learning tasks in target domain, which plays a critical role in real-world applications. Recently, lots of deep learning approaches based on autoencoders have achieved a significance performance in domain adaptation. However, most existing …
Unsupervised domain adaptation aiming to learn a specific task for one domain using another domain data has emerged to address the labeling issue in supervised learning, especially because it is difficult to obtain massive amounts of labeled data in practice. The existing methods have succeeded by reducing the differen…
Unsupervised domain adaptation is a promising way to generalize deep models to novel domains. However, the current literature assumes that the label distribution is domain-invariant and only aligns the feature distributions or vice versa. In this work, we explore the more realistic task of Class-imbalanced Domain Adapt…
A new method for adapting to label shifts using class probability matching.
problem Adapting to label shifts where class probabilities differ between source and target domains.
method Class Probability Matching using Kernel Methods (CPMKM) framework.
result CPMKM outperforms existing methods on real datasets.
New online feature selection method handles streaming data with concept drift.
problem Handling streaming data with concept drift and sparsity.
method Online feature screening method with model adaptation.
result Online screening methods with model adaptation outperform without model adaptation on data streams with concept drift.
Enhances DGPs with adaptive RKHS Fourier features for better non-stationary pattern modeling.
problem Capturing complex non-stationary patterns in non-linear dynamical systems.
method Integrates ODE-based RKHS Fourier features into DGPs using convolution operations for adaptive amplitude and phase modulation. Uses a doubly stochastic variational inference framework.
result Improved predictive performance across various regression tasks.
TAROT enhances robustness and domain adaptability with domain-invariant features.
problem Developing models robust to adversarial attacks across diverse domains.
method Derives a new generalization bound and proposes TAROT algorithm.
result TAROT outperforms state-of-the-art methods in accuracy and robustness.
This paper explores how effective sample size, dimensionality, and model performance are related in covariate shift adaptation.
problem Understanding the relationship between effective sample size, dimensionality, and generalization in covariate shift adaptation.
method Building a unified theory connecting effective sample size, data dimensionality, and generalization in the context of covariate shift adaptation.
result Dimensionality reduction or feature selection can increase effective sample size, supporting the practice of reducing dimensionality before covariate shift adaptation.
Adaptive kernels from neural networks improve model performance.
problem Improving neural network performance through adaptive kernels.
method Deriving adaptive kernels from infinite-width neural networks using feature learning and gradient flow training.
result Adaptive kernels achieve lower test loss compared to traditional kernels.
New method proves neural networks can select features consistently.
problem Feature selection for deep neural networks is challenging.
method Adaptive Group Lasso selection procedure with Group Lasso as the base estimator.
result Adaptive Group Lasso is selection-consistent for a wide class of neural networks.
Proposes a flexible feature allocation model for sparse factor analysis.
problem Sparse data and rigid assumptions in traditional exploratory tools.
method Adaptive latent feature sharing with control over feature sparsity.
result Derives a novel adaptive Factor analysis (aFA) and aPPCA for flexible dimensionality reduction.
SBAMDT uses adaptive soft splits to model complex decision boundaries.
problem Limited ability of standard decision trees to capture complex decision boundaries.
method Probabilistic additive decision tree model with adaptive soft multivariate splits.
result Demonstrated improved predictive performance on synthetic and real datasets.