The decremented learning algorithms are required in machine learning, to prune redundant nodes and remove obsolete inline training samples. In this paper, an efficient decremented learning algorithm to prune redundant nodes is deduced from the incremental learning algorithm 1 proposed in [9] for added nodes, and two de…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “decremental features”
EML model tackles evolving features in online metric learning.
problem Challenges in applying metric learning to evolving features.
method Develops a new Evolving Metric Learning (EML) model for incremental and decremental features.
result EML model handles instance and feature evolutions simultaneously.
This study presents an efficient incremental/decremental approach for big streams based on Kernel Ridge Regression (KRR), a frequently used data analysis in cloud centers. To avoid reanalyzing the whole dataset whenever sensors receive new training data, typical incremental KRR used a single-instance mechanism for upda…
Efficiently adds or deletes data in GBDT models.
problem Traditional GBDT training requires all data to be accessed simultaneously, limiting add/delete operations.
method Proposes an online learning framework for GBDT supporting incremental and decremental learning.
result First work to unify incremental and decremental learning on GBDT in-place.
Two new inverse-free ELM algorithms for incremental and decremental learning are proposed.
problem Efficiently updating and removing multiple hidden nodes in ELM.
method Improved inverse-free recursive algorithms for Tikhonov regularization.
result Inverse-free algorithms for ELM with multiple hidden nodes and redundant nodes.
This study presents a rapid multiple incremental and decremental mechanism based on Weight-Error Curves (WECs) for support-vector analysis. Recursion-free computation is proposed for predicting the Lagrangian multipliers of new samples. This study examines Ridge Support Vector Models, subsequently devising a recursion-…
In this paper we present an incremental variant of the Twin Support Vector Machine (TWSVM) called Fuzzy Bounded Twin Support Vector Machine (FBTWSVM) to deal with large datasets and learning from data streams. We combine the TWSVM with a fuzzy membership function, so that each input has a different contribution to each…
Designs a neural network to reduce training cost by mapping to higher dimensions.
problem High training cost in neural networks.
method Maps feature vectors to higher dimensional space, designs weight matrices to reduce cost, uses convex constraints.
result Reduces training cost as the number of layers increases, without cross-validation.
By and large the behavior of stochastic gradient is regarded as a challenging problem, and it is often presented in the framework of statistical machine learning. This paper offers a novel view on the analysis of on-line models of learning that arises when dealing with a generalized version of stochastic gradient that …
Tighter Bound Estimation of Sensitivity Analysis for Incremental and Decremental Data Modificationcs.LG
Algorithm estimates bounds of updated classifier coefficients efficiently.
problem Determining sensitivity of updated classifiers without retraining.
method Proposes an algorithm to estimate upper and lower bounds of updated classifier coefficients.
result Estimates bounds with low computational complexity and tightness.
Paper tackles non-monotonic resource utilization in sequential decision-making.
problem Sequential decision-making under uncertainty with resource constraints.
method Introduces a new MDP policy with constant regret against LP relaxation.
result Develops a learning algorithm with logarithmic regret for unknown outcome distributions.
Beta Basis Function Neural Network (BBFNN) is a special kind of kernel basis neural networks. It is a feedforward network typified by the use of beta function as a hidden activation function. Beta is a flexible transfer function representing richer forms than the common existing functions. As in every network, the arch…
Identifies features most relevant to concept drift in data.
problem Identifying features most relevant to concept drift.
method Distinguishing between drift inducing and faithfully drifting features; deriving minimal subsets of features to characterize drift.
result Derives a detection algorithm for concept drift.
Paper predicts EEG features from acoustic features using RNN and GAN.
problem Predicting EEG features from acoustic features.
method Recurrent Neural Network (RNN) and Generative Adversarial Network (GAN).
result Lower RMSE and normalized RMSE values compared to generating acoustic features from EEG features.
Relative Feature Importancestat.ML
Introduces RFI for assessing feature importance relative to any subset of features.
problem Lack of nuanced feature importance computation.
method Generalizes PFI and CFI to assess relative feature importance.
result Derives general interpretation rules for RFI.
A single pre-trained agent guides feature selection using knockoffs.
problem Feature selection challenges in AI-readiness of data.
method Generates knockoff features and uses reinforcement learning.
result Optimal feature subset identified with reduced dependency on target variable.
Feature networks link ML features via graph structure for enhanced learning.
problem Enhancing feature expressiveness and learning efficiency in machine learning.
method Graph representation of feature vectors, leveraging Fourier and functional analysis.
result Feature networks enable novel, complex feature dependencies.
Approach for selecting features by discarding nuisance and correlated ones.
problem Large datasets with correlated and nuisance features.
method Laplacian score criterion, autoencoder architecture, concrete layer.
result Outperforms similar approaches in clustering performance.
New stability measures for similar features improve feature selection accuracy.
problem Existing stability measures fail to distinguish similar features in highly correlated datasets.
method Introduce new adjusted stability measures that consider feature similarities.
result One new stability measure considers highly similar features as interchangeable.
Counterexamples show HSIC feature selection misses critical features.
problem Feature selection using HSIC misses important features.
method Feature selection via HSIC maximization.
result HSIC feature selection can miss critical features.
Validity of Feature Importance in Low-Performing Machine Learning for Tabular Biomedical Datastat.ML
This paper shows feature importance remains valid even in low-performing models.
problem Feature importance validity in low-performing machine learning models for biomedical data.
method Experiments with synthetic and real biomedical datasets to compare feature rank stability under different data reductions.
result Feature importance can be maintained even at low performance levels if data size is adequate.
New algorithms select and rank features from MTS without feature extraction.
problem Feature extraction step for MTS classification.
method Directly computes similarity between time series and assesses cluster structure matching labels.
result Techniques match labels well without feature extraction.
Pipeline learns topological features for protein stability prediction.
problem Predicting protein stability using topological features.
method Data-driven method to learn topological features, comparing with expert features.
result Topological features achieve 92%-99% of SME-based models' performance.
In text classification, dictionaries can be used to define human-comprehensible features. We propose an improvement to dictionary features called smoothed dictionary features. These features recognize document contexts instead of n-grams. We describe a principled methodology to solicit dictionary features from a teache…
FeAT improves OOD generalization by learning richer features.
problem Improving feature learning for out-of-distribution (OOD) generalization.
method Feature Augmented Training (FeAT) iteratively augments and retains features from different subsets of training data.
result FeAT effectively learns richer features, boosting OOD performance.
The paper defines and analyzes feature complexity in DNNs, proposing metrics for feature disentanglement and evaluation.
problem Understanding and quantifying the complexity of features learned by deep neural networks.
method Proposes a definition and disentanglement of feature complexity orders, introduces metrics for reliability and over-fitting evaluation.
result Establishes a relationship between feature complexity and DNN performance, and proposes a generic mathematical tool for network compression and knowledge distillation.
Conventional mutual information (MI) based feature selection (FS) methods are unable to handle heterogeneous feature subset selection properly because of data format differences or estimation methods of MI between feature subset and class label. A way to solve this problem is feature transformation (FT). In this study,…
A new method for measuring conditional feature importance using generative models.
problem Challenges in evaluating feature importance given other feature values.
method Adversarial Random Forest (ARF) for generating on-manifold data points.
result cARFi method yields robust importance scores adaptable for various feature importance notions.
Existing feature selection methods fail to properly account for interactions between features when evaluating feature subsets. In this paper, we attempt to remedy this issue by using orthogonal variance decomposition to evaluate features. The orthogonality of the decomposition allows us to directly calculate the total …
CAN approximates explicit feature interactions for CTR prediction.
problem Learning explicit feature interactions from sparse features.
method Co-Action Network approximates explicit pairwise feature interactions without introducing too many additional parameters.
result CAN outperforms state-of-the-art CTR models and the cartesian product method.
Proposes a new feature selection method integrating feature relationships.
problem Feature selection in machine learning models.
method Integrates feature-feature and feature-target relationships via penalized mRMR.
result Correctly identifies inactive features, reducing false discoveries.
Online feature selection has been an active research area in recent years. We propose a novel diverse online feature selection method based on Determinantal Point Processes (DPP). Our model aims to provide diverse features which can be composed in either a supervised or unsupervised framework. The framework aims to pro…
New method disentangles feature importance scores in machine learning.
problem Misinterpretation of feature importance scores due to interactions and dependencies.
method Derive DIP (Disentangled Importance) decomposition of feature importance scores.
result DIP decomposition uniquely separates standalone contributions from interactions and dependencies.
Feature selection has been proven a powerful preprocessing step for high-dimensional data analysis. However, most state-of-the-art methods tend to overlook the structural correlation information between pairwise samples, which may encapsulate useful information for refining the performance of feature selection. Moreove…
GRANITE: A Generalized Regional Framework for Identifying Agreement in Feature-Based Explanationsstat.ML
GRANITE unifies feature-based explanation methods to reduce disagreement.
problem Disagreement among feature-based explanation methods.
method GRANITE partitions feature space into regions minimizing interaction and distribution influences.
result Unified and consistent feature explanations.
Orthogonal random features approximate a Bessel kernel, offering sharper bounds than random Fourier features.
problem Approximating Gaussian kernel efficiently for large datasets.
method Use of Haar orthogonal matrices to construct orthogonal random features and analyze their bias and variance.
result Orthogonal random features approximate a Bessel kernel, not the Gaussian kernel, with sharper bounds.
New methods for assessing and visualizing feature groups in machine learning models.
problem Lack of methods for interpreting feature groups in machine learning models.
method Permutation-based, refitting, and Shapley-based techniques for grouped feature importance. Introduced a sequential procedure for identifying stable feature combinations. Developed a combined features effect plot.
result Effective methods for assessing and visualizing the importance and effect of feature groups in machine learning models.
Learning with streaming data has attracted much attention during the past few years. Though most studies consider data stream with fixed features, in real practice the features may be evolvable. For example, features of data gathered by limited-lifespan sensors will change when these sensors are substituted by new ones…
Supporting evidence for adaptive feature program across diverse models.
problem Analyzing feature learning in neural networks.
method Over-parameterized sequence models and feature error measure (FEM).
result FEM is decreasing during training of adaptive feature models.
Hermite polynomials improve private data generation by reducing feature count.
problem Infinite-dimensional features in kernel mean embedding are impractical for private data generation.
method Replace random features with Hermite polynomial features, leveraging their ordered nature.
result Hermite polynomial features yield a more accurate approximation of kernel mean embedding with fewer features.
Feature selection is a technique to screen out less important features. Many existing supervised feature selection algorithms use redundancy and relevancy as the main criteria to select features. However, feature interaction, potentially a key characteristic in real-world problems, has not received much attention. As a…
Paper defines feature impact and importance from data, not models.
problem Misinterpretation of feature importance as impact leads to flawed insights.
method Mathematical definitions of feature impact and importance derived from partial dependence curves.
result Feature rankings by these definitions are competitive with existing techniques.
Learning with feature evolution studies the scenario where the features of the data streams can evolve, i.e., old features vanish and new features emerge. Its goal is to keep the model always performing well even when the features happen to evolve. To tackle this problem, canonical methods assume that the old features …
NGMs create mirrored features to assess neural network feature importance.
problem Lack of feature relevance information in DNNs limits their applicability.
method Structured perturbation and kernel-based conditional dependence measure for feature importance evaluation.
result Controls feature selection error rate and maintains high selection power with correlated features.
Inf-FS selects features by graph paths, ranking them for infinite feature sets.
problem Feature selection in large datasets with relevance and redundancy.
method Graph-based feature selection with infinite paths, evaluating feature subsets using matrix power series and Markov chains.
result Inf-FS outperforms other methods in various feature selection scenarios.
GADGET framework decomposes global feature effects using recursive partitioning.
problem Misleading global feature effects when feature interactions are present.
method Generalized additive decomposition of global effects (GADGET) based on recursive partitioning.
result Minimizes interaction-related heterogeneity of local feature effects.
Neural networks outperform kernels by learning features better.
problem Current theories of feature learning do not adequately assess feature quality.
method Introduced feature quality metric and examined existing theories empirically.
result Current theories of feature learning do not provide a sufficient foundation for neural network generalization.
We propose a method for finding alternate features missing in the Lasso optimal solution. In ordinary Lasso problem, one global optimum is obtained and the resulting features are interpreted as task-relevant features. However, this can overlook possibly relevant features not selected by the Lasso. With the proposed met…