Private variable selection method controls FDR with simulations showing reasonable power.
problem Performing variable selection with privacy constraints.
method Private knockoff filter using Gaussian and Laplace mechanisms.
result Achieves controlled false discovery rate (FDR) in variable selection.
Convolutional neural networks (CNNs) achieve state-of-the-art performance in a wide variety of tasks in computer vision. However, interpreting CNNs still remains a challenge. This is mainly due to the large number of parameters in these networks. Here, we investigate the role of compression and particularly pruning fil…
Binary Stochastic Filtering (BSF), the algorithm for feature selection and neuron pruning is proposed in this work. The method defines filtering layer which penalizes amount of the information involved in the training process. This information could be the input data or output of the previous layer, which directly lead…
Max-plus operators improve neural network filter selection and pruning.
problem Improving neural network efficiency and reducing redundancy.
method Exploiting Max-plus operators in neural network layers for filter selection and model pruning.
result Max-plus layers enhance filter selection and reduce redundancy without performance loss.
FSinR is a comprehensive R package for feature selection.
problem Feature selection in machine learning.
method Filter and wrapper methods, search algorithms.
result Provides a variety of feature selection methods and search algorithms.
We find that CNNs with BN and ReLU exhibit feature sparsity through selective feature penalization.
problem Feature sparsity in CNNs trained with specific techniques.
method Empirical study and hypothesis testing on sparsification mechanisms.
result Selective feature penalization leads to feature sparsity in CNNs, comparable to explicit pruning.
Review and benchmark 58 feature selection methods for ML applications.
problem Selecting appropriate features for predictive modeling in ML applications.
method Synthesizes literature on feature selection benchmarking and evaluates 58 methods.
result Well-performing candidate methods identified: random forest, DISR, JIM.
Novel spam filter improves e-mail classification accuracy.
problem Uneven class distribution, unequal error cost, frequent content change, personalized discrimination.
method TFDCR feature selection, incremental learning, dynamic feature update.
result TFDCR outperforms in feature selection, incremental model improves classification accuracy.
One of the classical problems in machine learning and data mining is feature selection. A feature selection algorithm is expected to be quick, and at the same time it should show high performance. MeLiF algorithm effectively solves this problem using ensembles of ranking filters. This article describes two different wa…
New unsupervised feature selection method for imbalanced datasets.
problem Feature selection challenges in imbalanced multi-class datasets.
method Distance Rank Score using Spearman's Rank Correlation.
result Outperforms existing methods on clustering problems.
Bayesian neural network improves feature selection and prediction.
problem Improving feature selection and prediction accuracy in neural networks.
method BNN-ARD with l2-norm feature importance measure.
result Improves variable selection and predictive performance on real-world data.
A new feature selection method using random forest and Kolmogorov filter.
problem Ultra-high dimensional data feature selection.
method Fused Kolmogorov filter with random forest based recursive feature elimination.
result Selection and L2 consistency under weak conditions. ED-Filter improves eating disorder classification on Twitter.
problem High dimensionality and extensive feature sets in Twitter data for ED classification.
method Informed branch and bound search technique with hybrid greedy-based deep learning.
result Significant improvements in classification accuracy and efficiency.
Paper proves convergence of Kalman filter on Stiefel manifolds with measurement errors.
problem Filtering constant particle with measurement errors on Stiefel manifolds.
method Extended Kalman filter applied to Stiefel manifold-valued observations.
result Convergence of the extended Kalman filter proved for constant system process.
OBF optimally filters features under independent Gaussian models.
problem Biomarker discovery from complex data.
method Optimal Bayesian feature selection under independent Gaussian models.
result OBF is consistent and optimal under mild conditions.
Feature selection is frequently used as a pre-processing step to machine learning. It is a process of choosing a subset of original features so that the feature space is optimally reduced according to a certain evaluation criterion. The central objective of this paper is to reduce the dimension of the data by finding a…
A new ML-based filter improves data assimilation for nonlinear systems.
problem Improving data assimilation for nonlinear systems using ensemble methods.
method Developed a machine learning-based conditional mean filter (ML-EnCMF) integrating ANN and linear functions.
result ML-EnCMF outperforms EnKF and likelihood-based EnCMF in nonlinear systems.
New adaptive filters reduce energy consumption in electronic devices.
problem Reducing energy consumption in adaptive filtering algorithms.
method Data-selective adaptive filters, set-membership (SM) filters, trinion and quaternion systems, partial-updating, LMS and recursive LMS algorithms.
result Improved adaptive filtering algorithms with reduced computational complexity and enhanced stability.
DeepUnHide uses deep learning to reveal hidden demographic features in recommender systems.
problem Extracting hidden demographic features from recommender systems factors.
method Gradient-based localization in deep learning for feature extraction.
result DeepUnHide outperforms state-of-the-art feature selection methods.
This paper explores how balancing and filtering techniques affect predictive multiplicity in machine learning models.
problem Predictive multiplicity due to Rashomon effect in high-stakes environments.
method Investigates the impact of balancing and filtering techniques on predictive multiplicity using 21 real-world datasets.
result Data-centric AI strategies can mitigate predictive multiplicity, but preprocessing methods may introduce it.
Multi-objective optimization for hyperparameters and features.
problem Optimizing hyperparameters and selecting features for machine learning models.
method Two approaches: model-based optimization and NSGA-II-based wrapper approach using filter ensembles.
result NSGA-II approach is computationally less expensive but may require more evaluations.
Powershap speeds up feature selection for machine learning models.
problem Feature selection in machine learning models is crucial but computationally expensive.
method Powershap combines statistical hypothesis testing and Shapley values for quick feature selection.
result Powershap outperforms other methods with similar predictive performance and significantly faster execution.
Paper analyzes robustness of data-selective Volterra NLMS algorithm.
problem Robustness analysis of data-selective Volterra NLMS algorithm.
method The paper analyzes the local robustness and proposes a global bound for the error in the coefficient vector.
result The DS-VNLMS algorithm is robust against noise and improves parameter estimation for most iterations.
To select the best algorithm for a new problem is an expensive and difficult task. However, there are automatic solutions to address this problem: using Metalearning, which takes advantage of problem characteristics (i.e. metafeatures), one is able to predict the relative performance of algorithms. In the Collaborative…
New method improves portfolio selection by filtering noisy covariance matrices.
problem Noisy covariance matrices in financial datasets affect portfolio performance evaluation.
method Combinatorial Optimization approach using Mixed Integer Quadratic Programming.
result Our method outperforms existing filtering strategies for real financial datasets.
Inference of space-time varying signals on graphs emerges naturally in a plethora of network science related applications. A frequently encountered challenge pertains to reconstructing such dynamic processes, given their values over a subset of vertices and time instants. The present paper develops a graph-aware kernel…
FPPI selectively uses predictions to improve inference efficiency.
problem Improving statistical inference with limited labeled data and heterogeneous prediction quality.
method Filtered Prediction-Powered Inference (FPPI) framework.
result FPPI achieves strictly improved asymptotic efficiency compared to existing methods.
The paper deals with the adaptation of a new measure for the unsupervised feature selection problems. The proposed measure is based on space filling concept and is called the coverage measure. This measure was used for judging the quality of an experimental space filling design. In the present work, the coverage measur…
Two new deterministic offspring selection methods reduce statistical distance in SMC and pMCMC.
problem Improving the performance of resampling in SMC methods.
method Proposes two deterministic offspring selection methods to minimize KL divergence and TV distance.
result Our methods outperform or match state-of-the-art resampling schemes on benchmarks.
We consider the problem of online collaborative filtering in the online setting, where items are recommended to the users over time. At each time step, the user (selected by the environment) consumes an item (selected by the agent) and provides a rating of the selected item. In this paper, we propose a novel algorithm …
DsDm selects data to improve model performance, avoiding handpicked notions of quality.
problem Selecting data for model training can lead to worse performance than random selection.
method Formulates dataset selection as an optimization problem, maximizing model performance.
result Selected datasets improve language model performance by 2x over baseline methods.
Collaborative filtering is used to recommend items to a user without requiring a knowledge of the item itself and tends to outperform other techniques. However, collaborative filtering suffers from the cold-start problem, which occurs when an item has not yet been rated or a user has not rated any items. Incorporating …
SAUNA filters out noisy samples to boost RL performance.
problem Improving RL performance by filtering out non-informative samples.
method SAUNA selects samples based on the fraction of variance explained by the value function, rejecting non-informative transitions.
result SAUNA significantly improves RL performance on benchmark problems.
In this work, we consider the class of multi-state autoregressive processes that can be used to model non-stationary time-series of interest. In order to capture different autoregressive (AR) states underlying an observed time series, it is crucial to select the appropriate number of states. We propose a new model sele…
New method improves knockoff filter for correlated predictors.
problem Improving power of knockoff filters for correlated designs.
method Conditional Independence knockoff procedure for Gaussian tree graphical models.
result Conditional Independence knockoff outperforms sophisticated methods.
A new knockoff statistic using conditional prediction function improves variable selection in complex models.
problem Controlling false discovery rate in complex models with nonlinear relationships.
method Introducing a knockoff statistic based on the conditional prediction function for use with machine learning models.
result The CPF statistics provide superior power in detecting prognostic variables over existing knockoff statistics.
This study improves state estimation for nonlinear systems using conditional normalizing flows.
problem Performance degradation of traditional filtering algorithms in nonlinear systems with non-Gaussian uncertainty.
method Uses conditional normalizing flows with MLP, transformer, or state-space models for state and parameter estimation.
result Optimal-transport-inspired kinetic loss mitigates overparameterization in flows.
We study empirical covariance matrices in finance. Due to the limited amount of available input information, these objects incorporate a huge amount of noise, so their naive use in optimization procedures, such as portfolio selection, may be misleading. In this paper we investigate a recently introduced filtering proce…
Convolutional neural networks (CNNs) have recently emerged as a popular building block for natural language processing (NLP). Despite their success, most existing CNN models employed in NLP share the same learned (and static) set of filters for all input sentences. In this paper, we consider an approach of using a smal…
Proposes a copula-based filter for diabetes risk prediction.
problem Feature selection for robust and interpretable predictive modeling in medicine, especially for extreme patient strata.
method Copula-based supervised filter using Gumbel-copula implied upper-tail concordance score (lambda U).
result The proposed filter outperforms standard filters and provides clinically coherent predictors.
In this paper, we introduce a novel technique based on the Secure Selective Convolutional (SSC) techniques in the training loop that increases the robustness of a given DNN by allowing it to learn the data distribution based on the important edges in the input image. We validate our technique on Convolutional DNNs agai…
DEKF maintains stability in LSTM learning with bounded perturbations.
problem Stability of DEKF in LSTM-based online learning.
method Modeling DEKF as a perturbed extended Kalman filter and deriving stability conditions.
result DEKF learns LSTM parameters with similar stability properties to the global extended Kalman filter.
A new principle for optimizer selection improves training speed and performance.
problem Finding the best optimizer hyperparameters for faster training.
method Formulate optimizer selection as maximizing the expected drop rate in loss, treating gradients and updates as signals and an optimizer as a causal filter.
result Greedy optimizer selection yields stable and effective momentum rules.
Data-driven spatial filtering algorithms optimize scores such as the contrast between two conditions to extract oscillatory brain signal components. Most machine learning approaches for filter estimation, however, disregard within-trial temporal dynamics and are extremely sensitive to changes in training data and invol…
Convolutional Bayesian filtering generalizes state estimation by incorporating inequality conditions.
problem Standard Bayesian filtering assumes exact conditional probabilities, limiting its applicability.
method Introducing inequality conditions transforms conditional probabilities into convolutional forms, expanding the filtering framework.
result Convolutional Bayesian filtering encompasses standard Bayesian filtering and allows for more nuanced model consideration.
Extends knockoff filter for composite null hypotheses in variable selection.
problem Handling composite null hypotheses in variable selection.
method Developed two methods for composite inference with knockoffs: S-OLS and FRPP.
result Proposed heuristic variants of S-OLS outperforming BH procedure for composite nulls.
In this article, we present the elitist particle filter based on evolutionary strategies (EPFES) as an efficient approach for nonlinear system identification. The EPFES is derived from the frequently-employed state-space model, where the relevant information of the nonlinear system is captured by an unknown state vecto…
We compare in this paper several feature selection methods for the Naive Bayes Classifier (NBC) when the data under study are described by a large number of redundant binary indicators. Wrapper approaches guided by the NBC estimation of the classification error probability out-perform filter approaches while retaining …