Selective reinitialization improves adaptability of neural bandits in dynamic environments.
problem Loss of plasticity in neural bandits, leading to rigid neural network parameters.
method Selective Reinitialization (SeRe) framework that dynamically resets underutilized units.
result SeRe enhances adaptability of CNB algorithms, reducing cumulative regret in dynamic environments.
Gradients help find global optima in complex functions.
problem Finding global optima in functions with many local minima.
method A principle for generating search directions from non-local quadratic approximants based on gradients.
result The proposed algorithm and CMA-ES perform better than random reinitialized BFGS.
Heuristic optimisers which search for an optimal configuration of variables relative to an objective function often get stuck in local optima where the algorithm is unable to find further improvement. The standard approach to circumvent this problem involves periodically restarting the algorithm from random initial con…
Study examines effects of pruning techniques on deep learning models.
problem Understanding the impact of pruning methods on deep learning model structure and dynamics.
method Investigated differences in connectivity and learning dynamics of pruned models using various iterative pruning techniques.
result Emergence of structure in pruned models through magnitude-based unstructured pruning and weight rewinding.
We study the quantum synchronization between a pair of two-level systems inside two coupled cavities. By using a digital-analog decomposition of the master equation that rules the system dynamics, we show that this approach leads to quantum synchronization between both two-level systems. Moreover, we can identify in th…
The recent "Lottery Ticket Hypothesis" paper by Frankle & Carbin showed that a simple approach to creating sparse networks (keeping the large weights) results in models that are trainable from scratch, but only when starting from the same initial weights. The performance of these networks often exceeds the performance …
R. Cont and A. de Larrard (SIAM J. Finan. Math, 2013) introduced a tractable stochastic model for the dynamics of a limit order book, computing various quantities of interest such as the probability of a price increase or the diffusion limit of the price process. As suggested by empirical observations, we extend their …
Early neural network training reveals important sub-networks and weight distributions.
problem Understanding the early phases of neural network training.
method Extensive measurements and quantitative probing of weight distribution and dataset reliance.
result Deep networks are not robust to reinitializing with random weights while maintaining signs, and weight distributions are highly non-independent.
Model for cross-border markets with limited transmission capacities.
problem Limited transmission capacities between two countries' markets.
method Developed a regime-switching process model with high-frequency approximation.
result Analytic tractability allows computation of key market quantities.
A real-time federated neural architecture search approach reduces costs and improves performance.
problem High communication and computational demands in federated learning for large models.
method Evolutionary approach with double-sampling technique to optimize model performance and reduce costs.
result Effective real-time federated neural architecture search for deep models on edge devices.
RIFLE improves deep transfer learning by reinitializing fully-connected layers.
problem Limited improvement in transfer learning accuracy with pre-trained models on small datasets.
method Re-Initializing fully-connected layers with random scratch during fine-tuning.
result Significant improvement in deep transfer learning accuracy across various datasets.
Deep learning predicts curvature of 2D interfaces in level-set method.
problem Estimating curvature in level-set method for complex interfaces.
method Deep learning using feed-forward neural networks trained on synthetic data.
result Deep learning models approximate curvature with comparable precision to traditional methods.
Paper introduces a simulator-free approach to reinforcement learning policy distillation.
problem Learning multiplicity of cases corresponding to a given action in reinforcement learning.
method Generative adversarial approach to find multiple exemplars for each output class.
result Improves over state-of-the-art on data-free learning of student networks.
New method estimates selection event for black-box models.
problem Infeasibility of conditional selective inference due to unavailable selection event.
method Bootstrapping to estimate selection event and conditional distribution.
result Feasibility of conditional selective inference for models without exact selection event.
Cluster stability selection improves feature selection in correlated data.
problem Feature selection stability in correlated data.
method Cluster stability selection exploiting known cluster structure.
result Better predictive performance than lasso alone and stability selection.
New method corrects selection bias in post-selective inference for Group LASSO.
problem Inference after Group LASSO selection is unreliable.
method Develops a consistent, post-selective Bayesian method to adjust for selection bias.
result Corrects bias in recovering effects of selected variables.
Proposes PEMI for online selective conformal prediction with asymmetric rules.
problem Challenges of handling asymmetric selection mechanisms in online selective conformal prediction.
method PEMI: permutation-based framework for selective conformal prediction with arbitrary asymmetric selection rules.
result Achieves exact selection-conditional coverage for any asymmetric selection mechanism and any prediction model.
The paper addresses errors in online selective conformal prediction and proposes new strategies to ensure valid inference.
problem Online selective conformal prediction's exchangeability issues and false coverage rate control problems.
method Evaluation and correction of existing calibration selection strategies, proposing new ones that preserve exchangeability.
result Novel calibration selection strategies ensure both selection-conditional coverage and FCR control.
This work develops scalable model selection methods with fast update and selection.
problem Efficient model selection for large pools of candidate models.
method Isolated model embedding, which supports asymptotically fast update and selection.
result Standardized Embedder achieves competitive model selection performances.
Online method selects candidates from data streams, ensuring irreversible decisions.
problem Conformal selection's incompatibility with irreversible decisions in online scenarios.
method Online Conformal Selection with Accept-to-Reject Changes (OCS-ARC) incorporating online Benjamini-Hochberg procedure.
result OCS-ARC controls FDR at or below nominal level, improving selection power.
Paper improves k-NN predictive performance with efficient variable selection.
problem Improving predictive performance of k-NN models. method Efficient forward selection of predictor variables.
result Novel approach approaches outperformance of stepwise selection models.
Value selection reduces model size while maintaining accuracy.
problem Space efficiency in model size reduction.
method Two probabilistic methods based on information theory's metric: PVS and P + VS.
result Value selection achieves balance between accuracy and model size reduction.
Proposes HSIC-Lasso for selective inference in non-linear data.
problem Detecting influential features in non-linear and high-dimensional data.
method Model-free HSIC-Lasso based on truncated Gaussians and polyhedral lemma.
result Tight control of type-I error even for small sample sizes.
Paper proposes a statistical test for feature selection pipelines using selective inference.
problem Assessing the significance of feature selection pipelines in data analysis.
method Selective inference technique applied to feature selection pipelines composed of various algorithms.
result The proposed statistical test controls false positive feature selection probabilities.
New method improves feature selection by integrating stability paths.
problem Improving feature selection with tighter false positive control.
method Integrating stability paths to strengthen theoretical bounds on E(FP).
result Significantly more true positives with same E(FP) control.
UniFeat is an open-source Java tool for feature selection.
problem Efficient feature selection in various research areas.
method Provides a set of advanced feature selection methods.
result Facilitates rapid development of new feature selection algorithms.
We develop a general approach to valid inference after model selection. At the core of our framework is a result that characterizes the distribution of a post-selection estimator conditioned on the selection event. We specialize the approach to model selection by the lasso to form valid confidence intervals for the sel…
Flexible selective inference using flow-based transport maps.
problem Selective inference with complex selection events.
method Flow-based generative modeling for conditional distribution approximation.
result Valid p-values and confidence sets for adaptively selected hypotheses and parameters.
Recently, many regularized procedures have been proposed for variable selection in linear regression, but their performance depends on the tuning parameter selection. Here a criterion for the tuning parameter selection is proposed, which combines the strength of both stability selection and cross-validation and therefo…
Develops a more powerful selective inference method for stepwise feature selection.
problem Loss of power in existing conditional SI methods due to over-conditioning.
method Uses homotopy continuation approach to overcome over-conditioning.
result Shows improved power and efficiency in selective inference for feature selection.
Paper tackles selective regression using uncertainty estimation.
problem Selective regression for machine learning models to avoid predictions when uncertain.
method Model-agnostic non-parametric uncertainty estimation.
result Superior performance compared to state-of-the-art selective regressors.
Online selection of dynamic features has attracted intensive interest in recent years. However, existing online feature selection methods evaluate features individually and ignore the underlying structure of feature stream. For instance, in image analysis, features are generated in groups which represent color, texture…
A new method selects clean samples to train DNNs with noisy labels.
problem Training deep neural networks with noisy labeled data.
method Adaptive k-set selection to choose clean samples at each epoch.
result The method guarantees performance with a theoretical bound on regret.
IWeS selects examples by entropy-based importance sampling for subset selection.
problem Efficiently selecting examples for model training in batch settings.
method IWeS uses importance sampling based on model entropy to select examples.
result IWeS outperforms other subset selection algorithms on seven datasets.
ACS is an interactive framework for model-free selection with guaranteed error control.
problem Model-free selection with rigorous error control.
method Adaptive conformal selection with human-in-the-loop data exploration and new information incorporation.
result ACS provides concrete selection algorithms for various goals, including model update/selection, diversified selection, and incorporating new data.
NGP selects N features from P using neural networks in a greedy, iterative process.
problem Feature selection for non-linear prediction problems.
method Neural Greedy Pursuit (NGP) algorithm, selecting features sequentially in an iterative loss minimization procedure.
result NGP provides better performance than DeepLIFT and Drop-one-out loss methods.
Selective inference for group lasso estimators across various distributions and covariates.
problem Developing selective inference methods for group lasso estimators.
method Randomized group-regularized optimization problem with post-selection likelihood.
result Selective point estimator and Wald-type confidence regions for regression parameters.
Develops SCMs for latent selection to simplify causal analysis.
problem Latent selection complicates causal analysis.
method Introduces a conditioning operation for SCMs to encode latent selection.
result Conditioning operation preserves simplicity, acyclicity, and linearity of SCMs.
New method stabilizes model selection with theoretical guarantees.
problem Stability of model selection methods in the face of noisy or incomplete data.
method Combines bagging with an 'inflated' argmax operation to select a stable set of models.
result Stable model selection with high probability of overlap after removing any data point.
Suppose some classifiers are selected from a set of hypothesis classifiers to form an equally-weighted ensemble that selects a member classifier at random for each input example. Then the ensemble has an error bound consisting of the average error bound for the member classifiers, a term for selectivity that varies fro…
This paper analyzes and improves convergence in federated learning with biased client selection.
problem Analyzing convergence in federated learning with biased client selection.
method First convergence analysis of federated optimization for biased client selection strategies, proposing Power-of-Choice framework.
result Power-of-Choice strategies converge up to 3 times faster and give 10% higher test accuracy than random selection.
GOLFS selects features for clustering by combining global and local information.
problem Feature selection for high-dimensional clustering without labels.
method Combines global and local information via manifold learning and regularized self-representation.
result Improves feature selection and clustering accuracy.
Meta-algorithm selection aims to choose the best algorithm selector for a given problem instance.
problem Selecting the best algorithm selector for a specific problem instance.
method Apply algorithm selection to the selection of other algorithms (meta-algorithm selection).
result Meta-algorithm selection can be beneficial in some cases but faces challenges in solving the meta-level problem.
Paper proposes a novel unsupervised feature selection method using K-means and ADMM.
problem Finding a subset of features for high-dimensional unsupervised learning problems.
method Developed K-means Derived Unsupervised Feature Selection (K-means UFS) using ADMM to solve NP-hard optimization.
result K-means UFS outperforms baselines in feature selection for clustering.
A stability metric compares feature selection algorithms in machine learning.
problem Stability of feature selection algorithms in machine learning.
method Rank-based instability index to compare MDA, LIME, and SHAP algorithms.
result LIME and SHAP are more stable than MDA, with LIME being best for human interpretability.
A new method for feature selection robust to noise and design variability.
problem Feature selection in high-dimensional regression under sampling variability and measurement error.
method Injects controlled additive noise into the design matrix, fits a base selector, and aggregates selection frequencies.
result Improved robustness compared to Stability Selection and standard base selectors.
We describe a simple, efficient, permutation based procedure for selecting the penalty parameter in the LASSO. The procedure, which is intended for applications where variable selection is the primary focus, can be applied in a variety of structural settings, including generalized linear models. We briefly discuss conn…
In variable or graph selection problems, finding a right-sized model or controlling the number of false positives is notoriously difficult. Recently, a meta-algorithm called Stability Selection was proposed that can provide reliable finite-sample control of the number of false positives. Its benefits were demonstrated …