New approach to avoid bad incentives in reinforcement learning agents.
problem Designing safe reinforcement learning agents that avoid unnecessary disruptions.
method Break down side effects penalties into baseline state and deviation measure; introduce new stepwise inaction baseline and relative reachability deviation measure.
result Combination of new design choices avoids undesirable incentives, while simpler alternatives fail.
SplitWise enhances stepwise regression by adaptively encoding numeric predictors into binary features.
problem Capturing nonlinear relationships in regression models without sacrificing interpretability.
method Adaptive encoding of numeric predictors into binary features using shallow decision trees, assessed by AIC or BIC.
result Consistently produces more parsimonious and generalizable models than traditional techniques.
The current study proposes a dimension reduction method, stepwise support vector machine (SVM), to reduce the dimensions of large p small n datasets. The proposed method is compared with other dimension reduction methods, namely, the Pearson product difference correlation coefficient (PCCs), recursive feature eliminati…
A novel stepwise VI method using vine copulas for complex latent dependence.
problem Modeling complex latent dependence structures in probabilistic models.
method Stepwise estimation of vine copula parameters using Rényi divergence and a stopping criterion.
result Our method outperforms mean-field VI and is more parsimonious in complex applications.
Lasso is a widely used regression technique to find sparse representations. When the dimension of the feature space and the number of samples are extremely large, solving the Lasso problem remains challenging. To improve the efficiency of solving large-scale Lasso problems, El Ghaoui and his colleagues have proposed th…
A three-state model based on the Potts model is proposed to simulate financial markets. The three states are assigned to "buy", "sell" and "inactive" states. The model shows the main stylized facts observed in the financial market: fat-tailed distributions of returns and long time correlations in the absolute returns. …
Proposes a new algorithm for Sparse Bayesian Learning connected to Stepwise Regression.
problem Sparse Bayesian Learning for probabilistic models.
method Coordinate ascent algorithm (RMP) for SBL, showing connection to Stepwise Regression.
result RMP's noise variance parameter limit connects to Stepwise Regression, with derived guarantees.
In this article, we advocate the ensemble approach for variable selection. We point out that the stochastic mechanism used to generate the variable-selection ensemble (VSE) must be picked with care. We construct a VSE using a stochastic stepwise algorithm, and compare its performance with numerous state-of-the-art algo…
Proposes a new feature selection method integrating feature relationships.
problem Feature selection in machine learning models.
method Integrates feature-feature and feature-target relationships via penalized mRMR.
result Correctly identifies inactive features, reducing false discoveries.
Paper improves k-NN predictive performance with efficient variable selection.
problem Improving predictive performance of k-NN models. method Efficient forward selection of predictor variables.
result Novel approach approaches outperformance of stepwise selection models.
Develops a more powerful selective inference method for stepwise feature selection.
problem Loss of power in existing conditional SI methods due to over-conditioning.
method Uses homotopy continuation approach to overcome over-conditioning.
result Shows improved power and efficiency in selective inference for feature selection.
Proposes a novel BO algorithm for stepwise model selection in sequence prediction.
problem Optimal model selection in sequential settings where model performance varies over time.
method Bayesian optimization with deep kernel learning to handle multiple black-box functions.
result Outperforms standard and multi-objective BO algorithms on sequence prediction tasks.
Safe screening rule reduces computational costs for Group OWL models.
problem High computational costs and memory usage in solving Group OWL models.
method Safe screening rule for Group OWL models that identifies and removes inactive features.
result Significant computational gain and memory savings achieved without loss of accuracy.
We propose a general interpretation for long-range correlation effects in the activity and volatility of financial markets. This interpretation is based on the fact that the choice between `active' and `inactive' strategies is subordinated to random-walk like processes. We numerically demonstrate our scenario in the fr…
Adaptive batching improves Gaussian process surrogates for noisy level set estimation.
problem Learning the level set of noisy simulator responses.
method Developed four novel adaptive batching schemes for Gaussian process metamodels.
result Adaptive batching brings significant computational speed-ups with minimal loss of modeling fidelity.
A new algorithm speeds up feature selection by 65.77%.
problem Efficiently selecting features in high-dimensional data.
method Dropping Forward-Backward algorithm, reducing rescanning of features.
result Significantly faster than stepwise procedure, maintaining good performance.
Safe screening rule improves Group SLOPE efficiency.
problem Efficiently selecting groups of predictors in high-dimensional sparse learning.
method Safe screening rule for Group SLOPE, addressing block non-separable group effects.
result Significant computational efficiency gains without sacrificing accuracy.
We study the conditions for a nilpotent Lie group to be foliated into subgroups that have square integrable (relative discrete series) unitary representations, that fit together to form a filtration by normal subgroups. Then we use that filtration to construct a class of "stepwise square integrable" representations on …
A new random forest algorithm uncovers feature interdependencies better than traditional methods.
problem Tackles the sub-optimality of greedy decision tree implementations in random forests.
method Presented a 'stepwise lookahead' variation of random forests that considers multiple split nodes simultaneously.
result Significantly outperforms greedy random forests in uncovering feature interdependencies, especially in high-noise environments.
Maximal Rate of Stepwise Uncertainty Reduction selects simulations to reduce uncertainty efficiently.
problem Efficiently estimating quantities of interest from multi-fidelity simulations.
method Bayesian sequential strategy that maximizes the ratio of expected uncertainty reduction to simulation cost.
result MR-SUR strategy unifies and provides principled approaches to develop new methods.
Bayesian method corrects for model selection multiplicity in regression.
problem Model selection multiplicity in regression analysis.
method Developed a Bayesian prior distribution based on Holm procedure analogy.
result Adequate multiplicity correction requires sparsity not provided by recommended priors.
Flexible device participation improves federated learning convergence.
problem Strict device participation limits federated learning reach.
method Analytical results and new aggregation scheme for flexible participation.
result Convergence improved with flexible device participation.
Spiking neural networks perform similarly to deep networks on occluded images.
problem Robust object recognition in partially occluded images.
method Developed a two-layer spiking neural network trained on natural scenes with a biologically plausible learning rule, compared to deep convolutional networks.
result Spiking neural networks achieve good accuracy and robustness on stepwise pixel erasement tasks.
StepGAN improves conditional sequence generation with less computation.
problem Stabilizing training of sequence generative adversarial networks (SeqGAN).
method Modified discriminator assigns scores at each generation step.
result StepGAN outperforms previous methods on synthetic and chit-chat tasks.
Improved equation learning accuracy via comprehensive R²-elimination and Bayesian model selection.
problem Challenges in exhaustive equation learning due to multicollinearity and greedy steps.
method Combines R2 and Bayesian model evidence for a comprehensive yet efficient search. result Our approach surpasses all other methods in identification accuracy, especially in exact equation recovery.
New framework improves classification accuracy using Pillai's trace and ULDA.
problem Traditional LDA's limitations in noise sensitivity and non-invertible matrices.
method Integrates Pillai's trace with ULDA for a unified classifier.
result Effective control of Type I error rates and improved classification accuracy.
We proposed a model of interacting market agents based on the Ising spin model. The agents can take three actions: "buy," "sell," or "stay inactive." We defined a price evolution in terms of the system magnetization. The model reproduces main stylized facts of real markets such as: fat-tailed distribution of returns an…
Screening is an effective technique for speeding up the training process of a sparse learning model by removing the features that are guaranteed to be inactive the process. In this paper, we present a efficient screening technique for sparse support vector machine based on variational inequality. The technique is both …
Bayesian method for estimating inputs leading to specific probability outputs.
problem Estimating inputs for specific probability outputs of uncertain functions.
method Bayesian strategy using Gaussian process modeling and SUR principle.
result Surpassed performance of existing methods through numerical experiments.
ET-Lasso improves Lasso feature selection in high-dimensional data.
problem Challenges in feature selection consistency of Lasso in high-dimensional data.
method ET-Lasso uses pseudo-features to separate active and inactive features by adding permuted features.
result ET-Lasso effectively selects active features under various scenarios.
Study compares RL and DT-based control for hedging European call options.
problem Optimizing hedging strategies for European call options with transaction costs.
method Reinforcement Learning vs. Deep Trajectory-based Stochastic Control.
result RL and DT-based methods perform differently under stepwise mean-variance hedging.
Theoretical work shows integrating coherent reasoning improves LLM performance and error correction.
problem Improving reasoning and error correction in large language models (LLMs) with few-shot prompting.
method Theoretical analysis and sensitivity experiments on transformer behavior with coherent reasoning and corrupted demonstrations.
result The transformer gains better error correction ability and more accurate predictions when coherent reasoning is integrated.
A new distributed method speeds up sparse model training.
problem Efficiently training models with massive samples and high-dimensional features.
method Distributed Dynamic Safe Screening (DDSS) method for sparsity regularized models.
result Achieves linear convergence rate and eliminates almost all inactive features.
A new Bayesian optimization method tackles constrained optimization with uncertainties.
problem Optimizing functions with uncertain constraints.
method Bayesian optimization with a new acquisition criterion.
result The new criterion optimizes both objective function improvement and constraint reliability.
Sparse support vector machine (SVM) is a popular classification technique that can simultaneously learn a small set of the most interpretable features and identify the support vectors. It has achieved great successes in many real-world applications. However, for large-scale problems involving a huge number of samples a…
I consider unsupervised extensions of the fast stepwise linear regression algorithm \cite{efroymson1960multiple}. These extensions allow one to efficiently identify highly-representative feature variable subsets within a given set of jointly distributed variables. This in turn allows for the efficient dimensional reduc…
Least Angle Regression is a promising technique for variable selection applications, offering a nice alternative to stepwise regression. It provides an explanation for the similar behavior of LASSO (ℓ1-penalized regression) and forward stagewise regression, and provides a fast implementation of both. The idea has…
A new screening method speeds up submodular function minimization.
problem Efficiently solving submodular function minimization problems in large-scale applications.
method Safe screening method to identify active and inactive elements in SFM optimization.
result Significant speedups achieved on synthetic and real datasets.
LFM learns a sequence of smaller models to generate data from noise.
problem Learning continuous, invertible flows between distributions.
method Stepwise Local Flow Matching (LFM) model, matching diffusion processes up to time-step size.
result LFM achieves competitive generative performance compared to Flow Matching.
A model simulates financial market dynamics with three states: buy, sell, or inactive.
problem Simulate financial market dynamics with heterogeneous investor types.
method Agent-based model with three states: noise traders and contrarians. Probabilistic dynamics with local and global influences.
result Model accurately captures real financial time series features like heavy-tailed distributions and volatility clustering.
StepMix estimates mixture models with covariates for social science applications.
problem Estimating latent classes with covariates in social science models.
method Pseudo-likelihood estimation using one-, two-, and three-step approaches.
result Unified framework for expectation-maximization subroutines.
New method for accurate permutation inference in CCA.
problem Inaccurate permutation inference in CCA.
method Proposed solutions for permutation inference in CCA, including transforming residuals and stepwise estimation.
result Valid permutation tests for CCA with and without nuisance variables.
Many in-hospital mortality risk prediction scores dichotomize predictive variables to simplify the score calculation. However, hard thresholding in these additive stepwise scores of the form "add x points if variable v is above/below threshold t" may lead to critical failures. In this paper, we seek to develop risk pre…
In the present work we introduce a stochastic cellular automata model in order to simulate the dynamics of the stock market. A direct percolation method is used to create a hierarchy of clusters of active traders on a two dimensional grid. Active traders are characterised by the decision to buy, (+1), or sell, (-1), a …
Generative model learns to mimic time-series behavior to generate accurate trajectories.
problem Learning generative models for time-series data with accurate multi-step trajectories.
method Contrastive imitation framework combining autoregressive and adversarial elements.
result The method generates accurate and useful samples from real-world datasets.
We compute approximate solutions to L0 regularized linear regression using L1 regularization, also known as the Lasso, as an initialization step. Our algorithm, the Lass-0 ("Lass-zero"), uses a computationally efficient stepwise search to determine a locally optimal L0 solution given any L1 regularization solution. We …
This paper explores the following question: what kind of statistical guarantees can be given when doing variable selection in high-dimensional models? In particular, we look at the error rates and power of some multi-stage regression methods. In the first stage we fit a set of candidate models. In the second stage we s…
Investors seek to attribute performance to various features using Shapley value method.
problem Attributing performance to different features in an investment process.
method Use Shapley value method for attribution, either exactly or approximately.
result Shapley value method provides a preferred attribution approach.