SRF learns sparse rule models by screening out features efficiently.
problem Learning optimal sparse rule models is computationally intractable due to the large number of possible rules.
method SRF uses meta safe screening (mSS) to efficiently screen out multiple features, improving the learning of sparse rule models.
result SRF provides a general framework for fitting sparse rule models and can handle group regularization.
FIRE extracts interpretable rules from tree ensembles.
problem Building sparse and interpretable rule sets from tree ensembles.
method Optimization-based framework with fusion regularization and sparsity-inducing penalties.
result FIRE outperforms state-of-the-art rule ensemble algorithms.
Sparse oblique decision tree improves security rules for renewable power systems.
problem Identifying secure operating conditions in power systems with high renewable energy.
method Sparse weighted oblique decision tree to learn and embed linear security rules.
result The method significantly increases secure states and reduces solution time.
New rules reduce SLOPE model fitting time by screening out irrelevant variables.
problem Expensive tuning of regularization parameter in penalized regression models.
method Strong screening rules for group-based SLOPE models.
result Significant acceleration of fitting process for Group SLOPE and sparse-group SLOPE.
A new screening rule improves lasso solving speed.
problem Efficiently solving lasso problems with high correlation.
method Uses second-order information from the Hessian to screen predictors.
result Outperforms alternatives on simulated and real data.
In high dimensional settings, sparse structures are crucial for efficiency, either in term of memory, computation or performance. In some contexts, it is natural to handle more refined structures than pure sparsity, such as for instance group sparsity. Sparse-Group Lasso has recently been introduced in the context of l…
A method for concise fuzzy system modeling using ESSC-SL-CTSK-FS.
problem Complex nonlinear systems with high-dimensional data and large numbers of rules.
method Integrating ESSC for antecedents and SL for consequent parameters optimization.
result Effective reduction in the number of fuzzy rules for clearer and more interpretable models.
Proposes sparse local and regional counterfactual rules for robust recourses.
problem Challenges in counterfactual explanations, especially stability, synthesis, and implementation.
method Probabilistic framework using Random Forest to derive sparse local and regional counterfactual rules.
result Effective recourses derived from high-density regions, providing sparse and robust counterfactual rules.
Safe screening rule reduces computational costs for Group OWL models.
problem High computational costs and memory usage in solving Group OWL models.
method Safe screening rule for Group OWL models that identifies and removes inactive features.
result Significant computational gain and memory savings achieved without loss of accuracy.
A new screening rule 'dynamic Sasvi' improves sparse optimization speed.
problem Sparse optimization problem identification.
method Flexible framework based on Fenchel-Rockafellar duality for norm-regularized least squares.
result Dynamic Sasvi can eliminate more features and increase solver speed.
New method improves model explainability and accuracy with low computational cost.
problem Improving model explainability and accuracy in classification models.
method Distributionally robust optimization to learn sparse ensembles of rule sets.
result Improves model performance on various metrics compared to competing methods.
The l1-regularized logistic regression (or sparse logistic regression) is a widely used method for simultaneous classification and feature selection. Although many recent efforts have been devoted to its efficient implementation, its application to high dimensional data still poses significant challenges. In this paper…
Methodology for learning sparse models using all multiplicative interactions efficiently.
problem Learning high-order feature interactions with fine control.
method Fine Control Kernel framework, combining Fenchel Duality and Apriori algorithm.
result Efficiently solves large sparse learning problems with sparse feature screening rules.
Introduces screening rules for non-convex Lasso problems.
problem Efficiently solving non-convex Lasso problems with theoretical guarantees.
method Iterative majorization-minimization strategy with screening rule.
result Significant computational gain compared to classical methods.
DFR reduces the computational cost of sparse-group lasso and adaptive sparse-group lasso.
problem Sparse-group lasso's computational expense and need for tuning.
method Dual Feature Reduction (DFR) using strong screening rules and dual norms.
result DFR drastically reduces computational cost without affecting solution optimality.
High dimensional regression benefits from sparsity promoting regularizations. Screening rules leverage the known sparsity of the solution by ignoring some variables in the optimization, hence speeding up solvers. When the procedure is proven not to discard features wrongly the rules are said to be \emph{safe}. In this …
Safe screening rule improves Group SLOPE efficiency.
problem Efficiently selecting groups of predictors in high-dimensional sparse learning.
method Safe screening rule for Group SLOPE, addressing block non-separable group effects.
result Significant computational efficiency gains without sacrificing accuracy.
Meta decision trees explain user ratings in recommendation systems.
problem Building explainable recommendation systems with clear user explanations.
method Learned regression functions and sparse decision rules based on user embeddings.
result The method provides accurate and interpretable ratings.
Safe screening rules reduce computation time in logistic regression with ℓ0−ℓ2 regularization.
problem Efficiently solving logistic regression with many features and regularization.
method Screening rules based on Fenchel dual lower bounds of strong conic relaxations.
result A high percentage of features can be safely removed before solving, leading to substantial speed-up.
MOSS optimizes decision rules for accuracy and stability.
problem Constructing stable sets of decision rules.
method Multi-objective optimization framework incorporating sparsity, accuracy, and stability.
result MOSS outperforms state-of-the-art rule ensembles in predictive performance and stability.
Unified quadrature framework for large-scale kernel machines.
problem Efficiently approximating kernel functions for large-scale machine learning.
method Deterministic and randomized interpolatory rules for numerical integration of kernel functions.
result The proposed method reduces the number of nodes needed for accurate kernel approximation.
A new screening rule speeds up OWL regression solving.
problem High-dimensional sparse learning with OWL regression's computational cost and memory usage.
method Safe screening rule for OWL regression using iterative strategy.
result Significant computational gain without accuracy loss.
Sparse neural encoding can store more memories as targets become sparser.
problem Storing sparse input-target associations in neural networks.
method Mathematical proofs using properties of random polytopes and sub-gaussian random vector variables.
result The capacity of neural maps increases with sparsity in target layers.
We present the design and implementation of a custom discrete optimization technique for building rule lists over a categorical feature space. Our algorithm produces rule lists with optimal training performance, according to the regularized empirical risk, with a certificate of optimality. By leveraging algorithmic bou…
A method for learning transition models in uncertain domains using relational rules and neural networks.
problem Learning transition models in complex, uncertain domains.
method Relational rules, greedy algorithm, neural networks.
result The method is more versatile and sample efficient than monolithic models.
We develop a class of rules spanning the range between quadratic discriminant analysis and naive Bayes, through a path of sparse graphical models. A group lasso penalty is used to introduce shrinkage and encourage a similar pattern of sparsity across precision matrices. It gives sparse estimates of interactions and pro…
New learning rule simplifies Bayesian updates for deep learning.
problem Bayesian learning rule's complexity and manifold constraints.
method Lie-group approach to simplify Bayesian updates.
result New algorithm learns sparse features in deep learning.
Proposes a tail-adaptive shrinkage method for robust sparse estimation.
problem Robust Bayesian methods for high-dimensional regression under diverse sparse regimes.
method Global-local-tail (GLT) Gaussian mixture distribution with tail-adaptive shrinkage.
result GLT posterior contracts at minimax optimal rate for sparse normal mean models.
Taking into account high-order interactions among covariates is valuable in many practical regression problems. This is, however, computationally challenging task because the number of high-order interaction features to be considered would be extremely large unless the number of covariates is sufficiently small. In thi…
A new matrix factorization method for high-dimensional data.
problem Exploiting sparse structures in complex data for better interpretability.
method Bayesian shrinkage priors and flexible sparse patterns modeled through row and column dependencies.
result Demonstrated practical advantages through simulation and soccer heatmap analysis.
RIPE is a novel deterministic and easily understandable prediction algorithm developed for continuous and discrete ordered data. It infers a model, from a sample, to predict and to explain a real variable Y given an input variable X∈X (features). The algorithm extracts a sparse set of hyperrectangles $…
SIRUS creates interpretable rules from random forests for regression.
problem Lack of interpretability in complex machine learning models.
method Random forest with rule extraction for stability and simplicity.
result SIRUS produces stable and interpretable rule sets.
We consider the following conditional linear regression problem: the task is to identify both (i) a k-DNF condition c and (ii) a linear rule f such that the probability of c is (approximately) at least some given bound μ, and f minimizes the ℓp loss of predicting the target z in the distribution of …
A privacy-preserving algorithm for high-dimensional bandits.
problem High-dimensional stochastic contextual linear bandits with sparse parameters under privacy constraints.
method PrivateLASSO algorithm based on sparse hard-thresholding and episodic thresholding.
result Minimax private lower bounds and utility guarantees for PrivateLASSO.
In high dimensional regression settings, sparsity enforcing penalties have proved useful to regularize the data-fitting term. A recently introduced technique called screening rules propose to ignore some variables in the optimization leveraging the expected sparsity of the solutions and consequently leading to faster s…
We present sparse tree-based and list-based density estimation methods for binary/categorical data. Our density estimation models are higher dimensional analogies to variable bin width histograms. In each leaf of the tree (or list), the density is constant, similar to the flat density within the bin of a histogram. His…
A new framework compresses neural networks using sparse optimization.
problem Efficiently reducing the size of deep neural networks for practical deployment.
method Sparse optimization for model compression, tailored for stochastic learning.
result Up to 7.2 and 2.9 times FLOPs reduction with comparable accuracy.
Olshausen and Field (OF) proposed that neural computations in the primary visual cortex (V1) can be partially modeled by sparse dictionary learning. By minimizing the regularized representation error they derived an online algorithm, which learns Gabor-filter receptive fields from a natural image ensemble in agreement …
Improved Sparse Polyak for high-dimensional M-estimation with sparser solutions.
problem High-dimensional M-estimation problems with potential loss of sparsity and accuracy.
method Variant of Sparse Polyak with optimal thresholding operators.
result Retains desirable scaling properties while achieving sparser and more accurate solutions.
New method improves IV estimation with many weak and invalid instruments.
problem Identification in linear IV models with unknown validity.
method Non-convex penalized approaches, surrogate sparsest penalty.
result Advantages over other IV estimators in selection consistency and weak IV strength conditions.
There has been significant recent work on the theory and application of randomized coordinate descent algorithms, beginning with the work of Nesterov [SIAM J. Optim., 22(2), 2012], who showed that a random-coordinate selection rule achieves the same convergence rate as the Gauss-Southwell selection rule. This result su…
As a contribution to interpretable machine learning research, we develop a novel optimization framework for learning accurate and sparse two-level Boolean rules. We consider rules in both conjunctive normal form (AND-of-ORs) and disjunctive normal form (OR-of-ANDs). A principled objective function is proposed to trade …
Paper introduces probabilistic SNNs for efficient learning.
problem Training SNNs with biological plausibility and energy efficiency.
method Discrete-time probabilistic models, variational inference.
result Derives supervised and unsupervised learning rules.
We study the sparse non-negative least squares (S-NNLS) problem. S-NNLS occurs naturally in a wide variety of applications where an unknown, non-negative quantity must be recovered from linear measurements. We present a unified framework for S-NNLS based on a rectified power exponential scale mixture prior on the spars…
Method learns cell interaction rules from individual trajectories.
problem Inferring interaction rules from heterogeneous cellular data.
method WSINDy for second order IPSs, learning individual cell models.
result Efficiently identifies different species and best-fit models for each.
Many leading classification algorithms output a classifier that is a weighted average of kernel evaluations. Optimizing these weights is a nontrivial problem that still attracts much research effort. Furthermore, explaining these methods to the uninitiated is a difficult task. Letting all the weights be equal leads to …
The pseudo-likelihood method is one of the most popular algorithms for learning sparse binary pairwise Markov networks. In this paper, we formulate the L1 regularized pseudo-likelihood problem as a sparse multiple logistic regression problem. In this way, many insights and optimization procedures for sparse logistic…
This paper introduces probabilistic SNNs for efficient neural processing.
problem Training algorithms for SNNs lag behind hardware implementations.
method Discrete-time probabilistic models and variational inference.
result Derivation of learning rules for SNNs from first principles.