Variable selection is a fundamental task in statistical data analysis. Sparsity-inducing regularization methods are a popular class of methods that simultaneously perform variable selection and model estimation. The central problem is a quadratic optimization problem with an l0-norm penalty. Exactly enforcing the l0-no…
New group-sparse SVD models improve biclustering of gene expression data.
problem Identifying block patterns with similar expressions in high-dimensional gene expression data.
method Proposed GL1-SVD, GL0-SVD, OGL1-SVD, and OGL0-SVD models with group Lasso and L0-norm penalties, using alternating iterative strategies and ADMM.
result Effective in identifying biologically interpretable gene modules with gene prior group knowledge.
Paper proposes methods to train RBF networks under faults.
problem Training RBF networks with fault tolerance.
method Two novel ADMM-based algorithms using MCP and l0-norm.
result Both methods globally converge to a unique limit point.
New L0 norm added to TDA for market analysis.
problem Improving TDA tools for market prediction.
method Defined and applied L0 norm in TDA for four markets.
result Enhanced TDA tools for market analysis.
The paper develops a classification method using penalties on feature selection for high-dimensional data.
problem High-dimensional binary classification with many irrelevant features.
method Empirical risk minimization with l0-penalization for feature selection.
result The method achieves a sparse solution close to true sparsity with high probability and converges to low misclassification risk.
Improved sample efficiency in learning sparse Ising models.
problem Learning the graph of a sparse Ising model with limited samples.
method Combining L0 and L2 norms to induce sparsity and model non-zero coefficients.
result Improved sample complexity, achieving new state-of-the-art recovery guarantees.
New algorithm targets nonsmooth constraints for robust data interpolation and denoising.
problem Robust data interpolation and denoising with large outliers and varying amplitudes.
method Flexible algorithmic framework targeting nonsmooth level-set constraints (L1, Linf, L0 norms).
result Improved robustness to large outliers and significant amplitude variations in seismic data.
Learning the "blocking" structure is a central challenge for high dimensional data (e.g., gene expression data). Recently, a sparse singular value decomposition (SVD) has been used as a biclustering tool to achieve this goal. However, this model ignores the structural information between variables (e.g., gene interacti…
Proposes sparse QSVM for better generalization and interpretability.
problem Overfitting and difficulty in interpreting full quadratic classifiers.
method Enforces ℓ0-norm constraint to promote sparsity and develops a penalty decomposition algorithm. result The proposed model enhances generalization and produces sparse solutions.
We propose an algorithm for the non-negative factorization of an occurrence tensor built from heterogeneous networks. We use l0 norm to model sparse errors over discrete values (occurrences), and use decomposed factors to model the embedded groups of nodes. An efficient splitting method is developed to optimize the non…
This paper prunes deep neural networks by grouping channels and using a bounded L1-L0 norm.
problem Improving network efficiency by reducing the number of parameters in deep neural networks.
method Group-wise pruning with a bounded L1-L0 norm regularizer.
result Significant reduction in model size with minimal loss in accuracy.
Structured pruning method reduces RNN sizes and speeds up inference.
problem Large RNN models are hard to deploy on edge devices.
method Structured pruning through neuron selection, minimizing L0 norm of weight matrix.
result Nearly 20x speedup achieved without performance loss.
Signal recovery is one of the key techniques of Compressive sensing (CS). It reconstructs the original signal from the linear sub-Nyquist measurements. Classical methods exploit the sparsity in one domain to formulate the L0 norm optimization. Recent investigation shows that some signals are sparse in multiple domains.…
The paper explores nonconvex penalties for deep learning regularization.
problem Overfitting in deep learning neural networks.
method Examines and evaluates nonconvex penalties for DNN regularization.
result Nonconvex penalties, under certain conditions, can perform well in DNNs.
Method introduces topological regularization using information filtering networks.
problem Sparse probabilistic modeling and multicollinear regression.
method Topological regularization via information filtering network.
result Direct application to L0-norm regularized problems. Scoring systems are classification models that only require users to add, subtract and multiply a few meaningful numbers to make a prediction. These models are often used because they are practical and interpretable. In this paper, we introduce an off-the-shelf tool to create scoring systems that both accurate and inte…
Reweighted l1-algorithms have attracted a lot of attention in the field of applied mathematics. A unified framework of such algorithms has been recently proposed by Zhao and Li. In this paper we construct a few new examples of reweighted l1-methods. These functions are certain concave approximations of the l0-norm func…
Gradient penalty improves GAN performance by inducing a large-margin classifier.
problem Improving GAN performance and addressing vanishing gradients.
method A unifying framework of expected margin maximization, showing gradient penalties induce large-margin classifiers.
result Gradient penalties reduce vanishing gradients and produce better generated outputs.
Study examines insider trading with penalties, finding optimal penalties increase quickly for small orders.
problem Analyzing the impact of penalties on insider trading behavior and market efficiency.
method Formal economic model with penalty functions, existence and uniqueness theorems, and optimization.
result Optimal penalties increase quickly for small orders, signaling extreme events and incorporating information into prices.
One-bit measurements widely exist in the real world, and they can be used to recover sparse signals. This task is known as the problem of learning halfspaces in learning theory and one-bit compressive sensing (1bit-CS) in signal processing. In this paper, we propose novel algorithms based on both convex and nonconvex s…
The paper studies robust risk measures with linear penalties under uncertain distributions.
problem Risk measurement under distributional uncertainty.
method Robust distortion risk measures with linear penalty function under distributional constraints.
result Explicit characterization of optimal quantile distribution and value function.
The use of machine-learning in neuroimaging offers new perspectives in early diagnosis and prognosis of brain diseases. Although such multivariate methods can capture complex relationships in the data, traditional approaches provide irregular (l2 penalty) or scattered (l1 penalty) predictive pattern with a very limited…
Paper introduces a new SVR model using a combined reward and penalty loss function.
problem Regression problem, particularly handling data points outside and inside ε-tube.
method Combined reward cum penalty loss function to penalize and reward data points.
result Experimental results support the model's properties and effectiveness.
Goal: This paper deals with the problems that some EEG signals have no good sparse representation and single channel processing is not computationally efficient in compressed sensing of multi-channel EEG signals. Methods: An optimization model with L0 norm and Schatten-0 norm is proposed to enforce cosparsity and low r…
New sparse penalty improves biclustering for gene expression data.
problem Identifying significant clusters in gene expression data.
method Prenet penalty applied to SSVD for biclustering.
result Mixed Prenet penalty effectively clusters non-overlapped data.
New approach avoids excess empirical risk in domain generalization.
problem Learning models that generalize to unseen distributions from diverse data sets.
method Minimizes penalty under constraint of optimal empirical risk, leveraging rate-distortion theory.
result Significant improvements in domain generalization performance across multiple methods.
We study the problem of estimating high-dimensional regression models regularized by a structured sparsity-inducing penalty that encodes prior structural information on either the input or output variables. We consider two widely adopted types of penalties of this kind as motivating examples: (1) the general overlappin…
Curvature penalties improve interpretability of KANs without sacrificing accuracy.
problem Pathologically high-curvature oscillations in KANs activations make them hard to interpret.
method Derived a curvature penalty and proved an upper bound on model curvature.
result KANs with curvature penalties achieve substantially smoother activations while maintaining accuracy.
New method reduces bias in sparse Bayesian learning.
problem High sparsity in statistical models leads to significant bias.
method Variable-coefficient ℓ1 penalty with hyperpriors. result Reduces bias in sparse Bayesian learning.
PPO-B improves sampling efficiency by using a logarithmic barrier method.
problem Low sampling efficiency in PPO due to exterior penalty method.
method Introducing a surrogate objective with interior penalty method.
result PPO-B outperforms PPO in terms of sampling efficiency.
New nonconvex penalty smooths at origin for deep learning.
problem Improving variable selection and bias in high-dimensional statistical learning.
method Developed a new nonconvex penalty function smooth at origin.
result Asymptotic bias of new penalty function vanishes exponentially fast.
This study proves local stability of SGP μ-WGAN and shows penalizing data or sample manifold is key.
problem Stabilizing and regularizing WGAN with gradient penalty.
method Proves local stability of SGP μ-WGAN using measure valued differentiation.
result Penalizing data or sample manifold is key to regularizing WGAN.
We study the problem of learning high dimensional regression models regularized by a structured-sparsity-inducing penalty that encodes prior structural information on either input or output sides. We consider two widely adopted types of such penalties as our motivating examples: 1) overlapping group lasso penalty, base…
Multi-group learners suffer a penalty in transductive learning.
problem The penalty on multi-group learners in transductive learning.
method Analyzing the relationship between the number of groups and the error rate.
result The penalty can increase linearly with the number of groups, up to the square-root of the sample size.
Global minima found for multidimensional scaling with penalties.
problem Finding global minima in multidimensional scaling.
method Combining stress loss function with a quadratic penalty term to find minimizers.
result Trajectory of minimizers leads to global minima.
New method improves signal reconstruction with nonconvex penalties and parameter control.
problem Reconstructing sparse signals with nonconvex penalties and nonconvexity control.
method Introduces nonconvex penalties (SCAD, MCP) with nonconvexity parameters and controls them to guide AMP trajectory.
result Achieves perfect reconstruction for relatively dense signals with small nonconvexity parameters.
Improved penalty-based methods for bilevel optimization with reduced complexity.
problem Suboptimal complexity in solving bilevel optimization problems with large penalty terms.
method Novel penalty reformulation that decouples upper and lower-level variables, enabling larger step sizes and reduced iteration complexity.
result PBGD-Free algorithm that avoids inner loops for coupled constraint BLO problems, with reduced iteration complexity.
Proposes an alternative invariance penalty to address domain generalization issues.
problem Addressing domain generalization problems by finding invariant representations.
method Revisits the Gramian matrix of the data representation to propose an alternative invariance penalty.
result The proposed approach guarantees recovery of an invariant representation under mild conditions.
Paper proposes efficient algorithms for designing SLOPE penalty sequences.
problem Designing SLOPE penalty sequences is computationally expensive.
method Developed two efficient algorithms: PGD and CD for Gaussian and general data matrices respectively.
result Demonstrated improved mean squared error performance of SLOPE with designed penalties.
This paper proposes a new method for GLM estimation using distance penalties to handle constraints.
problem Handling constraints in generalized linear models (GLM) is complicated.
method The approach uses distance penalties to optimize the log-likelihood, avoiding shrinkage.
result Distance penalties provide a flexible and non-shrinking alternative to traditional penalties.
Paper proposes a new sparse group k-max regularization for sparsity constraints.
problem Linear inverse problems with sparsity constraints are NP-hard.
method Sparse group k-max regularization, iterative soft thresholding algorithm.
result Approximates l0 norm more closely and enhances group-wise and in-group sparsity.
Insider trading is reduced when penalized, affecting expected penalties in a non-monotone way.
problem Reducing insider trading behavior when insiders face legal penalties.
method Characterized via a backward stochastic differential equation (BSDE) with a non-linear operator.
result The insider's expected penalties are non-monotone in the fee structure and determined by relative entropy.
Survey on minimal penalty algorithms and slope heuristics.
problem Choosing optimal multiplicative constants from data.
method Minimal penalty and slope heuristics approach.
result Slope heuristics performs almost as well as residual-based estimators.
Sparse reconstruction approaches using the re-weighted l1-penalty have been shown, both empirically and theoretically, to provide a significant improvement in recovering sparse signals in comparison to the l1-relaxation. However, numerical optimization of such penalties involves solving problems with l1-norms in the ob…
Efficient ADMM algorithm solves nonconvex SVMs with various penalties.
problem Solving nonconvex penalized SVMs due to nondifferentiability, nonsmoothness, and nonconvexity.
method ADMM-based algorithm for a wide range of nonconvex penalties.
result The proposed algorithm outperforms other methods on benchmark datasets.
New method recovers signals from saturated data using linear loss and nonconvex penalties.
problem Signal recovery from saturated measurements with sign information loss.
method Linear loss and nonconvex penalties (e.g., minimax concave penalty, sorted ℓ1 norm).
result Estimation error is bounded and recovery performance improved.
In high-dimensional and/or non-parametric regression problems, regularization (or penalization) is used to control model complexity and induce desired structure. Each penalty has a weight parameter that indicates how strongly the structure corresponding to that penalty should be enforced. Typically the parameters are c…
The paper improves risk bounds for maximum likelihood estimation with arbitrary penalties.
problem Improving risk bounds for maximum likelihood estimation with arbitrary penalties.
method Developed a more general inequality for arbitrary penalties, leading to exact risk bounds of order 1/n.
result Derived exact risk bounds of order 1/n for iid parametric models, improving on previous bounds.