Regularization methods are often employed in deep learning neural networks (DNNs) to prevent overfitting. For penalty based DNN regularization methods, convex penalties are typically considered because of their optimization guarantees. Recent theoretical work have shown that nonconvex penalties that satisfy certain reg…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “penalty method”
We study the problem of estimating high-dimensional regression models regularized by a structured sparsity-inducing penalty that encodes prior structural information on either the input or output variables. We consider two widely adopted types of penalties of this kind as motivating examples: (1) the general overlappin…
New sparse penalty improves biclustering for gene expression data.
problem Identifying significant clusters in gene expression data.
method Prenet penalty applied to SSVD for biclustering.
result Mixed Prenet penalty effectively clusters non-overlapped data.
We study the problem of learning high dimensional regression models regularized by a structured-sparsity-inducing penalty that encodes prior structural information on either input or output sides. We consider two widely adopted types of such penalties as our motivating examples: 1) overlapping group lasso penalty, base…
New approach avoids excess empirical risk in domain generalization.
problem Learning models that generalize to unseen distributions from diverse data sets.
method Minimizes penalty under constraint of optimal empirical risk, leveraging rate-distortion theory.
result Significant improvements in domain generalization performance across multiple methods.
Proximal policy optimization(PPO) has been proposed as a first-order optimization method for reinforcement learning. We should notice that an exterior penalty method is used in it. Often, the minimizers of the exterior penalty functions approach feasibility only in the limits as the penalty parameter grows increasingly…
Efficient Penalty-Based Bilevel Methods: Improved Analysis, Novel Updates, and Flatness Conditionmath.OC
Improved penalty-based methods for bilevel optimization with reduced complexity.
problem Suboptimal complexity in solving bilevel optimization problems with large penalty terms.
method Novel penalty reformulation that decouples upper and lower-level variables, enabling larger step sizes and reduced iteration complexity.
result PBGD-Free algorithm that avoids inner loops for coupled constraint BLO problems, with reduced iteration complexity.
A fast method estimates group-adaptive elastic net penalties using co-data.
problem Computational inefficiency in estimating group-adaptive elastic net penalties.
method Derive low-dimensional representation of Taylor approximation for marginal likelihood and its derivative for group-adaptive ridge penalties; approximate elastic net marginal likelihood by ridge; transform ridge penalties to elastic net penalties.
result Significantly decreases computation time and outperforms other methods.
We consider a class of constrained optimization problems with a possibly nonconvex non-Lipschitz objective and a convex feasible set being the intersection of a polyhedron and a possibly degenerate ellipsoid. Such problems have a wide range of applications in data science, where the objective is used for inducing spars…
Two new methods improve block-sparse signal recovery from noisy data.
problem Recovering block-sparse signals with unknown partitions.
method LogLOP-l2/l1 and AdaLOP-l2/l1 methods using log-sum penalty and MCP.
result Our methods outperform existing techniques in estimation accuracy.
Deep Penalty Method solves high-dimensional optimal stopping problems using deep learning.
problem High-dimensional optimal stopping problems in American option pricing.
method Inspired by penalty method for PDEs, approximates penalized PDE with Deep BSDE framework.
result Error bound of DPM is .
Gradient-based Regularization Parameter Selection for Problems with Non-smooth Penalty Functionsstat.ML
In high-dimensional and/or non-parametric regression problems, regularization (or penalization) is used to control model complexity and induce desired structure. Each penalty has a weight parameter that indicates how strongly the structure corresponding to that penalty should be enforced. Typically the parameters are c…
In the multiple changepoint setting, various search methods have been proposed which involve optimising either a constrained or penalised cost function over possible numbers and locations of changepoints using dynamic programming. Such methods are typically computationally intensive. Recent work in the penalised optimi…
Paper develops algorithms for sparse linear regression with generalized elastic net penalty.
problem Sparse linear regression with robust penalty for high-dimensional data.
method Iterative Reweighted Framework based on ADMM and PMM with SNN.
result Efficient algorithms provide superior performance in both simulated and real data.
Accelerated gradient method tackles nonconvex penalties in sparse learning.
problem Optimizing nonconvex penalties in sparse statistical learning.
method Generalized Nesterov's accelerated gradient method with hyperparameter optimization.
result Convergence can be made considerably faster with optimal hyperparameters.
Paper solves convertible bond valuation using finite elements with penalty method.
problem Valuation of convertible bonds under penalty TF model.
method Solves TF system of equations using P1 and P2 finite elements with penalty method.
result Numerical solutions compare favorably with finite difference method.
New method solves complex bilevel optimization problems.
problem Challenging bilevel optimization problems in machine learning.
method Penalty-based bilevel gradient descent (PBGD) algorithm.
result PBGD algorithm converges for constrained bilevel problems.
A new method tackles nonconvex optimization with penalties and proximal terms.
problem Nonconvex optimization problems with equality and inequality constraints.
method Inexact proximal augmented Lagrangian method (P-ALM) with adaptive penalty and proximal parameters.
result Effective convergence properties and numerical superiority over traditional methods.
The use of machine-learning in neuroimaging offers new perspectives in early diagnosis and prognosis of brain diseases. Although such multivariate methods can capture complex relationships in the data, traditional approaches provide irregular (l2 penalty) or scattered (l1 penalty) predictive pattern with a very limited…
We demonstrate the existence of universal adversarial perturbations, which can fool a family of audio classification architectures, for both targeted and untargeted attack scenarios. We propose two methods for finding such perturbations. The first method is based on an iterative, greedy approach that is well-known in c…
New method reduces bias in sparse Bayesian learning.
problem High sparsity in statistical models leads to significant bias.
method Variable-coefficient penalty with hyperpriors.
result Reduces bias in sparse Bayesian learning.
In this paper, we present a novel penalty approach for the numerical solution of continuously controlled HJB equations and HJB obstacle problems. Our results include estimates of the penalisation error for a class of penalty terms, and we show that variations of Newton's method can be used to obtain globally convergent…
Two important goals of high-dimensional modeling are prediction and variable selection. In this article, we consider regularization with combined and concave penalties, and study the sampling properties of the global optimum of the suggested method in ultra-high dimensional settings. The -penalty provides th…
Paper proposes efficient algorithms for designing SLOPE penalty sequences.
problem Designing SLOPE penalty sequences is computationally expensive.
method Developed two efficient algorithms: PGD and CD for Gaussian and general data matrices respectively.
result Demonstrated improved mean squared error performance of SLOPE with designed penalties.
Safe screening improves generalized CGM's feature selection stability.
problem Improving feature selection stability in generalized CGM.
method Coupling safe screening with generalized CGM.
result Safe screening matches solution support at rate .
A new method reduces bias in adaptive Lasso estimates.
problem Bias in adaptive Lasso estimates.
method Proximal gradient approach to learn penalty coefficients as decision variables.
result Reduces bias in estimates and encourages arbitrary sparsity structure.
Paper estimates differences in multi-attribute Gaussian graphical models using non-convex penalties.
problem Estimating differences in multi-attribute Gaussian graphical models with similar structure.
method Penalized D-trace loss function with non-convex (log-sum and SCAD) penalties, proximal gradient descent methods.
result Theoretical analysis and numerical examples support consistency in support recovery and estimation.
Proposes spred for solving penalty with SGD.
problem Solving penalty in optimization problems.
method Reparametrization and SGD approach.
result Proves spred as an exact differentiable solver of .
Unified analysis of multi-attribute graph learning with non-convex penalties.
problem Graph inference from multi-attribute data.
method Penalized log-likelihood objective function with ADMM and local linear approximation.
result Local consistency in support recovery and precision matrix estimation for non-convex penalties.
In this paper we consider sparse approximation problems, that is, general minimization problems with the -"norm" of a vector being a part of constraints or objective function. In particular, we first study the first-order optimality conditions for these problems. We then propose penalty decomposition (PD) me…
Improved online penalty selection for time series models.
problem Efficiently selecting penalty parameters for lasso in time series models.
method Enhanced autoregressive model with online penalty selection.
result Significantly improved computational performance and forecast accuracy.
New single-loop algorithm tackles weakly convex constraints in stochastic optimization.
problem Optimization with weakly convex constraints in machine learning.
method Single-loop penalty-based stochastic algorithm using hinge-based penalty.
result Achieves state-of-the-art complexity for finding approximate KKT solutions.
Proposes using MLP for predicting optimal penalty in changepoint detection.
problem Predicting optimal penalty for changepoints in sequences.
method Uses a multilayer perceptron (MLP) with ReLU activation function to predict penalty.
result Improves accuracy and F1 score compared to existing models.
Paper tackles bilevel optimization problems using penalty methods.
problem Unconstrained and constrained bilevel optimization problems with nonsmooth lower levels.
method Introduces first-order penalty methods and and operation complexities.
result Establishes operation complexities for finding -KKT solutions.
Paper proposes a method to improve circular coordinate representation for detecting changes in high-dimensional datasets.
problem Detecting changes in high-dimensional datasets with preserved topological structures.
method Adapt circular coordinate framework using a generalized penalty function instead of an L2 penalty.
result Circular coordinates with generalized penalty can detect changes in high-dimensional datasets under different sampling schemes.
Nonconvex penalty methods for sparse modeling in linear regression have been a topic of fervent interest in recent years. Herein, we study a family of nonconvex penalty functions that we call the trimmed Lasso and that offers exact control over the desired level of sparsity of estimators. We analyze its structural prop…
Lipschitz continuity recently becomes popular in generative adversarial networks (GANs). It was observed that the Lipschitz regularized discriminator leads to improved training stability and sample quality. The mainstream implementations of Lipschitz continuity include gradient penalty and spectral normalization. In th…
This work addresses the issue of large covariance matrix estimation in high-dimensional statistical analysis. Recently, improved iterative algorithms with positive-definite guarantee have been developed. However, these algorithms cannot be directly extended to use a nonconvex penalty for sparsity inducing. Generally, a…
New method approximates sampling from smooth potential distributions using a vanishing penalty.
problem Sampling from smooth potential distributions on high-dimensional spaces.
method Penalized Langevin dynamics (PLD) with vanishing penalty.
result Established upper bound on Wasserstein-2 distance for PLD approximation.
SP-SPCA improves sparse PCA by adaptively adjusting variable penalties, enhancing interpretability and stability.
problem Poor interpretability and variable redundancy in PCA for high-dimensional data.
method Introduces a single equilibrium parameter to adaptively adjust variable penalties in the L2 regularization framework.
result Consistently outperforms standard sparse PCA methods in identifying sparse loading patterns and preserving cumulative variance.
In this paper we consider general rank minimization problems with rank appearing in either objective function or constraint. We first establish that a class of special rank minimization problems has closed-form solutions. Using this result, we then propose penalty decomposition methods for general rank minimization pro…
Piecewise Linear-Quadratic (PLQ) penalties are widely used to develop models in statistical inference, signal processing, and machine learning. Common examples of PLQ penalties include least squares, Huber, Vapnik, 1-norm, and their asymmetric generalizations. Properties of these estimators depend on the choice of pena…
A new method speeds up overlapping group lasso computations.
problem Time-consuming optimization of overlapping group lasso on large-scale problems.
method Non-overlapping statistical approximation to overlapping group lasso.
result The proposed penalty is statistically equivalent to overlapping group lasso.
Kernel-based mean-field games use MMD penalties for interaction and target costs.
problem Optimizing mean-field games with specific cost functions.
method Kernel structure, random Fourier U-statistics, neural network training.
result Sample-level convergence theorem and rate of convergence proved.
In this paper, we propose a framework for automatic classification of patients from multimodal genetic and brain imaging data by optimally combining them. Additive models with unadapted penalties (such as the classical group lasso penalty or -multiple kernel learning) treat all modalities in the same manner and ca…
A new penalty-free method optimizes portfolios without quantum annealing penalties.
problem Optimizing portfolios with quantum annealing penalties.
method Removing the penalty term and using a classical feasibility projector.
result Significant reduction in chain-break fractions and post-processed regret.
A new algorithm speeds up sparse-penalized quantile regression solving non-convex penalties.
problem Sparse-penalized quantile regression with non-convex penalties.
method Single-loop smoothing ADMM (SIAD) algorithm for faster convergence.
result SIAD method outperforms existing approaches in solving sparse-penalized quantile regression.
A new -means method HT -means uses penalty for sparsity.
problem Cluster center regularization for improved clustering performance.
method HT -means with penalty for sparsity.
result HT -means outperforms other regularized -means methods in simulations and real data.