Lap-VARD algorithm improves signal reconstruction in tomography.
problem Signal reconstruction in tomography with sparsity constraint.
method Laplacian prior and automatic relevance determination framework.
result Optimized Laplacian prior enhances balance between sparsity and accuracy.
The paper tackles sparse graph learning under Laplacian-related constraints, improving upon existing methods.
problem Learning a sparse undirected graph from multivariate data under Laplacian-related constraints.
method Modifications to penalized log-likelihood approaches to enforce total positivity and lasso/adaptive lasso penalties using ADMM.
result The proposed constrained adaptive lasso approach significantly outperforms existing Laplacian-based approaches.
Unified framework for multi-user bandits using Laplacian kernels.
problem Multi-user contextual bandits with graph-related users and non-linear rewards.
method Joint penalty combining graph smoothness and individual roughness in a unified RKHS.
result Unified multi-user RKHS and effective dimension for regret bounds.
Framework estimates precision matrices for heterogeneous populations.
problem Estimating precision matrices in populations with subpopulations.
method Laplacian shrinkage penalty, ADMM algorithm, hierarchical clustering.
result Consistent estimation of precision matrices in heterogeneous populations.
Unified analysis of multi-task functional linear regression with manifold and composite penalties.
problem Estimating slope functions from functional data with multi-task learning.
method Penalized splines with manifold constraint and composite quadratic penalty.
result Unified convergence upper bound and phase transition behaviors for estimators.
The ℓ1-norm fails to produce sparse solutions in Laplacian constrained graphical models, leading to a complete graph.
problem Learning a sparse graph under Laplacian constrained Gaussian graphical models.
method Introduced a nonconvex sparsity penalty and proposed a new estimator using a sequence of weighted ℓ1-norm penalized sub-problems. Developed a projected gradient descent algorithm with linear convergence rate. result The proposed estimator can recover the edges correctly with high probability and is effective on both synthetic and real-world data sets.
Graph poly-Laplacian method improves regression accuracy.
problem Regression with noisy labels on graphs.
method Graph poly-Laplacian regularization for non-parametric regression.
result Rate of convergence matches known results for smoothing splines.
Gradient descent training of neural networks leads to solutions close to natural cubic splines.
problem Understanding the implicit bias of gradient descent in neural networks.
method Analysis of gradient descent training for wide neural networks, focusing on the curvature penalty and initialization schemes.
result The solutions of gradient descent training are polyharmonic splines for certain initialization schemes.
New method learns under latent group sparsity using network dynamics.
problem Sparse learning under latent group structure without prior group information.
method Heat-flow-based local network dynamics incorporating Laplacian geometry.
result Automatic interpolation between lasso and group lasso penalties.
Graph Spectral Regularization makes neural network layers more interpretable.
problem Making neural network layers more interpretable without sacrificing performance.
method Using a graph Laplacian penalty to structure hidden layer activations.
result Encourages smooth activations within hidden layers, leading to better interpretability.
S2MAM improves semi-supervised learning by selecting relevant variables and updating similarity metrics.
problem Joint learning from labeled and unlabeled data with geometric structure.
method Bilevel optimization scheme for automatic variable selection and similarity matrix update.
result The proposed S2MAM achieves robust and interpretable predictions.
The paper analyzes graph Laplacian regularized estimators for learning latent variables from observations.
problem Learning latent variables from observations with a known topological structure.
method Graph Laplacian regularized estimator, penalized least squares with Laplacian penalty.
result Developed a non-asymptotic bound for estimation error, showing the advantage of graph Laplacian regularized estimators.
New algorithms improve robust PCA for vision tasks with heavy-tailed distributions.
problem Challenging non-convex, non-smooth, non-Lipschitz problems in robust PCA.
method Bilinear factor matrix norm minimization models with double nuclear and hybrid norms.
result Our methods yield more accurate solutions than original Schatten quasi-norm minimization.
New method improves tensor completion for weakly-dependent spatiotemporal data.
problem Improving tensor completion for weakly-dependent data on graphs.
method Introducing L1-norm and Graph Laplacian penalties for low-rank tensor decomposition and completion. result Improved performance in metro passenger flow prediction.
The paper explores nonconvex penalties for deep learning regularization.
problem Overfitting in deep learning neural networks.
method Examines and evaluates nonconvex penalties for DNN regularization.
result Nonconvex penalties, under certain conditions, can perform well in DNNs.
Study on ℓp-based Laplacian regularization for semi-supervised learning.
problem Improving semi-supervised learning performance by optimizing regularization parameters.
method Theoretical analysis of ℓp-based Laplacian regularization in a geometric random graph model. result Optimal choice of p=d+1 for maximum smoothness and sensitivity to underlying distribution. Paper proposes algorithms for robust 1-bit compressive sensing with nonconvex penalties.
problem Recovering sparse signals from one-bit measurements.
method Develops algorithms based on convex and nonconvex penalties, providing analytical solutions.
result Analytical solutions for several nonconvex penalties are found, making the recovery process faster and more efficient.
Gradient descent can optimize penalty parameters for non-smooth penalties.
problem Optimizing penalty parameters for non-smooth penalties in regression problems.
method Modified gradient descent algorithm for non-smooth penalty functions.
result Decreased generalization error with tuned penalty parameters.
Study examines insider trading with penalties, finding optimal penalties increase quickly for small orders.
problem Analyzing the impact of penalties on insider trading behavior and market efficiency.
method Formal economic model with penalty functions, existence and uniqueness theorems, and optimization.
result Optimal penalties increase quickly for small orders, signaling extreme events and incorporating information into prices.
Gradient penalty improves GAN performance by inducing a large-margin classifier.
problem Improving GAN performance and addressing vanishing gradients.
method A unifying framework of expected margin maximization, showing gradient penalties induce large-margin classifiers.
result Gradient penalties reduce vanishing gradients and produce better generated outputs.
The paper studies robust risk measures with linear penalties under uncertain distributions.
problem Risk measurement under distributional uncertainty.
method Robust distortion risk measures with linear penalty function under distributional constraints.
result Explicit characterization of optimal quantile distribution and value function.
The use of machine-learning in neuroimaging offers new perspectives in early diagnosis and prognosis of brain diseases. Although such multivariate methods can capture complex relationships in the data, traditional approaches provide irregular (l2 penalty) or scattered (l1 penalty) predictive pattern with a very limited…
Paper introduces a new SVR model using a combined reward and penalty loss function.
problem Regression problem, particularly handling data points outside and inside ε-tube.
method Combined reward cum penalty loss function to penalize and reward data points.
result Experimental results support the model's properties and effectiveness.
New approach avoids excess empirical risk in domain generalization.
problem Learning models that generalize to unseen distributions from diverse data sets.
method Minimizes penalty under constraint of optimal empirical risk, leveraging rate-distortion theory.
result Significant improvements in domain generalization performance across multiple methods.
We study the problem of estimating high-dimensional regression models regularized by a structured sparsity-inducing penalty that encodes prior structural information on either the input or output variables. We consider two widely adopted types of penalties of this kind as motivating examples: (1) the general overlappin…
New sparse penalty improves biclustering for gene expression data.
problem Identifying significant clusters in gene expression data.
method Prenet penalty applied to SSVD for biclustering.
result Mixed Prenet penalty effectively clusters non-overlapped data.
New method reduces bias in sparse Bayesian learning.
problem High sparsity in statistical models leads to significant bias.
method Variable-coefficient ℓ1 penalty with hyperpriors. result Reduces bias in sparse Bayesian learning.
Curvature penalties improve interpretability of KANs without sacrificing accuracy.
problem Pathologically high-curvature oscillations in KANs activations make them hard to interpret.
method Derived a curvature penalty and proved an upper bound on model curvature.
result KANs with curvature penalties achieve substantially smoother activations while maintaining accuracy.
PPO-B improves sampling efficiency by using a logarithmic barrier method.
problem Low sampling efficiency in PPO due to exterior penalty method.
method Introducing a surrogate objective with interior penalty method.
result PPO-B outperforms PPO in terms of sampling efficiency.
New nonconvex penalty smooths at origin for deep learning.
problem Improving variable selection and bias in high-dimensional statistical learning.
method Developed a new nonconvex penalty function smooth at origin.
result Asymptotic bias of new penalty function vanishes exponentially fast.
New method estimates large covariance matrices using nonconvex penalties.
problem Estimating large covariance matrices in high-dimensional data.
method Developed a first-order algorithm using generalized nonconvex penalties.
result Positive-definite covariance estimators using nonconvex penalties.
This study proves local stability of SGP μ-WGAN and shows penalizing data or sample manifold is key.
problem Stabilizing and regularizing WGAN with gradient penalty.
method Proves local stability of SGP μ-WGAN using measure valued differentiation.
result Penalizing data or sample manifold is key to regularizing WGAN.
We study the problem of learning high dimensional regression models regularized by a structured-sparsity-inducing penalty that encodes prior structural information on either input or output sides. We consider two widely adopted types of such penalties as our motivating examples: 1) overlapping group lasso penalty, base…
Multi-group learners suffer a penalty in transductive learning.
problem The penalty on multi-group learners in transductive learning.
method Analyzing the relationship between the number of groups and the error rate.
result The penalty can increase linearly with the number of groups, up to the square-root of the sample size.
The study bounds how smoothly embedded submanifolds can move and ensures gradient conditions for penalty functions.
problem Bounding the movement of smoothly embedded submanifolds and ensuring gradient conditions for penalty functions.
method Explicit lower bound and gradient condition for penalty functions in the space of embeddings.
result Explicit lower bound and gradient condition for penalty functions in the space of embeddings.
Global minima found for multidimensional scaling with penalties.
problem Finding global minima in multidimensional scaling.
method Combining stress loss function with a quadratic penalty term to find minimizers.
result Trajectory of minimizers leads to global minima.
New method improves signal reconstruction with nonconvex penalties and parameter control.
problem Reconstructing sparse signals with nonconvex penalties and nonconvexity control.
method Introduces nonconvex penalties (SCAD, MCP) with nonconvexity parameters and controls them to guide AMP trajectory.
result Achieves perfect reconstruction for relatively dense signals with small nonconvexity parameters.
Unified framework for faster low-rank matrix estimation with nonconvex penalties.
problem Efficient estimation of low-rank matrices with nonconvex penalties.
method Unified framework with nonconvex penalties for matrix estimation.
result The proposed estimator achieves faster rates and oracle property.
Improved penalty-based methods for bilevel optimization with reduced complexity.
problem Suboptimal complexity in solving bilevel optimization problems with large penalty terms.
method Novel penalty reformulation that decouples upper and lower-level variables, enabling larger step sizes and reduced iteration complexity.
result PBGD-Free algorithm that avoids inner loops for coupled constraint BLO problems, with reduced iteration complexity.
Proposes an alternative invariance penalty to address domain generalization issues.
problem Addressing domain generalization problems by finding invariant representations.
method Revisits the Gramian matrix of the data representation to propose an alternative invariance penalty.
result The proposed approach guarantees recovery of an invariant representation under mild conditions.
Paper proposes efficient algorithms for designing SLOPE penalty sequences.
problem Designing SLOPE penalty sequences is computationally expensive.
method Developed two efficient algorithms: PGD and CD for Gaussian and general data matrices respectively.
result Demonstrated improved mean squared error performance of SLOPE with designed penalties.
This paper proposes a new method for GLM estimation using distance penalties to handle constraints.
problem Handling constraints in generalized linear models (GLM) is complicated.
method The approach uses distance penalties to optimize the log-likelihood, avoiding shrinkage.
result Distance penalties provide a flexible and non-shrinking alternative to traditional penalties.
Insider trading is reduced when penalized, affecting expected penalties in a non-monotone way.
problem Reducing insider trading behavior when insiders face legal penalties.
method Characterized via a backward stochastic differential equation (BSDE) with a non-linear operator.
result The insider's expected penalties are non-monotone in the fee structure and determined by relative entropy.
Survey on minimal penalty algorithms and slope heuristics.
problem Choosing optimal multiplicative constants from data.
method Minimal penalty and slope heuristics approach.
result Slope heuristics performs almost as well as residual-based estimators.
Sparse reconstruction approaches using the re-weighted l1-penalty have been shown, both empirically and theoretically, to provide a significant improvement in recovering sparse signals in comparison to the l1-relaxation. However, numerical optimization of such penalties involves solving problems with l1-norms in the ob…
Efficient ADMM algorithm solves nonconvex SVMs with various penalties.
problem Solving nonconvex penalized SVMs due to nondifferentiability, nonsmoothness, and nonconvexity.
method ADMM-based algorithm for a wide range of nonconvex penalties.
result The proposed algorithm outperforms other methods on benchmark datasets.
New method recovers signals from saturated data using linear loss and nonconvex penalties.
problem Signal recovery from saturated measurements with sign information loss.
method Linear loss and nonconvex penalties (e.g., minimax concave penalty, sorted ℓ1 norm).
result Estimation error is bounded and recovery performance improved.
The paper improves risk bounds for maximum likelihood estimation with arbitrary penalties.
problem Improving risk bounds for maximum likelihood estimation with arbitrary penalties.
method Developed a more general inequality for arbitrary penalties, leading to exact risk bounds of order 1/n.
result Derived exact risk bounds of order 1/n for iid parametric models, improving on previous bounds.