Regularization methods are often employed in deep learning neural networks (DNNs) to prevent overfitting. For penalty based DNN regularization methods, convex penalties are typically considered because of their optimization guarantees. Recent theoretical work have shown that nonconvex penalties that satisfy certain reg…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “performance penalty”
We stabilize the activations of Recurrent Neural Networks (RNNs) by penalizing the squared distance between successive hidden states' norms. This penalty term is an effective regularizer for RNNs including LSTMs and IRNNs, improving performance on character-level language modeling and phoneme recognition, and outperfor…
Study evaluates various regularization methods for electricity price forecasting.
problem Improving accuracy of electricity price predictions.
method Applied ten different penalty functions to two model structures in two electricity markets.
result LQ and elastic net consistently produce more accurate forecasts than other regularization types.
One-bit measurements widely exist in the real world, and they can be used to recover sparse signals. This task is known as the problem of learning halfspaces in learning theory and one-bit compressive sensing (1bit-CS) in signal processing. In this paper, we propose novel algorithms based on both convex and nonconvex s…
A popular heuristic for improved performance in Generative adversarial networks (GANs) is to use some form of gradient penalty on the discriminator. This gradient penalty was originally motivated by a Wasserstein distance formulation. However, the use of gradient penalty in other GAN formulations is not well motivated.…
Wasserstein GAN(WGAN) is a model that minimizes the Wasserstein distance between a data distribution and sample distribution. Recent studies have proposed stabilizing the training process for the WGAN and implementing the Lipschitz constraint. In this study, we prove the local stability of optimizing the simple gradien…
ZNMF improves facial recognition performance using data-dependent penalties.
problem Facial recognition performance in the Cambridge ORL database.
method ZNMF uses data-dependent auxiliary constraints to modify NMF.
result ZNMF outperforms other constrained NMF algorithms in facial recognition.
New nonconvex penalty smooths at origin for deep learning.
problem Improving variable selection and bias in high-dimensional statistical learning.
method Developed a new nonconvex penalty function smooth at origin.
result Asymptotic bias of new penalty function vanishes exponentially fast.
New approach avoids excess empirical risk in domain generalization.
problem Learning models that generalize to unseen distributions from diverse data sets.
method Minimizes penalty under constraint of optimal empirical risk, leveraging rate-distortion theory.
result Significant improvements in domain generalization performance across multiple methods.
Efficient cross-validation for multi-penalty ridge regression.
problem Optimizing penalties in cross-validation for multi-penalty ridge regression.
method Developed a computationally efficient formula for the multi-penalty, sample-weighted hat-matrix.
result Significant speed-up in cross-validation for ridge regression models.
Optimal subset selection for hypothesis testing with penalties.
problem Optimal subset selection of information sources for hypothesis testing with misclassification penalties.
method Proposes a misclassification penalty framework and studies two variants of subset selection problems under centralized Bayesian learning.
result Proves the submodularity of the objective and constraints of the subset selection problems and establishes performance guarantees for greedy algorithms.
Proposes a new SPVM model for RVM with more flexible priors.
problem Improper priors on multiple penalty parameters in RVM lead to improper posteriors.
method Introduces a single penalty approach (SPRVM) and a semi-Bayesian fitting method.
result SPRVM allows for more flexible priors and has proven conditions for posterior propriety.
Paper develops algorithms for sparse linear regression with generalized elastic net penalty.
problem Sparse linear regression with robust penalty for high-dimensional data.
method Iterative Reweighted Framework based on ADMM and PMM with SNN.
result Efficient algorithms provide superior performance in both simulated and real data.
Recently, there has been focus on penalized log-likelihood covariance estimation for sparse inverse covariance (precision) matrices. The penalty is responsible for inducing sparsity, and a very common choice is the convex norm. However, the best estimator performance is not always achieved with this penalty. The …
Paper proposes efficient algorithms for designing SLOPE penalty sequences.
problem Designing SLOPE penalty sequences is computationally expensive.
method Developed two efficient algorithms: PGD and CD for Gaussian and general data matrices respectively.
result Demonstrated improved mean squared error performance of SLOPE with designed penalties.
Study develops a method to select penalty parameters for sparse neural networks without cross-validation.
problem Selecting optimal penalty parameters for sparse neural networks without cross-validation.
method Established theoretical foundation to bound the infinite norm of the gradient of the loss function at zero.
result Proposed method effectively selects penalty parameters for sparse neural networks.
Support vector machines (SVMs) with sparsity-inducing nonconvex penalties have received considerable attentions for the characteristics of automatic classification and variable selection. However, it is quite challenging to solve the nonconvex penalized SVMs due to their nondifferentiability, nonsmoothness and nonconve…
Improved online penalty selection for time series models.
problem Efficiently selecting penalty parameters for lasso in time series models.
method Enhanced autoregressive model with online penalty selection.
result Significantly improved computational performance and forecast accuracy.
Proximal policy optimization(PPO) has been proposed as a first-order optimization method for reinforcement learning. We should notice that an exterior penalty method is used in it. Often, the minimizers of the exterior penalty functions approach feasibility only in the limits as the penalty parameter grows increasingly…
In the classic sparsity-driven problems, the fundamental L-1 penalty method has been shown to have good performance in reconstructing signals for a wide range of problems. However this performance relies on a good choice of penalty weight which is often found from empirical experiments. We propose an algorithm called t…
Proposes using MLP for predicting optimal penalty in changepoint detection.
problem Predicting optimal penalty for changepoints in sequences.
method Uses a multilayer perceptron (MLP) with ReLU activation function to predict penalty.
result Improves accuracy and F1 score compared to existing models.
Two new regularization methods improve neural network performance and complexity control.
problem Improving neural network performance and complexity control with correlated or high-dimensional features.
method Two regularization strategies: covariance-aware ridge and covariance-aware lasso.
result Improves predictive performance and complexity control over standard penalties.
In a Gaussian graphical model, the conditional independence between two variables are characterized by the corresponding zero entries in the inverse covariance matrix. Maximum likelihood method using the smoothly clipped absolute deviation (SCAD) penalty (Fan and Li, 2001) and the adaptive LASSO penalty (Zou, 2006) hav…
This work addresses the issue of large covariance matrix estimation in high-dimensional statistical analysis. Recently, improved iterative algorithms with positive-definite guarantee have been developed. However, these algorithms cannot be directly extended to use a nonconvex penalty for sparsity inducing. Generally, a…
A new framework selects information sources to test hypotheses robustly, even with misclassifications.
problem Robust hypothesis testing with misclassification penalties.
method Introduces a misclassification penalty framework and an efficient greedy algorithm.
result Proposes a submodular surrogate metric for better selection.
Systematic trading strategies are rule-based procedures which choose portfolios and allocate assets. In order to attain certain desired return profiles, quantitative strategists must determine a large array of trading parameters. Backtesting, the attempt to identify the appropriate parameters using historical data avai…
Estimation in generalized linear models (GLM) is complicated by the presence of constraints. One can handle constraints by maximizing a penalized log-likelihood. Penalties such as the lasso are effective in high dimensions, but often lead to unwanted shrinkage. This paper explores instead penalizing the squared distanc…
HALO learns to prune neural networks by adaptively shrinking weights.
problem Sparsity and model size in deep neural networks.
method Bayesian hierarchical models and trainable parameters for adaptive sparsification.
result HALO learns to create highly sparse networks with significant performance gains.
Study ablated data augmentation techniques and their mathematical equivalence to penalties.
problem Lack of mathematical understanding of differences between ablated data augmentation techniques.
method Formal model of mean ablated data augmentation and inverted dropout for linear regression; empirical validation for deep networks.
result Ablated data augmentation and inverted dropout are mathematically equivalent to penalties in optimization.
Proposes spred for solving penalty with SGD.
problem Solving penalty in optimization problems.
method Reparametrization and SGD approach.
result Proves spred as an exact differentiable solver of .
In this paper, we propose a framework for automatic classification of patients from multimodal genetic and brain imaging data by optimally combining them. Additive models with unadapted penalties (such as the classical group lasso penalty or -multiple kernel learning) treat all modalities in the same manner and ca…
Sign information is the key to overcoming the inevitable saturation error in compressive sensing systems, which causes information loss and results in bias. For sparse signal recovery from saturation, we propose to use a linear loss to improve the effectiveness from existing methods that utilize hard constraints/hinge …
We consider supervised learning problems where the features are embedded in a graph, such as gene expressions in a gene network. In this context, it is of much interest to automatically select a subgraph with few connected components; by exploiting prior knowledge, one can indeed improve the prediction performance or o…
Birg{é} and Massart proposed in 2001 the slope heuristics as a way to choose optimally from data an unknown multiplicative constant in front of a penalty. It is built upon the notion of minimal penalty, and it has been generalized since to some "minimal-penalty algorithms". This paper reviews the theoretical results ob…
New method balances performance and cost in identifying best arm.
problem Identifying the best arm in multi-armed bandit models.
method Minimizes a risk functional that balances performance and cost.
result Proposes DBCARE algorithm that matches theoretical lower bounds.
Unified analysis of multi-attribute graph learning with non-convex penalties.
problem Graph inference from multi-attribute data.
method Penalized log-likelihood objective function with ADMM and local linear approximation.
result Local consistency in support recovery and precision matrix estimation for non-convex penalties.
This paper reformulates for better model performance and interpretation.
problem Optimizing model performance and interpretation using metric.
method Reformulate metric to facilitate statistical distributions and dynamic penalty weights.
result Better and interpretable results with a 14% boost in score for IMDB data.
Paper proposes a method to improve circular coordinate representation for detecting changes in high-dimensional datasets.
problem Detecting changes in high-dimensional datasets with preserved topological structures.
method Adapt circular coordinate framework using a generalized penalty function instead of an L2 penalty.
result Circular coordinates with generalized penalty can detect changes in high-dimensional datasets under different sampling schemes.
Feature subset selection arises in many high-dimensional applications of statistics, such as compressed sensing and genomics. The penalty is ideal for this task, the caveat being it requires the NP-hard combinatorial evaluation of all models. A recent area of considerable interest is to develop efficient algor…
This paper considers a multiple regression model and compares, under full model hypothesis, analytically as well as by simulation, the performance characteristics of some popular penalty estimators such as ridge regression, LASSO, adaptive LASSO, SCAD, and elastic net versus Least Squares Estimator, restricted estimato…
Accelerated gradient method tackles nonconvex penalties in sparse learning.
problem Optimizing nonconvex penalties in sparse statistical learning.
method Generalized Nesterov's accelerated gradient method with hyperparameter optimization.
result Convergence can be made considerably faster with optimal hyperparameters.
Paper optimizes DC pension fund management with VaR and relative performance constraints.
problem Optimizing DC pension fund performance under VaR and relative performance constraints.
method Introduced an auxiliary process to transform the problem into a self-financing problem, combined linearization, Lagrange dual, martingale, and concavification methods.
result Explicit investment strategies obtained for certain penalty and reward functions.
We introduce an iterative optimization scheme for convex objectives consisting of a linear loss and a non-separable penalty, based on the expectation-consistent approximation and the vector approximate message-passing (VAMP) algorithm. Specifically, the penalties we approach are convex on a linear transformation of the…
Improved quantile estimation model for VaR.
problem Improving quantile estimation under distribution estimation.
method Develops a compensatory model with a penalty term to control convergence error.
result Significant improvement in VaR performance.
In this paper, we study the Nystr{ö}m type subsampling for large scale kernel methods to reduce the computational complexities of big data. We discuss the multi-penalty regularization scheme based on Nystr{ö}m type subsampling which is motivated from well-studied manifold regularization schemes. We develop a theoretica…
High-dimensional data pose challenges in statistical learning and modeling. Sometimes the predictors can be naturally grouped where pursuing the between-group sparsity is desired. Collinearity may occur in real-world high-dimensional applications where the popular technique suffers from both selection inconsisten…
We consider a regularized least squares problem, with regularization by structured sparsity-inducing norms, which extend the usual and the group lasso penalty, by allowing the subsets to overlap. Such regularizations lead to nonsmooth problems that are difficult to optimize, and we propose in this paper a suit…
We tackle the problem of penalty selection of regularization on the basis of the minimum description length (MDL) principle. In particular, we consider that the design space of the penalty function is high-dimensional. In this situation, the luckiness-normalized-maximum-likelihood(LNML)-minimization approach is favorab…