arXiv research

This paper reformulates $F_β$ for better model performance and interpretation.

Generative diffusion models forecast implied vol surfaces without arbitrage issues.

A method of simultaneously optimizing both the structure of neural networks and the connection weights in a single training loop can reduce the enormous computational cost of neural architecture search. We focus on the probabilistic model-based dynamic neural network structure optimization that considers the probabilit…

Deep weight factorization improves neural network training through smooth optimization of sparse penalties.

Weight decay stabilizes training dynamics by slowing progressive sharpening.

In this paper we will provide a representation of the penalty term of general dynamic concave utilities (hence of dynamic convex risk measures) by applying the theory of g-expectations.

ETM identifies field-specific keywords in text classification.

This paper gives an overview of the theory of dynamic convex risk measures for random variables in discrete time setting. We summarize robust representation results of conditional convex risk measures, and we characterize various time consistency properties of dynamic risk measures in terms of acceptance sets, penalty …

New method approximates sampling from smooth potential distributions using a vanishing penalty.

In the classic sparsity-driven problems, the fundamental L-1 penalty method has been shown to have good performance in reconstructing signals for a wide range of problems. However this performance relies on a good choice of penalty weight which is often found from empirical experiments. We propose an algorithm called t…

It is well known that the out-of-sample performance of Markowitz's mean-variance portfolio criterion can be negatively affected by estimation errors in the mean and covariance. In this paper we address the problem by regularizing the mean-variance objective function with a weighted elastic net penalty. We show that the…

Algorithm minimizes loss and constraint violations in online convex optimization with smooth penalties.

HALO learns to prune neural networks by adaptively shrinking weights.

Efficient cross-validation for multi-penalty ridge regression.

Paper introduces stability in model averaging and proposes a L2-penalty method.

Proposes using MLP for predicting optimal penalty in changepoint detection.

New method detects changepoints in PDEs using optimized neural networks.

New framework solves dynamic bilevel optimization problems in reinforcement learning.

Elastic weight consolidation (EWC, Kirkpatrick et al, 2017) is a novel algorithm designed to safeguard against catastrophic forgetting in neural networks. EWC can be seen as an approximation to Laplace propagation (Eskin et al, 2004), and this view is consistent with the motivation given by Kirkpatrick et al (2017). In…

While most classical approaches to Granger causality detection assume linear dynamics, many interactions in real-world applications, like neuroscience and genomics, are inherently nonlinear. In these cases, using linear models may lead to inconsistent estimation of Granger causal interactions. We propose a class of non…

We analyze dropout in deep networks with rectified linear units and the quadratic loss. Our results expose surprising differences between the behavior of dropout and more traditional regularizers like weight decay. For example, on some simple data sets dropout training produces negative weights even though the output i…

Study proposes adaptive RL for dynamic portfolio optimization.

Two new regularization methods improve neural network performance and complexity control.

Sparse reconstruction approaches using the re-weighted l1-penalty have been shown, both empirically and theoretically, to provide a significant improvement in recovering sparse signals in comparison to the l1-relaxation. However, numerical optimization of such penalties involves solving problems with l1-norms in the ob…

In high-dimensional and/or non-parametric regression problems, regularization (or penalization) is used to control model complexity and induce desired structure. Each penalty has a weight parameter that indicates how strongly the structure corresponding to that penalty should be enforced. Typically the parameters are c…

As surrogate functions of $L_0$-norm, many nonconvex penalty functions have been proposed to enhance the sparse vector recovery. It is easy to extend these nonconvex penalty functions on singular values of a matrix to enhance low-rank matrix recovery. However, different from convex optimization, solving the nonconvex l…

The use of machine-learning in neuroimaging offers new perspectives in early diagnosis and prognosis of brain diseases. Although such multivariate methods can capture complex relationships in the data, traditional approaches provide irregular (l2 penalty) or scattered (l1 penalty) predictive pattern with a very limited…

We stabilize the activations of Recurrent Neural Networks (RNNs) by penalizing the squared distance between successive hidden states' norms. This penalty term is an effective regularizer for RNNs including LSTMs and IRNNs, improving performance on character-level language modeling and phoneme recognition, and outperfor…

Power-law portfolios improve diversification by scaling weights sub-linearly.

Paper proposes a method to estimate multiple dynamic quantiles jointly.

In this paper, we consider the joint task of simultaneously optimizing (i) the weights of a deep neural network, (ii) the number of neurons for each hidden layer, and (iii) the subset of active input features (i.e., feature selection). While these problems are generally dealt with separately, we present a simple regula…

Proposes a new ridge estimator for smooth covariates with adaptive centering.

New method reduces bias in sparse Bayesian learning.

Regularization improves stability and consistency of sparse autoencoders.

Adaptive dropout and regularization are shown to be dual in linear networks.

New method learns nonlinear projections for reduced-order modeling of complex dynamical systems.

Paper reformulates UOT as non-negative penalized linear regression for efficient algorithms.

Fisher et al. extend multi-VAR for better modeling of heterogeneous time series.

Proposes spred for solving $L_1$ penalty with SGD.

An optimal feedback controller for a given Markov decision process (MDP) can in principle be synthesized by value or policy iteration. However, if the system dynamics and the reward function are unknown, a learning agent must discover an optimal controller via direct interaction with the environment. Such interactive d…

Heavy-tailed regularization improves deep neural network performance.

We study channel number reduction in combination with weight binarization (1-bit weight precision) to trim a convolutional neural network for a keyword spotting (classification) task. We adopt a group-wise splitting method based on the group Lasso penalty to achieve over 50% channel sparsity while maintaining the netwo…

We consider high-dimensional binary classification by sparse logistic regression. We propose a model/feature selection procedure based on penalized maximum likelihood with a complexity penalty on the model size and derive the non-asymptotic bounds for the resulting misclassification excess risk. The bounds can be reduc…

This paper tackles the problem of selecting among several linear estimators in non-parametric regression; this includes model selection for linear regression, the choice of a regularization parameter in kernel ridge regression, spline smoothing or locally weighted regression, and the choice of a kernel in multiple kern…

Estimates and infers multi-stage stationary treatment policies with variable selection.

WrapNet optimizes inference for low-resolution neural networks by using 8-bit additions.

Neural networks have been widely used as predictive models to fit data distribution, and they could be implemented through learning a collection of samples. In many applications, however, the given dataset may contain noisy samples or outliers which may result in a poor learner model in terms of generalization. This pa…

Proposes fwelnet to improve prediction using feature information.