A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Regularizers change the geometric properties of loss functions in neural networks.
Study uses property elicitation to understand how fairness regularizers affect optimal decisions.
Paper improves learning rates for GSC loss functions using iterated Tikhonov regularization.
Evolutionary methods improve neural network loss functions, reducing overfitting.
Regularized empirical risk minimization including support vector machines plays an important role in machine learning theory. In this paper regularized pairwise learning (RPL) methods based on kernels will be investigated. One example is regularized minimization of the error entropy loss which has recently attracted qu…
New aggregation strategy handles unbounded losses with regret bounds.
This study explores star-shaped regularizers learned from critic-based losses.
New method prevents neural network breakdown by combining trimmed loss and variation regularization.
Deep networks can memorize random labels; symmetric loss improves this.
A method to identify important features without solving the full problem.
Enhances trading metrics with financially grounded loss functions.
Recent years have seen adversarial losses been applied to many fields. Their applications extend beyond the originally proposed generative modeling to conditional generative and discriminative settings. While prior work has proposed various output activation functions and regularization approaches, some open questions …
We introduce a novel algorithm for solving learning problems where both the loss function and the regularizer are non-convex but belong to the class of difference of convex (DC) functions. Our contribution is a new general purpose proximal Newton algorithm that is able to deal with such a situation. The algorithm consi…
We consider the problem of supervised learning with convex loss functions and propose a new form of iterative regularization based on the subgradient method. Unlike other regularization approaches, in iterative regularization no constraint or penalization is considered, and generalization is achieved by (early) stoppin…
Optimal sampling bounds for various classification losses under different regularization terms.
In this paper, we address the problem of embedded feature selection for ranking on top of the list problems. We pose this problem as a regularized empirical risk minimization with -norm push loss function () and sparsity inducing regularizers. We leverage the issues related to this challenging optimization…
Paper introduces a new topological loss for better convergence.
Multi-output is essential in machine learning that it might suffer from nonconforming residual distributions, i.e., the multi-output residual distributions are not conforming to the expected distribution. In this paper, we propose "Wrapped Loss Function" to wrap the original loss function to alleviate the problem. This…
We consider the problem of learning a forest of nonlinear decision rules with general loss functions. The standard methods employ boosted decision trees such as Adaboost for exponential loss and Friedman's gradient boosting for general loss. In contrast to these traditional boosting algorithms that treat a tree learner…
New algorithm ATENT improves adversarial robustness in neural networks.
The simplicity of gradient descent (GD) made it the default method for training ever-deeper and complex neural networks. Both loss functions and architectures are often explicitly tuned to be amenable to this basic local optimization. In the context of weakly-supervised CNN segmentation, we demonstrate a well-motivated…
Gradient descent on ReLU networks with square loss implicitly favors balanced weights.
Paper connects risk consistency to L_p consistency for broader loss functions.
The use of convex regularizers allows for easy optimization, though they often produce biased estimation and inferior prediction performance. Recently, nonconvex regularizers have attracted a lot of attention and outperformed convex ones. However, the resultant optimization problem is much harder. In this paper, for a …
SGD with large learning rates can achieve better test accuracy than expected.
We study the emergence of sparse representations in neural networks. We show that in unsupervised models with regularization, the emergence of sparsity is the result of the input data samples being distributed along highly non-linear or discontinuous manifold. We also derive a similar argument for discriminatively trai…
The goal of regression and classification methods in supervised learning is to minimize the empirical risk, that is, the expectation of some loss function quantifying the prediction error under the empirical distribution. When facing scarce training data, overfitting is typically mitigated by adding regularization term…
In this work, we study data preconditioning, a well-known and long-existing technique, for boosting the convergence of first-order methods for regularized loss minimization. It is well understood that the condition number of the problem, i.e., the ratio of the Lipschitz constant to the strong convexity modulus, has a h…
The success of deep learning has led to a rising interest in the generalization property of the stochastic gradient descent (SGD) method, and stability is one popular approach to study it. Existing works based on stability have studied nonconvex loss functions, but only considered the generalization error of the SGD in…
New approach uses hinge loss for iterative regularization in classification.
This paper analyzes M-estimators under infinite-variance noise in high dimensions.
Paper tackles noisy labels in deep learning networks.
Proposes a generalized XGBoost method for nonconvex loss functions.
This paper is concerned with the factorization form of the rank regularized loss minimization problem. To cater for the scenario in which only a coarse estimation is available for the rank of the true matrix, an -norm regularized term is added to the factored loss function to reduce the rank adaptively; and…
PANDA augments data to regularize GLM estimation and inference.
This paper studies Fenchel-Young losses, a generic way to construct convex loss functions from a regularization function. We analyze their properties in depth, showing that they unify many well-known loss functions and allow to create useful new ones easily. Fenchel-Young losses constructed from a generalized entropy, …
This study connects Jacobian regularization to adversarial robustness and improves generalization.
Regularized empirical risk minimization with constrained labels (in contrast to fixed labels) is a remarkably general abstraction of learning. For common loss and regularization functions, this optimization problem assumes the form of a mixed integer program (MIP) whose objective function is non-convex. In this form, t…
Optimizing full likelihoods adapts loss scales and shapes for robust modeling.
SGD converges globally to logistic loss minima for two-layer nets.
We propose and analyze a regularization approach for structured prediction problems. We characterize a large class of loss functions that allows to naturally embed structured outputs in a linear space. We exploit this fact to design learning algorithms using a surrogate loss approach and regularization techniques. We p…
Paper improves stability analysis of SGD for various loss functions and data distributions.
The paper proposes effective margin regularization to improve adversarial robustness in deep neural networks.
Regularization and normalization have become indispensable components in training deep neural networks, resulting in faster training and improved generalization performance. We propose the projected error function regularization loss (PER) that encourages activations to follow the standard normal distribution. PER rand…
Proposes a new regularization technique for neural networks using elliptic operators.
FairNN learns fair representations and decisions by optimizing a multi-objective loss function.
Existing graph convolutional networks focus on the neighborhood aggregation scheme. When applied to semi-supervised learning, they often suffer from the overfitting problem as the networks are trained with the cross-entropy loss on a small potion of labeled data. In this paper, we propose an unsupervised manifold smoot…
We propose an AdaPtive Noise Augmentation (PANDA) technique to regularize the estimation and construction of undirected graphical models. PANDA iteratively optimizes the objective function given the noise augmented data until convergence to achieve regularization on model parameters. The augmented noises can be designe…