A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Global implicit function theorem for Fréchet spaces, solving derivative loss problems.
A hybrid loss framework improves time series forecasting by balancing global and component errors.
SGD converges globally to logistic loss minima for two-layer nets.
This paper examines how different loss functions affect neural network features and performance.
We study the error landscape of deep linear and nonlinear neural networks with the squared error loss. Minimizing the loss of a deep linear neural network is a nonconvex problem, and despite recent progress, our understanding of this loss surface is still incomplete. For deep linear networks, we present necessary and s…
Adaptive rerouting reshapes impacts of maritime chokepoint disruptions
BCD algorithm finds global minima in neural networks.
This work justifies neural collapse under MSE loss and analyzes the optimization landscape.
Hill-ADAM optimizes loss landscapes by exploring state space deterministically.
The past decade has witnessed a successful application of deep learning to solving many challenging problems in machine learning and artificial intelligence. However, the loss functions of deep neural networks (especially nonlinear networks) are still far from being well understood from a theoretical aspect. In this pa…
This paper shows that every sublevel set of the loss function of a class of deep over-parameterized neural nets with piecewise linear activation functions is connected and unbounded. This implies that the loss has no bad local valleys and all of its global minima are connected within a unique and potentially very large…
Due to the success of deep learning to solving a variety of challenging machine learning tasks, there is a rising interest in understanding loss functions for training neural networks from a theoretical aspect. Particularly, the properties of critical points and the landscape around them are of importance to determine …
New methods connect low-loss points on neural network surfaces.
We consider deep linear networks with arbitrary convex differentiable loss. We provide a short and elementary proof of the fact that all local minima are global minima if the hidden layers are either 1) at least as wide as the input layer, or 2) at least as wide as the output layer. This result is the strongest possibl…
Deep ReLU networks with extra parameters have mostly good loss landscapes.
Poisson learning doesn't solve graph semi-supervised learning issues.
While optimizing convex objective (loss) functions has been a powerhouse for machine learning for at least two decades, non-convex loss functions have attracted fast growing interests recently, due to many desirable properties such as superior robustness and classification accuracy, compared with their convex counterpa…
Paper develops methods for non-quadratic loss low-rank matrix recovery.
It has been argued in the past that high-dimensional neural networks do not exhibit local minima capable of trapping an optimisation algorithm. However, the relationship between loss surface modality and the neural architecture parameters, such as the number of hidden neurons per layer and the number of hidden layers, …
Supervised training of neural networks for classification is typically performed with a global loss function. The loss function provides a gradient for the output layer, and this gradient is back-propagated to hidden layers to dictate an update direction for the weights. An alternative approach is to train the network …
Global minima found for multidimensional scaling with penalties.
For one-hidden-layer ReLU networks, we prove that all differentiable local minima are global inside differentiable regions. We give the locations and losses of differentiable local minima, and show that these local minima can be isolated points or continuous hyperplanes, depending on an interplay between data, activati…
Optimizing proper loss yields calibrated models under specific conditions.
We explore some mathematical features of the loss landscape of overparameterized neural networks. A priori one might imagine that the loss function looks like a typical function from to - in particular, nonconvex, with discrete global minima. In this paper, we prove that in at least one impo…
GCML preserves geometric structure in manifold clustering for diverse data types.
Many modern learning tasks involve fitting nonlinear models to data which are trained in an overparameterized regime where the parameters of the model exceed the size of the training dataset. Due to this overparameterization, the training loss may have infinitely many global minima and it is critical to understand the …
StochasticRank optimizes ranking metrics efficiently and guarantees global convergence.
We study cross-country GDP losses due to financial crises in terms of frequency (number of loss events per period) and severity (loss per occurrence). We perform the Loss Distribution Approach (LDA) to estimate a multi-country aggregate GDP loss probability density function and the percentiles associated to extreme eve…
Wide neural networks converge linearly to zero loss with feature learning.
We present GLASSES: Global optimisation with Look-Ahead through Stochastic Simulation and Expected-loss Search. The majority of global optimisation approaches in use are myopic, in only considering the impact of the next function value; the non-myopic approaches that do exist are able to consider only a handful of futu…
This paper addresses tracking of a moving target in a multi-agent network. The target follows a linear dynamics corrupted by an adversarial noise, i.e., the noise is not generated from a statistical distribution. The location of the target at each time induces a global time-varying loss function, and the global loss is…
Partial model averaging improves Federated Learning performance.
New deep learning method solves complex BSDEs efficiently.
Gradient flow in parameters equals linear interpolation in outputs.
The study identifies assets with local balance deviating from global balance to mitigate financial risk.
We analyze the loss landscape and expressiveness of practical deep convolutional neural networks (CNNs) with shared weights and max pooling layers. We show that such CNNs produce linearly independent features at a "wide" layer which has more neurons than the number of training samples. This condition holds e.g. for the…
We propose SEARNN, a novel training algorithm for recurrent neural networks (RNNs) inspired by the "learning to search" (L2S) approach to structured prediction. RNNs have been widely successful in structured prediction applications such as machine translation or parsing, and are commonly trained using maximum likelihoo…
Paper explains neural collapse in neural networks using a new model.
Global convergence of SGD proven for two-layer neural nets with regularization.
Study compares different scoring rules for machine-learned weather forecasts, finding scale-awareness improves forecast realism.
This paper introduces a scalable benchmark for evaluating local posterior sampling in neural networks.
Two-layer ReLU networks often converge to simpler solutions, improving generalization.
The permutation symmetry of neurons in each layer of a deep neural network gives rise not only to multiple equivalent global minima of the loss function, but also to first-order saddle points located on the path between the global minima. In a network of hidden layers with neurons in layers $k = 1, \ldots, …
While the optimization problem behind deep neural networks is highly non-convex, it is frequently observed in practice that training deep networks seems possible without getting stuck in suboptimal points. It has been argued that this is the case as all local minima are close to being globally optimal. We show that thi…
Study improves adversarial classification using distributionally robust models.
Recent work has noted that all bad local minima can be removed from neural network loss landscapes, by adding a single unit with a particular parameterization. We show that the core technique from these papers can be used to remove all bad local minima from any loss landscape, so long as the global minimum has a loss o…
This paper analyzes the landscape of supervised contrastive loss in over-parameterized networks.
A new algorithm improves federated learning by combining knowledge distillation and weighted combination loss.