A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Smoothed fitness landscape improves protein optimization.
Realistic evolutionary fitness landscapes are notoriously difficult to construct. A recent cutting-edge model of virus assembly consists of a dodecahedral capsid with corresponding packaging signals in three affinity bands. This whole genome/phenotype space consisting of genomes has been explored via comp…
Machine learning techniques are being increasingly used as flexible non-linear fitting and prediction tools in the physical sciences. Fitting functions that exhibit multiple solutions as local minima can be analysed in terms of the corresponding machine learning landscape. Methods to explore and visualise molecular pot…
We explore the energy landscape of a simple neural network. In particular, we expand upon previous work demonstrating that the empirical complexity of fitted neural networks is vastly less than a naive parameter count would suggest and that this implicit regularization is actually beneficial for generalization from fit…
SGD vs quasi-Newton optimization in neural networks: different landscapes, different generalizability.
Training an artificial neural network involves an optimization process over the landscape defined by the cost (loss) as a function of the network parameters. We explore these landscapes using optimisation tools developed for potential energy landscapes in molecular science. The number of local minima and transition sta…
Deep networks can interpolate noisy data without losing generalization.
We study how the behavior of deep policy gradient algorithms reflects the conceptual framework motivating their development. To this end, we propose a fine-grained analysis of state-of-the-art methods based on key elements of this framework: gradient estimation, value prediction, and optimization landscapes. Our result…
Computational models in fields such as computational neuroscience are often evaluated via stochastic simulation or numerical approximation. Fitting these models implies a difficult optimization problem over complex, possibly noisy parameter landscapes. Bayesian optimization (BO) has been successfully applied to solving…
In this paper we study the problem of learning a shallow artificial neural network that best fits a training data set. We study this problem in the over-parameterized regime where the number of observations are fewer than the number of parameters in the model. We show that with quadratic activations the optimization la…
Persistence landscapes map persistence diagrams into a function space, which may often be taken to be a Banach space or even a Hilbert space. In the latter case, it is a feature map and there is an associated kernel. The main advantage of this summary is that it allows one to apply tools from statistics and machine lea…
An evolutionary algorithm (EA) is developed as an alternative to the EM algorithm for parameter estimation in model-based clustering. This EA facilitates a different search of the fitness landscape, i.e., the likelihood surface, utilizing both crossover and mutation. Furthermore, this EA represents an efficient approac…
Reviews recent findings on neural network landscapes.
Quantification of the stationary points and the associated basins of attraction of neural network loss surfaces is an important step towards a better understanding of neural network loss surfaces at large. This work proposes a novel method to visualise basins of attraction together with the associated stationary points…
Notions of Darwinian selection have been implicit in economic theory for at least sixty years. Richard Nelson and Sidney Winter have argued that while evolutionary thinking was prevalent in prewar economics, the postwar Neoclassical school became almost entirely preoccupied with equilibrium conditions and their mathema…
We argue that in fully-connected networks a phase transition delimits the over- and under-parametrized regimes where fitting can or cannot be achieved. Under some general conditions, we show that this transition is sharp for the hinge loss. In the whole over-parametrized regime, poor minima of the loss are not encounte…
The study examines when MAML's objective has a benign landscape.
Gradient descent trains both layers of a ReLU network to fit a linear model.
Black holes offer insights into machine learning's loss landscapes.
Adversarial training makes logistic regression weight loss landscapes sharper.
AWP improves robustness by flattening weight loss landscape.
Efficiently infers graph edges from genetic similarity data in landscape genetics.
Researchers improve visualization of neural network loss landscapes.
Deeper models have a more favorable optimization landscape, making them more robust to noise.
New sampler tackles complex discrete energy landscapes efficiently.
We analyze the optimization landscape of α-loss in logistic models.
Study reveals structure of local minima in GMMs, identifying key cluster centers.
Adversarial training shows promise as an approach for training models that are robust towards adversarial perturbation. In this paper, we explore some of the practical challenges of adversarial training. We present a sensitivity analysis that illustrates that the effectiveness of adversarial training hinges on the sett…
Study reveals sharp characterisation of local minima in neural network loss landscapes.
It has been argued in the past that high-dimensional neural networks do not exhibit local minima capable of trapping an optimisation algorithm. However, the relationship between loss surface modality and the neural architecture parameters, such as the number of hidden neurons per layer and the number of hidden layers, …
Experimental fractal landscape dynamics observed in emulsions.
Study visualizes actor-critic loss landscapes for inventory optimization.
We study rough high-dimensional landscapes in which an increasingly stronger preference for a given configuration emerges. Such energy landscapes arise in glass physics and inference. In particular we focus on random Gaussian functions, and on the spiked-tensor model and generalizations. We thoroughly analyze the stati…
Considering the creation of persistence landscape on a parametrized curve and structure of sampling, there exists a random process for which a finite mixture model of persistence landscape (FMMPL) can provide a better description for a given dataset. In this paper, a nonparametric approach for computing integrated mean…
Analyzes adversarial training's impact on loss landscape, proposing PAS to improve model performance.
LSAM optimizes deep learning training with improved efficiency.
Paper characterizes optimization landscape of Tucker decomposition.
Researchers improve NCE by addressing its flat loss landscape issues.
Embedding principle explains loss landscape of deep neural networks.
There are many surprising and perhaps counter-intuitive properties of optimization of deep neural networks. We propose and experimentally verify a unified phenomenological model of the loss landscape that incorporates many of them. High dimensionality plays a key role in our model. Our core idea is to model the loss la…
Bayesian network (BN) modelling is extensively used in systems epidemiology. Usually it consists in selecting and reporting the best-fitting structure conditional to the data. A major practical concern is avoiding overfitting, on account of its extreme flexibility and its modelling richness. Many approaches have been p…
The landscape of empirical risk has been widely studied in a series of machine learning problems, including low-rank matrix factorization, matrix sensing, matrix completion, and phase retrieval. In this work, we focus on the situation where the corresponding population risk is a degenerate non-convex loss function, nam…
The study tests inferences about neural network optimization from linear interpolation of loss landscapes.
The paper analyzes phase retrieval under limited samples, ensuring a benign local landscape for convergence.
Deep ReLU networks with extra parameters have mostly good loss landscapes.
We investigate the structure of the profit landscape obtained from the most basic, fluctuation based, trading strategy applied for the daily stock price data. The strategy is parameterized by only two variables, p and q. Stocks are sold and bought if the log return is bigger than p and less than -q, respectively. Repet…
This work shows how to efficiently simulate parts of quantum landscapes using classical computers.
This work justifies neural collapse under MSE loss and analyzes the optimization landscape.