A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Mimetic initialization improves Transformer training on small datasets.
In this paper higher order mimetic discretizations are introduced which are firmly rooted in the geometry in which the variables are defined. The paper shows how basic constructs in differential geometry have a discrete counterpart in algebraic topology. Generic maps which switch between the continuous differential for…
We give a stochastic microscopic modelling of stock markets driven by continuous double auction. If we take into account the mimetic behavior of traders, when they place limit order, our virtual markets shows the power-law tail of the distribution of returns with the exponent outside the Levy stable region, the short m…
We propose a tensor neural network (-NN) framework that offers an exciting new paradigm for designing neural networks with multidimensional (tensor) data. Our network architecture is based on the -product (Kilmer and Martin, 2011), an algebraic formulation to multiply tensors via circulant convolution. In this $t…
Machine learning (ML) classifiers always benefit from more informative input features. We seek to auto-generate stronger feature sets in order to address the difficulty that ML methods often experience given limited training data. A wide range of biological neural nets (BNNs) excel at fast learning, implying that they …
Framework detects tipping points in complex systems using ML.
Generates tubular and membranous shapes using curvature functionals.
We design non-singular cloaks enabling objects to scatter waves like objects with smaller size and very different shapes. We consider the Schrodinger equation which is valid e.g. in the contexts of geometrical and quantum optics. More precisely, we introduce a generalized non-singular transformation for star domains, a…
Online DEM improves tracking of latent states in dynamic systems.
Paper optimizes neural network initialization using SMT solvers.
New method for better initial centers in clustering with improved accuracy and privacy.
Study shows different initialization schemes for LoRA finetuning impact performance.
New method learns good initialization for gradient descent from past solutions.
Initial data for -wave spacetimes constructed in 4D.
In this paper, we firstly give a brief introduction of expectation maximization (EM) algorithm, and then discuss the initial value sensitivity of expectation maximization algorithm. Subsequently, we give a short proof of EM's convergence. Then, we implement experiments with the expectation maximization algorithm (We im…
The paper establishes principles for initializing and designing GNNs with ReLU activations to avoid oversmoothing and correlation collapse.
The performance of the Self-Organizing Map (SOM) algorithm is dependent on the initial weights of the map. The different initialization methods can broadly be classified into random and data analysis based initialization approach. In this paper, the performance of random initialization (RI) approach is compared to that…
New distributed clustering algorithms show resilience to initialization issues.
Initializing the weights and the biases is a key part of the training process of a neural network. Unlike the subsequent optimization phase, however, the initialization phase has gained only limited attention in the literature. In this paper we discuss some consequences of commonly used initialization strategies for va…
The selection of initial parameter values for gradient-based optimization of deep neural networks is one of the most impactful hyperparameter choices in deep learning systems, affecting both convergence times and model performance. Yet despite significant empirical and theoretical analysis, relatively little has been p…
Barren plateaus are not an average-case phenomenon, but a highly non-unique problem.
Standard practice in training neural networks involves initializing the weights in an independent fashion. The results of recent work suggest that feature "diversity" at initialization plays an important role in training the network. However, other initialization schemes with reduced feature diversity have also been sh…
Class I CR manifolds have initial G-structure a certain 4-dimensional subgroup of GL_3(C). Class II CR manifolds have initial G-structure a certain 10-dimensional subgroup of GL_4(C). Class III-1 CR manifolds have initial G-structure a certain 10-dimensional subgroup of GL_5(C). Class III-2 CR manifolds have initial G-…
The dying ReLU refers to the problem when ReLU neurons become inactive and only output 0 for any input. There are many empirical and heuristic explanations of why ReLU neurons die. However, little is known about its theoretical analysis. In this paper, we rigorously prove that a deep ReLU network will eventually die in…
New method for initializing RBM weights without datasets.
Paper proves rigidity of initial data sets with boundary and capillary MOTS.
Deep networks retain initial bias after training, affecting generalization.
AutoInit automatically finds good neural network initialization.
Residual networks (ResNet) and weight normalization play an important role in various deep learning applications. However, parameter initialization strategies have not been studied previously for weight normalized networks and, in practice, initialization methods designed for un-normalized networks are used as a proxy.…
Neural nets trained with linear discriminant initialization converge faster and more accurately.
Study evaluates initialization strategies for infinite hidden Markov models.
This paper presents a phase diagram for two-layer neural networks under different initialization scales.
Proves density and mass theorems for specific initial data sets.
A new neural network initialization method is proposed for faster and more accurate training.
When working with asymptotically hyperbolic initial data sets for general relativity it is convenient to assume certain simplifying properties. We prove that the subset of initial data sets with such properties is dense in the set of physically reasonable asymptotically hyperbolic initial data sets. More specifically, …
The success of lottery ticket initializations (Frankle and Carbin, 2019) suggests that small, sparsified networks can be trained so long as the network is initialized appropriately. Unfortunately, finding these "winning ticket" initializations is computationally expensive. One potential solution is to reuse the same wi…
Two new scalable K-means initialization methods proposed for large-scale clustering.
Improves LSTM performance by initializing states via manifold learning.
Estimates bandwidth for CMC initial data sets.
Survey on preserving curvature bounds for non-smooth Ricci flow.
Article strengthens initial data rigidity theorem to show unique spacetime extension.
This paper examines weight initialization for 1-Lipschitz networks to improve robustness against adversarial attacks.
In this paper, we present a novel approach for initializing deep neural networks, i.e., by turning PCA into neural layers. Usually, the initialization of the weights of a deep neural network is done in one of the three following ways: 1) with random values, 2) layer-wise, usually as Deep Belief Network or as auto-encod…
New method shows random, diverse initializations are not essential for deep neural networks.
Due to the iterative nature of most nonnegative matrix factorization (\textsc{NMF}) algorithms, initialization is a key aspect as it significantly influences both the convergence and the final solution obtained. Many initialization schemes have been proposed for NMF, among which one of the most popular class of methods…
Deep neural networks achieve state-of-the-art performance for a range of classification and inference tasks. However, the use of stochastic gradient descent combined with the nonconvexity of the underlying optimization problems renders parameter learning susceptible to initialization. To address this issue, a variety o…
Our study analyzes how neural network initialization affects privacy and utility in overparameterized models.
Optimizes deep neural network initialization variance for better performance.