A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
The exploding and vanishing gradient problem has been the major conceptual principle behind most architecture and training improvements in recurrent neural networks (RNNs) during the last decade. In this paper, we argue that this principle, while powerful, might need some refinement to explain recent developments. We r…
Variational methods that rely on a recognition network to approximate the posterior of directed graphical models offer better inference and learning than previous methods. Recent advances that exploit the capacity and flexibility in this approach have expanded what kinds of models can be trained. However, as a proposal…
Improved analysis for clipped gradient methods in nonsmooth convex optimization under heavy-tailed noise.
problem Optimization under heavy-tailed noise in nonsmooth convex problems.
method Refined analysis of Clipped Stochastic Gradient Descent (Clipped SGD) with new rates and improved utilization of Freedman's inequality.
result New rates O(σldmeff−1/2pln1−1/p(1/δ)T1/p−1) and O(σl2dmeff−1/pln2−2/p(1/δ)T2/p−2) for nonsmooth convex and strongly convex problems, respectively.
Over the past decade there has been considerable interest in spectral algorithms for learning Predictive State Representations (PSRs). Spectral algorithms have appealing theoretical guarantees; however, the resulting models do not always perform well on inference tasks in practice. One reason for this behavior is the m…
Study on gradient h-almost Yamabe solitons with scalar curvature estimation.
problem Exploring triviality and scalar curvature estimation of gradient h-almost Yamabe solitons.
method Established sufficient conditions for triviality and scalar curvature estimation under integral inequalities involving the scalar curvature and soliton function.
result Extended and refined former works on almost and h-almost Yamabe solitons, characterizing their geometric structures.
Graph refinement, or the task of obtaining subgraphs of interest from over-complete graphs, can have many varied applications. In this work, we extract trees or collection of sub-trees from image data by, first deriving a graph-based representation of the volumetric data and then, posing the tree extraction as a graph …
The paper analyzes stability and generalization of shallow neural networks using gradient methods.
problem Understanding the generalization of overparameterized shallow neural networks.
method The paper uses gradient descent and stochastic gradient descent to study shallow neural networks, developing consistent excess risk bounds.
result The analysis improves on existing methods by providing a refined estimation of iterates and Hessian eigenvalues, leading to better excess risk bounds.
The challenge of assigning importance to individual neurons in a network is of interest when interpreting deep learning models. In recent work, Dhamdhere et al. proposed Total Conductance, a "natural refinement of Integrated Gradients" for attributing importance to internal neurons. Unfortunately, the authors found tha…
The standard practice in Generative Adversarial Networks (GANs) discards the discriminator during sampling. However, this sampling method loses valuable information learned by the discriminator regarding the data distribution. In this work, we propose a collaborative sampling scheme between the generator and the discri…
Classically, the time complexity of a first-order method is estimated by its number of gradient computations. In this paper, we study a more refined complexity by taking into account the `lingering' of gradients: once a gradient is computed at xk, the additional time to compute gradients at xk+1,xk+2,… m…
The study analyzes how many neurons are needed for two-layer neural networks trained with gradient descent.
problem Determining the minimum number of neurons required for effective training of shallow neural networks.
method Analyzes two-layer neural networks in the NTK regime, trained with gradient descent. Derives fast rates of convergence and tracks the number of hidden neurons required for generalization.
result Derives fast rates of convergence and improves on existing results for the number of hidden neurons needed for generalization.
We study refined topological string theory in the presence of orientifolds by counting second-quantized BPS states in M-theory. This leads us to propose a new integrality condition for both refined and unrefined topological strings when orientifolds are present. We define the SO(2N) refined Chern-Simons theory which co…
In this paper, we consider unregularized online learning algorithms in a Reproducing Kernel Hilbert Spaces (RKHS). Firstly, we derive explicit convergence rates of the unregularized online learning algorithms for classification associated with a general gamma-activating loss (see Definition 1 in the paper). Our results…
We consider the L2 gradient flow for the Willmore functional in Riemannian manifolds of bounded geometry. In the euclidean case E.\;Kuwert and R.\;Schätzle [\textsl{Gradient flow for the Willmore functional,} Comm. Anal. Geom., 10: 307-339, 2002] established a lower bound of a smooth solution of such a flow, which d…
Generative model initializes 2-layer network weights for small datasets.
problem Approximating functions with 2-layer networks using small datasets and gradient-based training.
method Initialize hidden weights with a learned proposal distribution parameterized as a deep generative model. Refine with gradient-based post-processing and regularization.
result Demonstrates effectiveness of the approach with numerical examples.
We investigate 1) the rate at which refined properties of the empirical risk---in particular, gradients---converge to their population counterparts in standard non-convex learning tasks, and 2) the consequences of this convergence for optimization. Our analysis follows the tradition of norm-based capacity control. We p…
We complete the picture of sharp eigenvalue estimates for the p-Laplacian on a compact manifold by providing sharp estimates on the first nonzero eigenvalue of the nonlinear operator Δp when the Ricci curvature is bounded from below by a negative constant. We assume that the boundary of the manifold is convex, and p…