Essay on deep learning's optimization mystery.
problem Understanding how neural networks learn and generalize.
method Review and connections between works on optimization.
result Connections between optimization works in deep learning.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,932 papers · 148 categories
Trend · papers per month
48 results for “mystery”
Galor discovered many mysteries of the growth process. He lists them in his Unified Growth Theory and wonders how they can be explained. Close inspection of his mysteries reveals that they are of his own creation. They do not exist. He created them by his habitually distorted presentation of data. One of his self-creat…
Deep learning's anomalous generalization explained by standard frameworks.
problem Anomalous generalization in deep neural networks.
method Intuitive understanding and rigorous characterization using PAC-Bayes and countable hypothesis bounds.
result Deep learning's anomalous generalization can be explained by soft inductive biases.
Lisa Piccirillo solved the mystery of the Conway knot's sliceness.
problem Determining if the Conway knot is slice.
method The main idea of the proof is given in the title.
result The Conway knot is not slice.
The so-called great divergence in the income per capita is described in the Unified Growth Theory as the mind-boggling and unresolved mystery about the growth process. This mystery has now been solved: the great divergence never happened. It was created by the manipulation of data. Economic growth in various regions is…
This is an expository paper discussing some parallels between the Khovanov and knot Floer homologies. We describe the formal similarities between the theories and give some examples which illustrate a somewhat mysterious correspondence between them.
The local structure of Finsler metrics of constant flag curvature have been historically mysterious. It is proved that every Matsumoto metric of constant flag curvature on a manifold of dimension n \geq 3 is either Riemannian or locally Minkowskian.
We reveal a model rank that predicts successful recovery of target functions at overparameterization.
problem Understanding the mysterious good generalization performance of overparameterized nonlinear models.
method Rank stratification and linear stability theory for general nonlinear models.
result Linearly stable functions are preferred by nonlinear training, and model rank predicts minimal training data size.
As a new step in the study of rectangularly-colored knot polynomials, we reformulate the prescription of arXiv:1606.06015 for twist knots in the double-column representations in terms of skew Schur polynomials. These, however, are mysteriously shifted from the standard topological locus, what makes further gen…
Bach-Flat Kaehler Surfacesmath.DG
A Riemannian metric on a compact 4-manifold is said to be Bach-flat if it is a critical point for the L2-norm of the Weyl curvature. When the Riemannian 4-manifold in question is a Kaehler surface, we provide a rough classification of solutions, followed by detailed results regarding each case in the classification. Th…
Study uses machine learning to analyze solar emissions.
problem Explaining the high temperature of the solar corona.
method Unsupervised machine learning to characterize impulsive emissions.
result Characterized over 34,500 features as 2D elliptical Gaussians.
Any generalized distance-squared mapping of equidimensional case has singularities, and their singularity types are wrapped into mystery in higher dimensional cases. Any generalized distance-squared mapping of equidimensional case is not injective. Nevertheless, in this paper, it is shown that the non-singular property…
The Teichmuller unipotent flow can be defined concretely on certain moduli spaces of singular flat surfaces by shearing polygonal presentations of the surfaces. Thurston's earthquake flow on moduli spaces of hyperbolic surfaces is more mysterious. Both flows have deep and important connections to other areas of mathema…
Harish-Chandra's volume formula shows that the volume of a flag manifold , where the measure is induced by an invariant inner product on the Lie algebra of , is determined up to a scalar by the algebraic properties of . This article explains how to deduce Harish-Chandra's formula from Weyl's law by utilizing…
We claim that the recently discovered universal-matrix precursor for the functions, which define the differential expansion of colored polynomials for twist and double braid knots, can be extended from rectangular to non-rectangular representations. This case is far more interesting, because it involves multiplicit…
New methods reveal colored Jones polynomials from quantum R-matrices and knot invariants.
problem Understanding colored Jones polynomials of knots.
method Two realizations: quantum R-matrices and refined quantum modularity conjecture.
result New insights into knot invariants from quantum R-matrices and matrix conjectures.
Truncated SGD with heavy-tailed noise eliminates sharp local minima.
problem Avoiding sharp local minima in deep learning models.
method Truncated SGD with heavy-tailed gradient noise.
result Truncated SGD can eliminate sharp local minima entirely from its training trajectory.
Explains planetary motion in a sub-Riemannian setting.
problem Kepler-Heisenberg problem
method Classical Kepler problem adapted to sub-Riemannian geometry
result Rich and mysterious dynamical system with tractable questions
We prove a formula relating the analytic torsion and Reidemeister torsion on manifolds with boundary in the general case when the metric is not necessarily a product near the boundary. The product case has been established by W. Luck and S. M. Vishik. We find that the extra term that comes in here in the nonproduct cas…
We prove that the degenerate part of the distributive homology of a multispindle is determined by the normalized homology. In particular, when the multispindle is a quandle , the degenerate homology of is completely determined by the quandle homology of . For this case (and generally for two term homology of …
The purpose of the present paper is to introduce and explore two surprises that arise when we apply a standard procedure to study the number of finite type invariants of 3-manifolds introduced independently by M. Goussarov and K. Habiro based on surgery on claspers, Y-graphs or clovers, \cite{Gu,Ha,GGP}. One surprise i…
This work uses visualizations to make generalization of neural networks more intuitive.
problem Understanding the reasons behind neural networks' ability to generalize to unseen data.
method Visualization methods to explain the geometry of loss landscapes and the role of dimensionality in optimization.
result Visualization helps in understanding how optimizers settle into minima that generalize well.
The Langlands Program was launched in the late 60s with the goal of relating Galois representations and automorphic forms. In recent years a geometric version has been developed which leads to a mysterious duality between certain categories of sheaves on moduli spaces of (flat) bundles on algebraic curves. Three years …
Construction of (colored) knot polynomials for double-fat graphs is further generalized to the case when "fingers" and "propagators" are substituting R-matrices in arbitrary closed braids with m-strands. Original version of arXiv:1504.00371 corresponds to the case m=2, and our generalizations sheds additional light on …
Adam optimizes linear classifiers with separable data.
problem Understanding Adam's implicit bias in linear logistic regression.
method Study of Adam's behavior on linearly separable data.
result Adam converges to a linear classifier with maximum -margin.
Graph neural networks struggle with proving unsatisfiability in complex logical formulas.
problem Proving unsatisfiability in complex logical formulas.
method Investigating the limitations of graph neural networks in logical reasoning tasks.
result Graph neural networks may fail in certifying unsatisfiability in Boolean formulae.
Why and how that deep learning works well on different tasks remains a mystery from a theoretical perspective. In this paper we draw a geometric picture of the deep learning system by finding its analogies with two existing geometric structures, the geometry of quantum computations and the geometry of the diffeomorphic…
GeoAdaLer enhances geometric understanding of Adam for stochastic optimization.
problem Understanding geometric principles behind Adam's success in stochastic optimization.
method Introduces GeoAdaLer, an adaptive learning method based on geometric properties.
result Extends interpretability and effectiveness in complex optimization scenarios.
Analyzes the generality of solitons for structures.
problem Understanding the space of solitons for the Laplacian flow of closed -structures.
method Constructs a natural exterior differential system whose integral manifolds describe solitons and applies Cartan-Kahler theory.
result For closed solitons, the germs depend on 16 functions of 6 variables.
Harer-Zagier formulas generalized to knot matrix models.
problem Understanding knot polynomials through matrix models.
method Defined knot matrix models and extracted averages.
result Harer-Zagier formulas factorize for torus knots but not for others.
SMILE improves explainability of machine learning models.
problem Difficulty in understanding and trusting the conclusions of black-box machine learning models.
method Statistical Model-agnostic Interpretability with Local Explanations (SMILE).
result SMILE makes machine learning models more interpretable.
Why deep neural networks (DNNs) capable of overfitting often generalize well in practice is a mystery [#zhang2016understanding]. To find a potential mechanism, we focus on the study of implicit biases underlying the training process of DNNs. In this work, for both real and synthetic datasets, we empirically find that a…
Motivated by physical constructions of homological knot invariants, we study their analogs for closed 3-manifolds. We show that fivebrane compactifications provide a universal description of various old and new homological invariants of 3-manifolds. In terms of 3d/3d correspondence, such invariants are given by the Q-c…
To pave the way for the journey from geometry to conformal field theory (CFT), these notes present the background for some basic CFT constructions from Calabi-Yau geometry. Topics include the complex and Kaehler geometry of Calabi-Yau manifolds and their classification in low dimensions. I furthermore discuss CFT const…
Given functional data from a survival process with time-dependent covariates, we derive a smooth convex representation for its nonparametric log-likelihood functional and obtain its functional gradient. From this, we devise a generic gradient boosting procedure for estimating the hazard function nonparametrically. An i…
New methods stabilize Q-learning with linear approximations.
problem Stabilizing -learning with linear function approximation.
method Target network and truncation.
result Provably stable -learning with linear function approximation.
The paper explores algebraic properties of Alexander polynomials and Reidemeister torsions for torus knots.
problem Investigating algebraic properties of Alexander polynomials and Reidemeister torsions for torus knots.
method Analyzing twisted Alexander polynomials and Reidemeister torsions of torus knots associated with irreducible -representations.
result Proves that coefficients of twisted Alexander polynomials are locally constant functions on the -character variety.
The AlphaGo, AlphaGo Zero, and AlphaZero series of algorithms are remarkable demonstrations of deep reinforcement learning's capabilities, achieving superhuman performance in the complex game of Go with progressively increasing autonomy. However, many obstacles remain in the understanding of and usability of these prom…
Learning Linear Regression with Low-Rank Tasks in-Contextcond-mat.dis-nn
This work analyzes how transformers learn common linear regression tasks.
problem Understanding how in-context learning operates in real-world applications with common task structures.
method Analyzing a linear attention model trained on low-rank regression tasks.
result Statistical fluctuations in finite pre-training data induce an implicit regularization, leading to a sharp phase transition in generalization error.
Differential Expansion for antiparallel triple pretzels: the way the factorization is deformedhep-th
Factorization of DE coefficients is violated in antiparallel triple pretzels, but described elegantly.
problem Understanding the origins of factorization in double braids and its extension to antiparallel triple pretzels.
method Defect-preserving deformation from trefoil to antiparallel triple pretzels, analysis of DE coefficients.
result Factorization of DE coefficients is violated but described by an elegant formula for symmetric representations.
Study of Schwarzian derivative on Finsler manifolds with constant curvature.
problem Characterizing the role of the Schwarzian derivative in Finsler manifolds of constant curvature.
method Developed integrability conditions and rigidity results for Möbius equations on Finsler manifolds.
result Complete Finsler manifolds of positive constant Ricci curvature are homeomorphic to the n-sphere if they admit non-trivial Möbius mappings.
Explains neural network properties with a simple operation.
problem Understanding neural network performance and generalization.
method Analyze a single operation: expand-and-sparsify.
result Clarifies phenomena like Lottery Ticket Hypothesis and overparameterized models.
By analyzing how the Borel regulator classes vanish on various groups related to , we define three series of secondary characteristic classes for subgroups of automorphism groups of free groups. The first case is the -automorphism groups and we show that our classes coincide with…
New method explains GNNs using power iteration clustering.
problem Mysterious mechanism of message passing in GNNs.
method Subspace power iteration clustering (SPIC) models.
result Message passing in GNNs can be understood through power iteration.
This paper explains spectral clustering and its equivalence to PCA, breaking it into fully connected and multi-connected cases.
problem Understanding the mathematics behind spectral clustering and its equivalence to PCA.
method Dividing spectral clustering into two categories based on graph connectivity and proving the equivalence to PCA.
result Spectral clustering and PCA are equivalent, with specific proofs for fully connected and multi-connected graphs.
Oeljeklaus-Toma (OT) manifolds are certain compact complex manifolds built from number fields. Conversely, we show that the fundamental group often pins down the number field uniquely. We relate the first homology to some interesting ideal. OT manifolds are never Kähler, but carry an LCK metric (locally conformally Käh…
The AdaBoost algorithm has the superiority of resisting overfitting. Understanding the mysteries of this phenomena is a very fascinating fundamental theoretical problem. Many studies are devoted to explaining it from statistical view and margin theory. In this paper, we illustrate it from feature learning viewpoint, an…
Dropout regularization of deep neural networks has been a mysterious yet effective tool to prevent overfitting. Explanations for its success range from the prevention of "co-adapted" weights to it being a form of cheap Bayesian inference. We propose a novel framework for understanding multiplicative noise in neural net…