arXiv research

Proves properties of neural network basins of attraction and their expressiveness.

Straight lines are a basin of attraction for the elastic flow at least to level 1.9615π.

Quantification of the stationary points and the associated basins of attraction of neural network loss surfaces is an important step towards a better understanding of neural network loss surfaces at large. This work proposes a novel method to visualise basins of attraction together with the associated stationary points…

New method finds basins of attraction without needing system models.

New method detects metastable basins in high dimensions using trajectory sampling.

Recently, a general method for analyzing the statistical accuracy of the EM algorithm has been developed and applied to some simple latent variable models [Balakrishnan et al. 2016]. In that method, the basin of attraction for valid initialization is required to be a ball around the truth. Using Stein's Lemma, we exten…

Framework analyzes neural network dynamics for better understanding and optimization.

Neural networks' optimization dynamics are confined to a single basin despite connected basins in the loss landscape.

Multifractality in time series arises from temporal correlations, not just fat tails.

Biological neurons learn tensor decompositions of higher-order correlations using nonlinear Hebbian plasticity.

We consider compressed sensing formulated as a minimization problem of nonconvex sparse penalties, Smoothly Clipped Absolute deviation (SCAD) and Minimax Concave Penalty (MCP). The nonconvexity of these penalties is controlled by nonconvexity parameters, and L1 penalty is contained as a limit with respect to these para…

Study SRB measures for Anosov actions on manifolds.

We introduce a mathematical model on the dynamics of demand and supply incorporating collectability and saturation factors. Our analysis shows that when the fluctuation of the determinants of demand and supply is strong enough, there is chaos in the demand-supply dynamics. Our numerical simulation shows that such a cha…

Recent work has demonstrated the effectiveness of gradient descent for directly recovering the factors of low-rank matrices from random linear measurements in a globally convergent manner when initialized properly. However, the performance of existing algorithms is highly sensitive in the presence of outliers that may …

Density mode clustering is a nonparametric clustering method. The clusters are the basins of attraction of the modes of a density estimator. We study the risk of mode-based clustering. We show that the clustering risk over the cluster cores --- the regions where the density is high --- is very small even in high dimens…

Study of holomorphic correspondences combining entire maps and Fuchsian groups.

We analyze the dynamical properties of a tetrahedron transformation on the space of non-degenerate tetrahedra which can be identified with the non-compact globally symmetric $8$-dimensional space $\mbox{Sl}(3,\mathbb{R}) / \mbox{So}(3,\mathbb{R})$. We establish the existence of a local attractor which coincides with th…

Mode clustering is a nonparametric method for clustering that defines clusters using the basins of attraction of a density estimator's modes. We provide several enhancements to mode clustering: (i) a soft variant of cluster assignment, (ii) a measure of connectivity between clusters, (iii) a technique for choosing the …

In this paper we investigate the mean curvature flow (MCF) of a regular leaf of a closed generalized isoparametric foliation as initial datum, generalizing previous results of Radeschi and first author. We show that, under bounded curvature conditions, any finite time singularity is a singular leaf, and the singularity…

Let $M$ be a manifold or (more generally) a locally compact, metrizable ANR. If $K$ is an attractor for a flow in $M$, with basin of attraction $\mathcal{A}(K)$, it is well known that the inclusion $i : K \subseteq \mathcal{A}(K)$ is always a shape equivalence. In this paper we investigate to what extent this generaliz…

The study identifies two minimal orbits of foliations on complex projective plane and explores their properties.

New method improves ensemble quality by exploring the pre-train basin more effectively.

SLT reveals how grokking occurs via basin selection in training.

An image pattern can be represented by a probability distribution whose density is concentrated on different low-dimensional subspaces in the high-dimensional image space. Such probability densities have an astronomical number of local modes corresponding to typical pattern appearances. Related groups of modes can join…

Wide networks are often believed to have a nice optimization landscape, but what rigorous results can we prove? To understand the benefit of width, it is important to identify the difference between wide and narrow networks. In this work, we prove that from narrow to wide networks, there is a phase transition from havi…

Quantization-aware training can recover accuracy lost by post-training quantization.

Kleinberg introduced three natural clustering properties, or axioms, and showed they cannot be simultaneously satisfied by any clustering algorithm. We present a new clustering property, Monotonic Consistency, which avoids the well-known problematic behaviour of Kleinberg's Consistency axiom, and the impossibility resu…

Proposes a method to sample from flat basins of posterior distributions in Bayesian deep learning.

Novel Ricci flow normalization for homogeneous spaces, focusing on flag manifolds.

Study characterizes spike deconvolution basin for noisy data.

A graph clustering method that moves nodes to highest-degree neighbors.

The study revisits Hopfield's associative memory model and calculates its capacity for two specific pattern basins.

Little is known about the global structure of the basins of attraction of Newton's method in two or more complex variables. We make the first steps by focusing on the specific Newton mapping to solve for the common roots of $P(x,y) = x(1-x)$ and $Q(x,y) = y^2+Bxy-y$. There are invariant circles $S_0$ and $S_1$ within t…

Proposes a probabilistic model to improve hydrology predictions and trust.

This research explains why SGD generalizes better than ADAM in deep learning.

Study of separatrix configurations in holomorphic flows with real time.

New method controls mean exit time in stochastic systems using machine learning and quasipotential.

Flooding is a destructive and dangerous hazard and climate change appears to be increasing the frequency of catastrophic flooding events around the world. Physics-based flood models are costly to calibrate and are rarely generalizable across different river basins, as model outputs are sensitive to site-specific parame…

It is widely observed that deep learning models with learned parameters generalize well, even with much more model parameters than the number of training samples. We systematically investigate the underlying reasons why deep neural networks often generalize well, and reveal the difference between the minima (with the s…

Study global geometry of dynamical systems with entire vector fields.

Based on the Multifractal Detrended Fluctuation Analysis (MFDFA) and on the Wavelet Transform Modulus Maxima (WTMM) methods we investigate the origin of multifractality in the time series. Series fluctuating according to a qGaussian distribution, both uncorrelated and correlated in time, are used. For the uncorrelated …

Regional rainfall-runoff modeling is an old but still mostly out-standing problem in Hydrological Sciences. The problem currently is that traditional hydrological models degrade significantly in performance when calibrated for multiple basins together instead of for a single basin alone. In this paper, we propose a nov…

We consider the problem of sparse coding, where each sample consists of a sparse linear combination of a set of dictionary atoms, and the task is to learn both the dictionary elements and the mixing coefficients. Alternating minimization is a popular heuristic for sparse coding, where the dictionary and the coefficient…

We study codimension one foliations with singularities defined locally by Bott-Morse functions on closed oriented manifolds. We carry to this setting the classical concepts of holonomy of invariant sets and stability, and prove a stability theorem in the spirit of the local stability theorem of Reeb. This yields, among…

Study shows bifurcating price dynamics in ASME with traders.

In this paper, we consider parameter recovery for non-overlapping convolutional neural networks (CNNs) with multiple kernels. We show that when the inputs follow Gaussian distribution and the sample size is sufficiently large, the squared loss of such CNNs is $\mathit{~locally~strongly~convex}$ in a basin of attraction…

Researchers use Gaussian processes with non-stationary kernels to model precipitation patterns in the Upper Indus Basin.

The process of transforming observed data into predictive mathematical models of the physical world has always been paramount in science and engineering. Although data is currently being collected at an ever-increasing pace, devising meaningful models out of such observations in an automated fashion still remains an op…