arXiv research

We develop an improved bound for the approximation error of the Nyström method under the assumption that there is a large eigengap in the spectrum of kernel matrix. This is based on the empirical observation that the eigengap has a significant impact on the approximation error of the Nyström method. Our approach is bas…

A new GCN variant tackles large eigengaps in dense graphs and hypergraphs.

This paper speeds up spectral clustering for large graphs by dilating their eigenspectrum.

We consider the problem of principal component analysis (PCA) in a streaming stochastic setting, where our goal is to find a direction of approximate maximal variance, based on a stream of i.i.d. data points in $\reals^d$. A simple and computationally cheap algorithm for this is stochastic gradient descent (SGD), which…

We identify spectral conditions for reliable neural probe interpretation.

Spectral clustering algorithms typically require a priori selection of input parameters such as the number of clusters, a scaling parameter for the affinity measure, or ranges of these values for parameter tuning. Despite efforts for automating the process of spectral clustering, the task of grouping data in multi-scal…

We study $k$-GenEV, the problem of finding the top $k$ generalized eigenvectors, and $k$-CCA, the problem of finding the top $k$ vectors in canonical-correlation analysis. We propose algorithms $\mathtt{LazyEV}$ and $\mathtt{LazyCCA}$ to solve the two problems with running times linearly dependent on the input size and…

This paper tackles distributed estimation of the top-L eigenspace in PCA for large data sets.

We study streaming principal component analysis (PCA), that is to find, in $O(dk)$ space, the top $k$ eigenvectors of a $d\times d$ hidden matrix $\bf Σ$ with online vectors drawn from covariance matrix $\bf Σ$. We provide $\textit{global}$ convergence for Oja's algorithm which is popularly used in practice but lacks t…

We consider the problem of clustering with the longest-leg path distance (LLPD) metric, which is informative for elongated and irregularly shaped clusters. We prove finite-sample guarantees on the performance of clustering with respect to this metric when random samples are drawn from multiple intrinsically low-dimensi…

New asynchronous algorithms improve speed in decentralized optimization networks.

Study spectral learning for odeco tensors, addressing initialization bottlenecks.

Algorithm estimates principal eigenvector with adaptive sensing, improving over non-adaptive methods.

This paper characterizes and explains the disagreement between two graph embedding methods.

In this paper, we propose an acceleration scheme for online memory-limited PCA methods. Our scheme converges to the first $k>1$ eigenvectors in a single data pass. We provide empirical convergence results of our scheme based on the spiked covariance model. Our scheme does not require any predefined parameters such as t…

We consider the minimization of a function defined on a Riemannian manifold $\mathcal{M}$ accessible only through unbiased estimates of its gradients. We develop a geometric framework to transform a sequence of slowly converging iterates generated from stochastic gradient descent (SGD) on $\mathcal{M}$ to an averaged i…

Tensor factorization arises in many machine learning applications, such knowledge base modeling and parameter estimation in latent variable models. However, numerical methods for tensor factorization have not reached the level of maturity of matrix factorization methods. In this paper, we propose a new method for CP te…

We consider the problem of detecting a tight community in a sparse random network. This is formalized as testing for the existence of a dense random subgraph in a random graph. Under the null hypothesis, the graph is a realization of an Erdös-Rényi graph on $N$ vertices and with connection probability $p_0$; under the …

Spectral clustering is one of the most widely used techniques for extracting the underlying global structure of a data set. Compressed sensing and matrix completion have emerged as prevailing methods for efficiently recovering sparse and partially observed signals respectively. We combine the distance preserving measur…

In this paper, we determine the optimal convergence rates for strongly convex and smooth distributed optimization in two settings: centralized and decentralized communications over a network. For centralized (i.e. master/slave) algorithms, we show that distributing Nesterov's accelerated gradient descent is optimal and…

We consider decentralized stochastic optimization with the objective function (e.g. data samples for machine learning task) being distributed over $n$ machines that can only communicate to their neighbors on a fixed communication graph. To reduce the communication bottleneck, the nodes compress (e.g. quantize or sparsi…

Paper develops DP methods for low-rank matrix estimation with near-optimal performance.

The paper studies the graph geometry of finite groups, creating a dataset and analyzing its properties.

A geometric theory explains loss functions for robust representation learning.

Introduces resemblance structure for large scale geometry.

Minimal surfaces with negative curvature found in large spheres.

Ranky solves SVD for large sparse matrices in distributed systems.

Study shows HFT benefits large traders under certain conditions.

Study large deviations in life insurance portfolios without identical distributions.

IVON optimizes large neural networks, matching or outperforming Adam.

An extra large metric is a spherical cone metric with all cone angles greater than 2 pi and every closed geodesic longer than 2pi. We show that every two-dimensional extra large metric can be triangulated with vertices at cone points only. The argument implies the same result for Euclidean and hyperbolic cone metrics, …

Large deviations for fat tailed distributions, i.e. those that decay slower than exponential, are not only relatively likely, but they also occur in a rather peculiar way where a finite fraction of the whole sample deviation is concentrated on a single variable. The regime of large deviations is separated from the regi…

Study large deviations for hypoelliptic diffusion on sub-Riemannian manifolds.

Extends saddle-point method for large-time volatility smiles.

SNGM improves large-batch training accuracy.

GD with large init shows incremental learning in matrix factorization.

DReg boosts large-batch SGD's generalization and convergence.

Paper proves large deviation principle for stochastic approximations.

As it is known in the finance risk and macroeconomics literature, risk-sharing in large portfolios may increase the probability of creation of default clusters and of systemic risk. We review recent developments on mathematical and computational tools for the quantification of such phenomena. Limiting analysis such as …

Study shows how large neural networks avoid overfitting through decoupling of feature learning and complexity growth.

We study large deviations and rare default clustering events in a dynamic large heterogeneous portfolio of interconnected components. Defaults come as Poisson events and the default intensities of the different components in the system interact through the empirical default rate and via systematic effects that are comm…

Large learning rates enhance model robustness and compressibility.

We study the concept of coarse disjointness and large scale $n$-to-$1$ functions. As a byproduct, we obtain an Ostrand-type characterization of asymptotic dimension for coarse structures. It is shown that properties like finite asymptotic dimension, coarse finitism, large scale weak paracompactness, ect. are all invari…

Study large deviations in random walks on Lie groups.

Study large deviations in fractional volatility models with non-Gaussian volatility.

In these notes, we present some methods and applications of large deviations to finance and insurance. We begin with the classical ruin problem related to the Cramer's theorem and give en extension to an insurance model with investment in stock market. We then describe how large deviation approximation and importance s…

GNTK reveals convergence of GNNs on large graphs.

Training deep neural networks using a large batch size has shown promising results and benefits many real-world applications. However, the optimizer converges slowly at early epochs and there is a gap between large-batch deep learning optimization heuristics and theoretical underpinnings. In this paper, we propose a no…