arXiv research

New analysis improves SGD for robust and quantile regression with sub-quadratic convergence.

Study sharp convergence rates of empirical UOT for spatio-temporal point processes.

Study shows uniqueness of solutions on complex manifolds without requiring solution decay.

Efficient sparse modern Hopfield models are introduced for memory retrieval and learning tasks.

New findings on gradient expanding Ricci solitons with finite scalar curvature ratio.

Adam converges to stationary points under relaxed conditions.

Quadratic regression involves modeling the response as a (generalized) linear function of not only the features $x^{j_1}$ but also of quadratic terms $x^{j_1}x^{j_2}$. The inclusion of such higher-order "interaction terms" in regression often provides an easy way to increase accuracy in already-high-dimensional problem…

We present a new algorithm, trimed, for obtaining the medoid of a set, that is the element of the set which minimises the mean distance to all other elements. The algorithm is shown to have, under certain assumptions, expected run time O(N^(3/2)) in R^d where N is the set size, making it the first sub-quadratic exact m…

We consider complete non-compact manifolds with either a sub-quadratic growth of the norm of the Riemann curvature, or a sub-quadratic growth of both the norm of the Ricci curvature and the squared inverse of the injectivity radius. We show the existence on such a manifold of a distance-like function with bounded gradi…

The paper tests properties of trees in graphical models using covariance queries.

The family of temporal difference (TD) methods span a spectrum from computationally frugal linear methods like TD(λ) to data efficient least squares methods. Least square methods make the best use of available data directly computing the TD solution and thus do not require tuning a typically highly sensitive learning r…

The covariance matrix of a $p$-dimensional random variable is a fundamental quantity in data analysis. Given $n$ i.i.d. observations, it is typically estimated by the sample covariance matrix, at a computational cost of $O(np^{2})$ operations. When $n,p$ are large, this computation may be prohibitively slow. Moreover, …

We study manifolds satisfying a weighed Poincare inequality, which was first introduced by Li-Wang. We generalized one of their results by relaxing the Ricci curvature bound condition only being satisfied outside a compact set and established a finitely many ends result. We proved a vanishing result for $L^2$ harmonic …

Improved MMD test for two-sample testing with random Fourier features.

New Performer model tackles long-sequence protein modeling.

The so-called {\it kissing number} for hyperbolic surfaces is the maximum number of homotopically distinct systoles a surface of given genus $g$ can have. These numbers, first studied (and named) by Schmutz Schaller by analogy with lattice sphere packings, are known to grow, as a function of genus, at least like $g^{\s…

Inversion-free natural gradient method for Riemannian manifolds.

Study calculates stable norm of slit tori using Farey sequence.

SGD with constant stepsize converges to a non-Gaussian limit near flat minima.

New compression methods handle biased input sequences for more accurate posterior summaries.

Semidefinite programs (SDP) are important in learning and combinatorial optimization with numerous applications. In pursuit of low-rank solutions and low complexity algorithms, we consider the Burer--Monteiro factorization approach for solving SDPs. We show that all approximate local optima are global optima for the pe…

We describe the first sub-quadratic sampling algorithm for the Multiplicative Attribute Graph Model (MAGM) of Kim and Leskovec (2010). We exploit the close connection between MAGM and the Kronecker Product Graph Model (KPGM) of Leskovec et al. (2010), and show that to sample a graph from a MAGM it suffices to sample sm…

Mamba struggles with long context lengths, but spectrum scaling improves performance.

Empirical risk minimization (ERM) is ubiquitous in machine learning and underlies most supervised learning methods. While there has been a large body of work on algorithms for various ERM problems, the exact computational complexity of ERM is still not understood. We address this issue for multiple popular ERM problems…

DBSCAN is a classical density-based clustering procedure with tremendous practical relevance. However, DBSCAN implicitly needs to compute the empirical density for each sample point, leading to a quadratic worst-case time complexity, which is too slow on large datasets. We propose DBSCAN++, a simple modification of DBS…

A known failing of many popular random graph models is that the Aldous-Hoover Theorem guarantees these graphs are dense with probability one; that is, the number of edges grows quadratically with the number of nodes. This behavior is considered unrealistic in observed graphs. We define a notion of edge exchangeability …

We propose an original particle-based implementation of the Loopy Belief Propagation (LPB) algorithm for pairwise Markov Random Fields (MRF) on a continuous state space. The algorithm constructs adaptively efficient proposal distributions approximating the local beliefs at each note of the MRF. This is achieved by cons…

Sparse Gaussian processes with compact kernels for faster inference.

Most of machine learning approaches have stemmed from the application of minimizing the mean squared distance principle, based on the computationally efficient quadratic optimization methods. However, when faced with high-dimensional and noisy data, the quadratic error functionals demonstrated many weaknesses including…

We analyze computational limits of modern Hopfield models based on pattern norms.

We consider support recovery in the quadratic logistic regression setting - where the target depends on both p linear terms $x_i$ and up to $p^2$ quadratic terms $x_i x_j$. Quadratic terms enable prediction/modeling of higher-order effects between features and the target, but when incorporated naively may involve solvi…

A new framework for efficient sequence maps using Bayesian filtering and covariance.

We consider the problem of recovering a signal $\mathbf{x}^* \in \mathbf{R}^n$, from magnitude-only measurements $y_i = |\left\langle\mathbf{a}_i,\mathbf{x}^*\right\rangle|$ for $i=[m]$. Also called the phase retrieval, this is a fundamental challenge in bio-,astronomical imaging and speech processing. The problem abov…

A common analytical problem in neuroscience is the interpretation of neural activity with respect to sensory input or behavioral output. This is typically achieved by regressing measured neural activity against known stimuli or behavioral variables to produce a "tuning function" for each neuron. Unfortunately, because …

New algorithm reduces runtime for robust sparse mean estimation.

Single-head transformers with a single self-attention layer can approximate any sequence-to-sequence function and are efficient under certain conditions.

QATS efficiently decodes HMMs with polylogarithmic complexity.

Prototype selection improved using topological data analysis.

Efficiently accelerates attention calculation for Transformers with relative positional encoding.

We analyze the computational limits of LoRA for transformer models using fine-grained complexity theory.

Max-convolution is an important problem closely resembling standard convolution; as such, max-convolution occurs frequently across many fields. Here we extend the method with fastest known worst-case runtime, which can be applied to nonnegative vectors by numerically approximating the Chebyshev norm $\| \cdot \|_\infty…

Improved algorithm for low-discrepancy colorings with practical time complexity.

Proves weak convergence equals mean convergence in GGC.

This is an intuitive survey of extrinsic and intrinsic notions of convergence of manifolds complete with pictures of key examples and a discussion of the properties associated with each notion. We begin with a description of three extrinsic notions which have been applied to study sequences of submanifolds in Euclidean…

The abstract discusses convergence properties of Lipschitz functions and sets defined by equations.

Study shows intrinsic timed Hausdorff convergence leads to Gromov-Hausdorff and big bang convergence.

Studied SGD convergence under weak conditions.

Study on convergence rate of $Q$-curvature flow in 6 dimensions.