arXiv research

IntHT solves sparse quadratic regression in sub-quadratic time and space.

New analysis improves SGD for robust and quantile regression with sub-quadratic convergence.

We present a new algorithm, trimed, for obtaining the medoid of a set, that is the element of the set which minimises the mean distance to all other elements. The algorithm is shown to have, under certain assumptions, expected run time O(N^(3/2)) in R^d where N is the set size, making it the first sub-quadratic exact m…

Study shows uniqueness of solutions on complex manifolds without requiring solution decay.

The covariance matrix of a $p$-dimensional random variable is a fundamental quantity in data analysis. Given $n$ i.i.d. observations, it is typically estimated by the sample covariance matrix, at a computational cost of $O(np^{2})$ operations. When $n,p$ are large, this computation may be prohibitively slow. Moreover, …

New findings on gradient expanding Ricci solitons with finite scalar curvature ratio.

Study sharp convergence rates of empirical UOT for spatio-temporal point processes.

Improved MMD test for two-sample testing with random Fourier features.

We consider complete non-compact manifolds with either a sub-quadratic growth of the norm of the Riemann curvature, or a sub-quadratic growth of both the norm of the Ricci curvature and the squared inverse of the injectivity radius. We show the existence on such a manifold of a distance-like function with bounded gradi…

The paper tests properties of trees in graphical models using covariance queries.

Efficient sparse modern Hopfield models are introduced for memory retrieval and learning tasks.

Empirical risk minimization (ERM) is ubiquitous in machine learning and underlies most supervised learning methods. While there has been a large body of work on algorithms for various ERM problems, the exact computational complexity of ERM is still not understood. We address this issue for multiple popular ERM problems…

We study manifolds satisfying a weighed Poincare inequality, which was first introduced by Li-Wang. We generalized one of their results by relaxing the Ricci curvature bound condition only being satisfied outside a compact set and established a finitely many ends result. We proved a vanishing result for $L^2$ harmonic …

We describe the first sub-quadratic sampling algorithm for the Multiplicative Attribute Graph Model (MAGM) of Kim and Leskovec (2010). We exploit the close connection between MAGM and the Kronecker Product Graph Model (KPGM) of Leskovec et al. (2010), and show that to sample a graph from a MAGM it suffices to sample sm…

DBSCAN is a classical density-based clustering procedure with tremendous practical relevance. However, DBSCAN implicitly needs to compute the empirical density for each sample point, leading to a quadratic worst-case time complexity, which is too slow on large datasets. We propose DBSCAN++, a simple modification of DBS…

We analyze computational limits of modern Hopfield models based on pattern norms.

We consider support recovery in the quadratic logistic regression setting - where the target depends on both p linear terms $x_i$ and up to $p^2$ quadratic terms $x_i x_j$. Quadratic terms enable prediction/modeling of higher-order effects between features and the target, but when incorporated naively may involve solvi…

The so-called {\it kissing number} for hyperbolic surfaces is the maximum number of homotopically distinct systoles a surface of given genus $g$ can have. These numbers, first studied (and named) by Schmutz Schaller by analogy with lattice sphere packings, are known to grow, as a function of genus, at least like $g^{\s…

New algorithm reduces runtime for robust sparse mean estimation.

Study calculates stable norm of slit tori using Farey sequence.

Semidefinite programs (SDP) are important in learning and combinatorial optimization with numerous applications. In pursuit of low-rank solutions and low complexity algorithms, we consider the Burer--Monteiro factorization approach for solving SDPs. We show that all approximate local optima are global optima for the pe…

Single-head transformers with a single self-attention layer can approximate any sequence-to-sequence function and are efficient under certain conditions.

New compression methods handle biased input sequences for more accurate posterior summaries.

A common analytical problem in neuroscience is the interpretation of neural activity with respect to sensory input or behavioral output. This is typically achieved by regressing measured neural activity against known stimuli or behavioral variables to produce a "tuning function" for each neuron. Unfortunately, because …

A known failing of many popular random graph models is that the Aldous-Hoover Theorem guarantees these graphs are dense with probability one; that is, the number of edges grows quadratically with the number of nodes. This behavior is considered unrealistic in observed graphs. We define a notion of edge exchangeability …

The family of temporal difference (TD) methods span a spectrum from computationally frugal linear methods like TD(λ) to data efficient least squares methods. Least square methods make the best use of available data directly computing the TD solution and thus do not require tuning a typically highly sensitive learning r…

We propose an original particle-based implementation of the Loopy Belief Propagation (LPB) algorithm for pairwise Markov Random Fields (MRF) on a continuous state space. The algorithm constructs adaptively efficient proposal distributions approximating the local beliefs at each note of the MRF. This is achieved by cons…

Improved algorithm for low-discrepancy colorings with practical time complexity.

Sparse Gaussian processes with compact kernels for faster inference.

Adam converges to stationary points under relaxed conditions.

Mamba struggles with long context lengths, but spectrum scaling improves performance.

Efficiently accelerates attention calculation for Transformers with relative positional encoding.

We analyze the computational limits of LoRA for transformer models using fine-grained complexity theory.

A new framework for efficient sequence maps using Bayesian filtering and covariance.

New Performer model tackles long-sequence protein modeling.

Inversion-free natural gradient method for Riemannian manifolds.

SGD with constant stepsize converges to a non-Gaussian limit near flat minima.

QATS efficiently decodes HMMs with polylogarithmic complexity.

Prototype selection improved using topological data analysis.

Max-convolution is an important problem closely resembling standard convolution; as such, max-convolution occurs frequently across many fields. Here we extend the method with fastest known worst-case runtime, which can be applied to nonnegative vectors by numerically approximating the Chebyshev norm $\| \cdot \|_\infty…

Most of machine learning approaches have stemmed from the application of minimizing the mean squared distance principle, based on the computationally efficient quadratic optimization methods. However, when faced with high-dimensional and noisy data, the quadratic error functionals demonstrated many weaknesses including…

We consider the problem of recovering a signal $\mathbf{x}^* \in \mathbf{R}^n$, from magnitude-only measurements $y_i = |\left\langle\mathbf{a}_i,\mathbf{x}^*\right\rangle|$ for $i=[m]$. Also called the phase retrieval, this is a fundamental challenge in bio-,astronomical imaging and speech processing. The problem abov…

Analyzes intrinsic time in financial markets, linking it to physical time.

New continuous-time optimization algorithms converge in finite time to local minima.

Consider power utility maximization of terminal wealth in a 1-dimensional continuous-time exponential Levy model with finite time horizon. We discretize the model by restricting portfolio adjustments to an equidistant discrete time grid. Under minimal assumptions we prove convergence of the optimal discrete-time strate…

New distances defined between space-times, proving some definite.

Proposes a method to allocate time budgets in mixed criticality systems.

Paper analyzes venture capital exit decisions under inconsistent preferences.