arXiv research

A clustering algorithm for natural hierarchical clusters with near-linear time complexity.

CTT compresses samples to test distributions near-linearly, outperforming existing methods.

Computing optimal transport distances such as the earth mover's distance is a fundamental problem in machine learning, statistics, and computer vision. Despite the recent introduction of several algorithms with good empirical performance, it is unknown whether general optimal transport distances can be approximated in …

New algorithm approximates distributions with near-linear time and optimal sample efficiency.

Compress++ speeds up distribution compression to near-linear time.

Improved efficient robust regression with near-linear time and subquadratic samples.

We consider the fundamental learning problem of estimating properties of distributions over large domains. Using a novel piecewise-polynomial approximation technique, we derive the first unified methodology for constructing sample- and time-efficient estimators for all sufficiently smooth, symmetric and non-symmetric, …

In this work, we propose a robust approach to design distributed controllers for unknown-but-sparse linear and time-invariant systems. By leveraging modern techniques in distributed controller synthesis and structured linear inverse problems as applied to system identification, we show that near-optimal distributed con…

New framework for DP-SMO with near-optimal privacy-loss trade-off.

New algorithm trains neural networks in near-linear time, overcoming slow convergence issues.

Quantum algorithms reduce clustering input size, achieving near-linear approximation.

Gaussian processes (GP) are one of the most successful frameworks to model uncertainty. However, GP optimization (e.g., GP-UCB) suffers from major scalability issues. Experimental time grows linearly with the number of evaluations, unless candidates are selected in batches (e.g., using GP-BUCB) and evaluated in paralle…

New method for robust linear regression in nearly linear time.

This paper develops a fast algorithm for solving nonlinear PDEs using sparse Cholesky factorization.

New insights into variable selection with different model assumptions.

Local polynomial regression (Fan and Gijbels 1996) is an important class of methods for nonparametric density estimation and regression problems. However, straightforward implementation of local polynomial regression has quadratic time complexity which hinders its applicability in large-scale data analysis. In this pap…

WildCat efficiently compresses neural network attention mechanisms.

Paper proposes FedQ-Advantage for federated Q-learning with near-optimal regret and low communication cost.

Algorithm finds frequencies, amplitudes, and phases of sinusoids in noisy data.

New algorithm recovers sparse signals from linearly sparse dictionaries efficiently.

Direct proof shows adaptive gradient descent converges near-linearly for convex functions.

Matrix completion is the problem of recovering a low rank matrix by observing a small fraction of its entries. A series of recent works [KOM12,JNS13,HW14] have proposed fast non-convex optimization based iterative algorithms to solve this problem. However, the sample complexity in all these results is sub-optimal in it…

Implicit schemes are popular methods for the integration of time dependent PDEs such as hyperbolic and parabolic PDEs. However the necessity to solve corresponding linear systems at each time step constitutes a complexity bottleneck in their application to PDEs with rough coefficients. We present a generalization of ga…

Improved sketching for logistic and $\ell_1$ regression with near-linear dimensions.

A nonparametric method for time series analysis extracts envelopes, detects peaks, and clusters data.

This work generalizes transformer attention to capture higher-order correlations efficiently.

A new algorithm estimates mean adaptively to covariance, faster and more flexible than existing methods.

The profile of a sample is the multiset of its symbol frequencies. We show that for samples of discrete distributions, profile entropy is a fundamental measure unifying the concepts of estimation, inference, and compression. Specifically, profile entropy a) determines the speed of estimating the distribution relative t…

We propose a graph spectral representation of time series data that 1) is parsimoniously encoded to user-demanded resolution; 2) is unsupervised and performant in data-constrained scenarios; 3) captures event and event-transition structure within the time series; and 4) has near-linear computational complexity in both …

Finding the diameter of a dataset in multidimensional Euclidean space is a well-established problem, with well-known algorithms. However, most of the algorithms found in the literature do not scale well with large values of data dimension, so the time complexity grows exponentially in most cases, which makes these algo…

Computable Stein discrepancies have been deployed for a variety of applications, ranging from sampler selection in posterior inference to approximate Bayesian inference to goodness-of-fit testing. Existing convergence-determining Stein discrepancies admit strong theoretical guarantees but suffer from a computational co…

Although distributed computing can significantly reduce the training time of deep neural networks, scaling the training process while maintaining high efficiency and final accuracy is challenging. Distributed asynchronous training enjoys near-linear speedup, but asynchrony causes gradient staleness - the main difficult…

We propose a stochastic variance reduced optimization algorithm for solving sparse learning problems with cardinality constraints. Sufficient conditions are provided, under which the proposed algorithm enjoys strong linear convergence guarantees and optimal estimation accuracy in high dimensions. We further extend the …

Efficiently infers time-varying sparse MRFs with strong statistical guarantees.

SOR-Mamba improves Mamba for robust time series forecasting by minimizing channel order bias.

This work optimizes mean estimation under varying user privacy demands.

Causal structure learning has been a challenging task in the past decades and several mainstream approaches such as constraint- and score-based methods have been studied with theoretical guarantees. Recently, a new approach has transformed the combinatorial structure learning problem into a continuous one and then solv…

Near-logarithmic regret per switch achieved for mixable/exp-concave losses.

SAMBA predicts stock returns efficiently using Mamba and graph neural networks.

New method reduces summary points for datasets while maintaining quality.

CLASSIX is a fast and explainable clustering method that sorts data and merges groups.

Efficiently simulates the Heston model with large time steps using a novel method.

Capturing the dynamical properties of time series concisely as interpretable feature vectors can enable efficient clustering and classification for time-series applications across science and industry. Selecting an appropriate feature-based representation of time series for a given application can be achieved through s…

Efficiently generates models resistant to falsification.

We provide a computational complexity analysis for the Sinkhorn algorithm that solves the entropic regularized Unbalanced Optimal Transport (UOT) problem between two measures of possibly different masses with at most $n$ components. We show that the complexity of the Sinkhorn algorithm for finding an $\varepsilon$-appr…

We consider the problem of Robust PCA in the fully and partially observed settings. Without corruptions, this is the well-known matrix completion problem. From a statistical standpoint this problem has been recently well-studied, and conditions on when recovery is possible (how many observations do we need, how many co…

New method speeds up analysis of computer experiments.

A new model predicts discrete events with flexible, nonparametric baseline and excitation.