arXiv research

Near-optimal algorithms for mean estimation and linear regression with Gaussian covariates and Huber contamination.

Linformer reduces transformer complexity to linear, improving efficiency.

A clustering algorithm for natural hierarchical clusters with near-linear time complexity.

New method for robust linear regression in nearly linear time.

In this work, we propose a robust approach to design distributed controllers for unknown-but-sparse linear and time-invariant systems. By leveraging modern techniques in distributed controller synthesis and structured linear inverse problems as applied to system identification, we show that near-optimal distributed con…

New Spencer complexes for Lie groupoids developed.

This paper analyzes error in SKI for Gaussian Processes, providing conditions for linear time inference.

DTW calculates the similarity or alignment between two signals, subject to temporal warping. However, its computational complexity grows exponentially with the number of time-series. Although there have been algorithms developed that are linear in the number of time-series, they are generally quadratic in time-series l…

Efficiently completes low-rank matrices with nearly linear time complexity.

New method speeds up diffusion models inference to sub-linear time.

ALT improves TSC by capturing complex patterns in time series data.

Ginger efficiently approximates curvature with linear complexity for neural networks.

Paper studies linear CMDPs, improving sample complexity for both models.

Skeinformer accelerates self-attention for long sequences with linear complexity.

Linear time algorithm for random walk kernels on sparse graphs.

Linear autoregressive models serve as basic representations of discrete time stochastic processes. Different attempts have been made to provide non-linear versions of the basic autoregressive process, including different versions based on kernel methods. Motivated by the powerful framework of Hilbert space embeddings o…

Local polynomial regression (Fan and Gijbels 1996) is an important class of methods for nonparametric density estimation and regression problems. However, straightforward implementation of local polynomial regression has quadratic time complexity which hinders its applicability in large-scale data analysis. In this pap…

New method discovers causal relationships in complex time series data.

An important problem in fiber-optic communications is to invert the nonlinear Schrödinger equation in real time to reverse the deterministic effects of the channel. Interestingly, the popular split-step Fourier method (SSFM) leads to a computation graph that is reminiscent of a deep neural network. This observation all…

Study shows long-term solutions for complex equations on curved spaces.

The OLS estimator optimally identifies stable linear systems with a finite number of samples.

Noise reduction is an important part of modern hearing aids and is included in most commercially available devices. Deep learning-based state-of-the-art algorithms, however, either do not consider real-time and frequency resolution constrains or result in poor quality under very noisy conditions. To improve monaural sp…

Paper proposes an efficient causal discovery method with linear computational complexity.

Wavelet SGM accelerates generative modeling with linear time complexity.

We show that the genus problem for alternating knots with $n$ crossings has linear time complexity and is in Logspace$(n)$. Almost all alternating knots of given genus possess additional combinatorial structure, we call them standard. We show that the genus problem for these knots belongs to $TC^0$ circuit complexity c…

Implicit schemes are popular methods for the integration of time dependent PDEs such as hyperbolic and parabolic PDEs. However the necessity to solve corresponding linear systems at each time step constitutes a complexity bottleneck in their application to PDEs with rough coefficients. We present a generalization of ga…

Study shows efficient algorithms for noiseless linear regression require quadratic sample complexity in contamination rate.

We study the problem of estimating low-rank matrices from linear measurements (a.k.a., matrix sensing) through nonconvex optimization. We propose an efficient stochastic variance reduced gradient descent algorithm to solve a nonconvex optimization problem of matrix sensing. Our algorithm is applicable to both noisy and…

E$^4$ algorithm optimizes batched linear bandits with minimal regret and batches.

Motivated by their broad applications in reinforcement learning, we study the linear two-time-scale stochastic approximation, an iterative method using two different step sizes for finding the solutions of a system of two equations. Our main focus is to characterize the finite-time complexity of this method under time-…

Transformers with linear space and time complexity for accurate attention estimation.

Polynomial-time reachability for LTI systems with TLL NN controllers is achieved.

We present a simple, general technique for reducing the sample complexity of matrix and tensor decomposition algorithms applied to distributions. We use the technique to give a polynomial-time algorithm for standard ICA with sample complexity nearly linear in the dimension, thereby improving substantially on previous b…

We prove that stochastic gradient descent efficiently converges to the global optimizer of the maximum likelihood objective of an unknown linear time-invariant dynamical system from a sequence of noisy observations generated by the system. Even though the objective function is non-convex, we provide polynomial running …

Develops an efficient method for real-time data analysis and visualization.

Generalized Linear Bandits (GLBs), a natural extension of the stochastic linear bandits, has been popular and successful in recent years. However, existing GLBs scale poorly with the number of rounds and the number of arms, limiting their utility in practice. This paper proposes new, scalable solutions to the GLB probl…

LITE efficiently estimates Gaussian PoM with linear time and memory complexity.

Optimal unimodal fitting for linear loss functions in a sequential, efficient manner.

New RL algorithm reduces deployment cost for linear function approximations.

Paper models non-linear dynamics from time series data.

SASE improves attributed graph clustering for large graphs with linear time and space complexity.

PureTS uses simple linear models to improve long-term time series forecasting.

Based on conservation laws for surface layer integrals for critical points of causal variational principles, it is shown how jet spaces can be endowed with an almost-complex structure. We analyze under which conditions the almost-complex structure can be integrated to a canonical complex structure. Combined with the sc…

This technical note extends recent results on the computational complexity of globally minimizing the error of piecewise-affine models to the related problem of minimizing the error of switching linear regression models. In particular, we show that, on the one hand the problem is NP-hard, but on the other hand, it admi…

Paper introduces a new method for Transformers with linear complexity.

First proper learning algorithm for Gaussian halfspaces with matching sample and computational complexity.

A new algorithm splits Gaussian processes for efficient streaming data.

New Gibbs sampling reduces GLMB filtering complexity to linear time.