Study sharp convergence rates of empirical UOT for spatio-temporal point processes.
problem Statistical analysis of UOT for spatio-temporal point processes.
method Empirical plug-in estimators for Kantorovich-Rubinstein distance between intensity measures.
result Sharp convergence rates of empirical UOT in terms of intrinsic dimensions of measures.
New analysis improves SGD for robust and quantile regression with sub-quadratic convergence.
problem Improving SGD for robust and quantile regression with sub-quadratic convergence.
method Piecewise Lyapunov function for first-order differentiable functions.
result First geometrical convergence result for sub-quadratic SGD.
Study shows uniqueness of solutions on complex manifolds without requiring solution decay.
problem Uniqueness of solutions to Monge-Ampere equation on complex manifolds.
method Caccioppoli inequality techniques applied to Kähler manifolds with sub-quadratic volume growth.
result Uniqueness of bounded C1,1 solutions to Monge-Ampere equation without decay requirement. New findings on gradient expanding Ricci solitons with finite scalar curvature ratio.
problem Understanding the behavior of gradient expanding Ricci solitons with finite scalar curvature ratio.
method Analyzing complete gradient expanding Ricci solitons with nonnegative Ricci curvature.
result Riemann curvature tensor must have at least sub-quadratic decay for finite asymptotic scalar curvature ratio.
Adam converges to stationary points under relaxed conditions.
problem Understanding and proving convergence of Adam under realistic assumptions.
method New proof of boundedness of gradients and variance-reduced Adam.
result Adam converges to ε-stationary points with O(ε⁻⁴) gradient complexity under realistic conditions.
We present a new algorithm, trimed, for obtaining the medoid of a set, that is the element of the set which minimises the mean distance to all other elements. The algorithm is shown to have, under certain assumptions, expected run time O(N^(3/2)) in R^d where N is the set size, making it the first sub-quadratic exact m…
We consider complete non-compact manifolds with either a sub-quadratic growth of the norm of the Riemann curvature, or a sub-quadratic growth of both the norm of the Ricci curvature and the squared inverse of the injectivity radius. We show the existence on such a manifold of a distance-like function with bounded gradi…
The paper tests properties of trees in graphical models using covariance queries.
problem Testing properties of trees in graphical models.
method Covariance queries model, randomized tests for tree properties.
result Efficient testing of global tree properties using sub-quadratic number of queries.
Efficient sparse modern Hopfield models are introduced for memory retrieval and learning tasks.
problem Efficient modern Hopfield models for memory retrieval and learning tasks.
method Nonparametric interpretation of Hopfield models as a regression problem.
result Sparse-structured modern Hopfield models with sub-quadratic complexity.
The covariance matrix of a p-dimensional random variable is a fundamental quantity in data analysis. Given n i.i.d. observations, it is typically estimated by the sample covariance matrix, at a computational cost of O(np2) operations. When n,p are large, this computation may be prohibitively slow. Moreover, …
We study manifolds satisfying a weighed Poincare inequality, which was first introduced by Li-Wang. We generalized one of their results by relaxing the Ricci curvature bound condition only being satisfied outside a compact set and established a finitely many ends result. We proved a vanishing result for L2 harmonic …
Quadratic regression involves modeling the response as a (generalized) linear function of not only the features xj1 but also of quadratic terms xj1xj2. The inclusion of such higher-order "interaction terms" in regression often provides an easy way to increase accuracy in already-high-dimensional problem…
Improved MMD test for two-sample testing with random Fourier features.
problem Quadratic-time complexity of MMD test for large-scale analysis.
method Approximated MMD test using random Fourier features, investigating time-power trade-off.
result Sub-quadratic time complexity with same minimax separation rates as MMD test.
The so-called {\it kissing number} for hyperbolic surfaces is the maximum number of homotopically distinct systoles a surface of given genus g can have. These numbers, first studied (and named) by Schmutz Schaller by analogy with lattice sphere packings, are known to grow, as a function of genus, at least like $g^{\s…
Study calculates stable norm of slit tori using Farey sequence.
problem Computing the stable norm of slit tori.
method Explicit computations using the Farey sequence and gluing slit tori.
result Estimates the asymptotic counting of simple homology classes.
Semidefinite programs (SDP) are important in learning and combinatorial optimization with numerous applications. In pursuit of low-rank solutions and low complexity algorithms, we consider the Burer--Monteiro factorization approach for solving SDPs. We show that all approximate local optima are global optima for the pe…
We describe the first sub-quadratic sampling algorithm for the Multiplicative Attribute Graph Model (MAGM) of Kim and Leskovec (2010). We exploit the close connection between MAGM and the Kronecker Product Graph Model (KPGM) of Leskovec et al. (2010), and show that to sample a graph from a MAGM it suffices to sample sm…
Empirical risk minimization (ERM) is ubiquitous in machine learning and underlies most supervised learning methods. While there has been a large body of work on algorithms for various ERM problems, the exact computational complexity of ERM is still not understood. We address this issue for multiple popular ERM problems…
DBSCAN is a classical density-based clustering procedure with tremendous practical relevance. However, DBSCAN implicitly needs to compute the empirical density for each sample point, leading to a quadratic worst-case time complexity, which is too slow on large datasets. We propose DBSCAN++, a simple modification of DBS…
A known failing of many popular random graph models is that the Aldous-Hoover Theorem guarantees these graphs are dense with probability one; that is, the number of edges grows quadratically with the number of nodes. This behavior is considered unrealistic in observed graphs. We define a notion of edge exchangeability …
The family of temporal difference (TD) methods span a spectrum from computationally frugal linear methods like TD(λ) to data efficient least squares methods. Least square methods make the best use of available data directly computing the TD solution and thus do not require tuning a typically highly sensitive learning r…
We propose an original particle-based implementation of the Loopy Belief Propagation (LPB) algorithm for pairwise Markov Random Fields (MRF) on a continuous state space. The algorithm constructs adaptively efficient proposal distributions approximating the local beliefs at each note of the MRF. This is achieved by cons…
Sparse Gaussian processes with compact kernels for faster inference.
problem Efficient Gaussian process inference with high computational complexity.
method Parametric families of compactly-supported kernels for sparse matrix representations.
result Sub-quadratic inference complexity and improved performance on real-world tasks.
We analyze computational limits of modern Hopfield models based on pattern norms.
problem Understanding the efficiency of modern Hopfield models from a fine-grained complexity perspective.
method Fine-grained complexity analysis and upper bound criterion for pattern norms.
result Below a specific norm threshold, efficient variants of modern Hopfield models exist.
We consider support recovery in the quadratic logistic regression setting - where the target depends on both p linear terms xi and up to p2 quadratic terms xixj. Quadratic terms enable prediction/modeling of higher-order effects between features and the target, but when incorporated naively may involve solvi…
A new framework for efficient sequence maps using Bayesian filtering and covariance.
problem Designing efficient recurrent sequence maps from explicit memory assumptions.
method Design-model framework, exact Bayesian filtering, query-dependent readout, linear-Gaussian instantiation.
result Improved robustness and retrieval performance across various benchmarks.
New Performer model tackles long-sequence protein modeling.
problem Challenges of training complex Transformer models for long sequences.
method Linearly scalable long-context Transformer architecture, Performer.
result Performer provides strong theoretical guarantees and is effective for protein sequence modeling.
A common analytical problem in neuroscience is the interpretation of neural activity with respect to sensory input or behavioral output. This is typically achieved by regressing measured neural activity against known stimuli or behavioral variables to produce a "tuning function" for each neuron. Unfortunately, because …
Inversion-free natural gradient method for Riemannian manifolds.
problem Hindered by the need for Euclidean space, Fisher information matrix inversion, and computational cost.
method Intrinsic, inversion-free natural gradient method on Riemannian manifolds, using moving approximation of inverse FIM.
result Almost-sure convergence rates and sub-quadratic storage complexity for large-scale applications.
New algorithm reduces runtime for robust sparse mean estimation.
problem Efficiently estimating mean from corrupted data with sparse constraints.
method Subquadratic time algorithm using poly(k, log d, 1/ε) samples.
result First subquadratic time algorithm for robust sparse mean estimation.
New compression methods handle biased input sequences for more accurate posterior summaries.
problem Handling biased input sequences for accurate posterior summaries.
method Stein kernel thinning, low-rank SKT, Stein recombination, Stein Cholesky.
result Achieves accurate posterior summaries with biased input sequences.
Single-head transformers with a single self-attention layer can approximate any sequence-to-sequence function and are efficient under certain conditions.
problem Statistical and computational limits of prompt tuning for transformer-based models.
method Investigation of single-head transformers with a single self-attention layer, proving universality and efficiency under SETH.
result Existence of almost-linear time prompt tuning inference algorithms under certain conditions.
QATS efficiently decodes HMMs with polylogarithmic complexity.
problem Efficiently decoding hidden Markov models from noisy observations.
method Divide-and-conquer procedure with polylogarithmic sequence complexity and cubic state space complexity.
result QATS outperforms Viterbi and PMAP in speed and accuracy.
Prototype selection improved using topological data analysis.
problem Improving prototype selection methods for data compression.
method Introducing two topological prototype selector variants: TPS and BoundaryTPS.
result BoundaryTPS achieves the lowest mean Friedman rank on H1 persistence-diagram preservation. Empirical study finds variance swap rate is affine in spot variance for S&P500 data.
problem Investigating the relationship between variance swap rate and spot variance.
method Empirical analysis using S&P500 data from 2006-2018, testing different models.
result Affine relationship between variance swap rate and spot variance is supported.
This paper tackles variance issues in GNN training by proposing a method to reduce both embedding and gradient variances.
problem High variance in estimating stochastic gradients in GNN training, especially in large graphs.
method The paper proposes a decoupled variance reduction strategy that employs approximate gradient information to adaptively sample nodes with minimal variance.
result The proposed method achieves faster convergence and better generalization compared to existing sampling methods.
Study shows gradient variance increases during deep learning training, contrary to common belief.
problem Understanding and minimizing gradient variance in deep learning models.
method Gradient Clustering method using stratified sampling to minimize gradient variance.
result Gradient variance increases during training, and smaller learning rates coincide with higher variance.
Paper solves a control problem with robust methods.
problem Monotone mean-variance problems with stochastic coefficients.
method Finding saddle point through BSDEs with unbounded coefficients.
result Optimal control and value match mean-variance problems.
The paper analyzes the bias-variance tradeoff for Bregman divergences.
problem Understanding the bias-variance tradeoff for Bregman divergences.
method Analyzes the bias-variance tradeoff through operations in dual space.
result Derives several results including a generalized law of total variance and ensembling operations.
Mamba struggles with long context lengths, but spectrum scaling improves performance.
problem Mamba's performance degrades with increasing context length.
method Spectrum scaling applied to pre-trained Mamba models to improve long-context generalization.
result Spectrum scaling significantly improves performance in long-context settings.
Efficiently accelerates attention calculation for Transformers with relative positional encoding.
problem Quadratic complexity of attention in long sequences.
method Kernelized attention with Fast Fourier Transform (FFT) for RPE.
result Achieves O(n log n) time complexity, mitigates training instability, and outperforms other models.
Normal distributions ensure asymptotic variance reduction in moment matching Monte Carlo.
problem Asymptotic variance reduction in general integration problems.
method Characterization of conditions for asymptotic variance reduction using normal distributions.
result Asymptotic variance reduction is guaranteed for normal distributions in moment matching Monte Carlo.
A new statistical concept, lepto-variance, is defined for stock returns using Regression Trees.
problem Understanding the underlying structure of stock returns using statistical methods.
method Defining lepto-variance as the variance that cannot be removed by any regression tree of a specific depth and analyzing stock returns with 1- and 2-bit Regression Trees.
result Lepto-variance quantifies the resolving power of Regression Trees for stock returns, decomposing total variance into lepto-variance and macro-variance.
Paper tackles unknown variances in best-arm identification.
problem Identifying the best arm with unknown variances in Gaussian distributions.
method Two approaches: empirical variance plugging or adapting transportation costs.
result The impact of unknown variances is small on sample complexity.
New algorithms improve best-arm identification with varying rewards.
problem Identifying the best arm with varying reward variances in fixed budget.
method Proposed two algorithms: SHVar for known variances, SHAdaVar for unknown variances; uses non-uniform budget allocation.
result Bounding misidentification probabilities for both algorithms.
Before training a neural net, a classic rule of thumb is to randomly initialize the weights so the variance of activations is preserved across layers. This is traditionally interpreted using the total variance due to randomness in both weights \emph{and} samples. Alternatively, one can interpret the rule of thumb as pr…
A new measure k-variance captures local distributional shape.
problem Summarizing distributional shape with local information.
method Random bipartite matchings and stochastic approximation.
result Easily approximated k-variance measures capture local distributional properties. In the continuous time mean-variance model, we want to minimize the variance (risk) of the investment portfolio with a given mean at terminal time. However, the investor can stop the investment plan at any time before the terminal time. To solve this kind of problem, we consider to minimize the variances of the investm…