Decentralized optimization algorithms have attracted intensive interests recently, as it has a balanced communication pattern, especially when solving large-scale machine learning problems. Stochastic Path Integrated Differential Estimator Stochastic First-Order method (SPIDER-SFO) nearly achieves the algorithmic lower…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
38 results for “SPIDER”
Tripod spiders' energy control analyzed for Hooke and Coulomb potentials.
problem Control of tripod spiders' energy configurations.
method Morse theory for Hooke potential, stationary charges for Coulomb energy.
result For positive charges in a regular triangle, the domain of robust control is non-void.
SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path Integrated Differential Estimatormath.OC
In this paper, we propose a new technique named \textit{Stochastic Path-Integrated Differential EstimatoR} (SPIDER), which can be used to track many deterministic quantities of interest with significantly reduced computational cost. We apply SPIDER to two tasks, namely the stochastic first-order and zeroth-order method…
Paper proposes a faster SPIDER-EM variant for large-scale nonconvex optimization.
problem High computational cost of EM algorithm in large-scale learning.
method Extension of SPIDER-EM for nonconvex finite-sum optimization problems.
result Achieves state-of-the-art complexity bounds and linear convergence under certain conditions.
Study bounds variance modulation function for K-spider distributions.
problem Bounding variance modulation function for K-spider distributions.
method Used folded moments and total probabilities of spider legs.
result Gave an interval for the variance modulation function.
Study spider mechanism configuration spaces using squared distance function.
problem Understand configuration spaces of spider mechanisms.
method Use Morse theory of squared distance function from body to fixed point.
result List and describe critical manifolds of squared distance function as products of polygon spaces.
The topology of -representation varieties of the fundamental groups of planar webs so that the meridians are sent to matrices with trace equal to are explored, and compared to data coming from spider evaluation of the webs. Corresponding to an evaluation of a web as a spider is a rooted tree. We associate t…
A faster ADMM method for nonconvex optimization with improved complexity.
problem Nonconvex optimization problems in machine learning.
method SPIDER-ADMM, a stochastic ADMM method using a new differential estimator.
result Achieves optimal IFO complexity of for finding an -approximate stationary point.
Spider category comparison proves equivalence to Sikora's quotient category.
problem Comparing skein theories of .
method Proved equivalence between spider category and Sikora's quotient category.
result Spider category is equivalent to Sikora's quotient category.
Spider GAN accelerates GAN training with a new approach.
problem Stable training of Generative adversarial networks (GANs).
method Spider GAN leverages a novel approach to identify closely related datasets (friendly neighborhoods) and uses a new measure (signed inception distance) to accelerate GAN training.
result Spider GAN achieves faster convergence and state-of-the-art FID values with one-fifth of the training iterations.
Two types of zeroth-order stochastic algorithms have recently been designed for nonconvex optimization respectively based on the first-order techniques SVRG and SARAH/SPIDER. This paper addresses several important issues that are still open in these methods. First, all existing SVRG-type zeroth-order algorithms suffer …
Given a point (the "spider") on a rectangular box, we would like to find the minimal distance along the surface to its opposite point (the "fly" - the reflection of the spider across the center of the box). Without loss of generality, we can assume that the box has dimensions with the spider on one …
SARAH and SPIDER are two recently developed stochastic variance-reduced algorithms, and SPIDER has been shown to achieve a near-optimal first-order oracle complexity in smooth nonconvex optimization. However, SPIDER uses an accuracy-dependent stepsize that slows down the convergence in practice, and cannot handle objec…
We show that the clasps in the Karoubi envelope of spider satisfy the recursive formula of the two-variable Chebyshev polynomials of the second kind associated with a root system of type . The spider is a diagrammatic description of the representation category for and the $…
A new EM algorithm improves inference from large datasets.
problem Efficient inference in latent variable models with large datasets.
method Introduces SPIDER-EM, a novel EM algorithm using SPIDER estimator.
result Finite-time complexity bounds for smooth non-convex likelihood.
Self-Organizing Maps (SOM) are popular unsupervised artificial neural network used to reduce dimensions and visualize data. Visual interpretation from Self-Organizing Maps (SOM) has been limited due to grid approach of data representation, which makes inter-scenario analysis impossible. The paper proposes a new way to …
Improved optimization technique reduces training complexity for non-convex problems.
problem Training non-convex optimization problems with exploding gradients.
method Employed variance reduction technique (SPIDER) with carefully designed learning rate.
result Improved stochastic gradient complexity to for -stationary solutions.
Paper develops momentum schemes with variance reduction for non-convex composition optimization.
problem Lack of convergence guarantee and efficient momentum design in existing algorithms.
method Develops various momentum schemes with SPIDER-based variance reduction.
result Achieves near-optimal sample complexity and linear convergence rate.
Improved variance reduction for Riemannian non-convex optimization with adaptive batch size.
problem Optimizing non-convex functions on Riemannian manifolds.
method Batch size adaptation in R-SVRG, R-SRG, and R-SPIDER.
result Achieves lower total complexities for various non-convex functions.
Regarding the Specht modules associated to the two-row partition , we provide a combinatorial path model to study the transitioning matrix from the tableau basis to the -web basis (i.e. cup diagrams), and prove that the entries in this matrix are positive in the upper-triangular portion with respect to a ce…
SPIDER uses deep neural networks for streaming tensor factorization.
problem Lack of effective approach for deep tensor factorization of streaming data.
method Bayesian neural networks with spike-and-slab prior, Taylor expansions, moment matching, and EPI framework.
result Effective incremental updates for latent factors and NN weights.
In this paper, we propose a distributed algorithm for stochastic smooth, non-convex optimization. We assume a worker-server architecture where nodes, each having (potentially infinite) number of samples, collaborate with the help of a central server to perform the optimization task. The global objective is to m…
We define and study the category of symmetric -webs. This category is a combinatorial description of the category of all finite dimensional quantum -modules. Explicitly, we show that (the additive closure of) the symmetric -spider is (braided monoidally) equivalent to …
Confluence Theory for Graphsmath.QA
We develop a theory of confluence of graphs. We describe an algorithm for proving that a given system of reduction rules for abstract graphs and graphs in surfaces is locally confluent. We apply this algorithm to show that each simple Lie algebra of rank at most 2, gives rise to a confluent system of reduction rules of…
We study natural bases for two constructions of the irreducible representation of the symmetric group corresponding to : the {\em reduced web} basis associated to Kuperberg's combinatorial description of the spider category; and the {\em left cell basis} for the left cell construction of Kazhdan and Lusztig. I…
Let G be a simple algebraic group. Labelled trivalent graphs called webs can be used to product invariants in tensor products of minuscule representations. For each web, we construct a configuration space of points in the affine Grassmannian. Via the geometric Satake correspondence, we relate these configuration spaces…
Zeroth-order (a.k.a, derivative-free) methods are a class of effective optimization methods for solving complex machine learning problems, where gradients of the objective functions are not available or computationally prohibitive. Recently, although many zeroth-order methods have been developed, these approaches still…
The sl_3 spider is a diagrammatic category used to study the representation theory of the quantum group U_q(sl_3). The morphisms in this category are generated by a basis of non-elliptic webs. Khovanov- Kuperberg observed that non-elliptic webs are indexed by semistandard Young tableaux. They establish this bijection v…
System tackles indeterminacies in automated audio captioning.
problem Word selection and sentence length indeterminacies in automated audio captioning.
method Solves caption generation and sub-indeterminacy problems through multi-task learning to estimate keywords and sentence length.
result Model achieved 20.7 SPIDEr score, significantly outperforming baseline.
New method samples manifolds efficiently using Dirichlet distribution.
problem Sampling on complex manifolds efficiently.
method Data-driven Dirichlet sampling on manifolds.
result Efficient sampling respects manifold structure with low computational effort.
When translating natural language questions into SQL queries to answer questions from a database, we would like our methods to generalize to domains and database schemas outside of the training set. To handle complex questions and database schemas with a neural encoder-decoder paradigm, it is critical to properly encod…
We reconsider the su(3) link homology theory defined by Khovanov in math.QA/0304375 and generalized by Mackaay and Vaz in math.GT/0603307. With some slight modifications, we describe the theory as a map from the planar algebra of tangles to a planar algebra of (complexes of) `cobordisms with seams' (actually, a `canopo…
Variance-reduced algorithms, although achieve great theoretical performance, can run slowly in practice due to the periodic gradient estimation with a large batch of data. Batch-size adaptation thus arises as a promising approach to accelerate such algorithms. However, existing schemes either apply prescribed batch-siz…
Paper proposes faster method to find local minima in nonconvex optimization.
problem Escaping saddle points and finding local minima in nonconvex optimization.
method LENA (Last stEp shriNkAge) framework for faster perturbed stochastic gradient methods.
result LENA finds -approximate local minima within evaluations.
To access data stored in relational databases, users need to understand the database schema and write a query using a query language such as SQL. To simplify this task, text-to-SQL models attempt to translate a user's natural language question to corresponding SQL query. Recently, several generative text-to-SQL models …
Freya PAGE optimizes nonconvex optimization with heterogeneous, asynchronous workers.
problem Optimizing nonconvex finite-sum problems with varying worker processing times.
method Freya PAGE, a parallel method robust to stragglers and adaptive to slow computations.
result Freya PAGE offers improved time complexity guarantees compared to previous methods.
Regularization and data augmentation can be class-dependent, leading to poor performance on some classes.
problem Class-dependent effects of regularization and data augmentation.
method Evaluation of regularization and data augmentation techniques on Imagenet and INaturalist datasets.
result Regularization and data augmentation can lead to significant performance drops on some classes.
Improved analysis for nonconvex SGD methods with flexible sampling.
problem Finding approximately stationary points of nonconvex functions with gradient evaluations.
method Generalized SPIDER and PAGE algorithms with flexible sampling mechanisms.
result Sharper complexity bounds for optimal SGD methods in smooth nonconvex settings.