D-SPIDER-SFO solves nonconvex optimization problems faster on decentralized networks.
problem Finding a decentralized algorithm with similar convergence rate to SPIDER-SFO.
method Proposed D-SPIDER-SFO, a decentralized variant of SPIDER-SFO.
result Achieves a similar gradient computation cost to centralized SPIDER-SFO.
In this paper, we propose a new technique named \textit{Stochastic Path-Integrated Differential EstimatoR} (SPIDER), which can be used to track many deterministic quantities of interest with significantly reduced computational cost. We apply SPIDER to two tasks, namely the stochastic first-order and zeroth-order method…
Paper proposes a faster SPIDER-EM variant for large-scale nonconvex optimization.
problem High computational cost of EM algorithm in large-scale learning.
method Extension of SPIDER-EM for nonconvex finite-sum optimization problems.
result Achieves state-of-the-art complexity bounds and linear convergence under certain conditions.
A faster ADMM method for nonconvex optimization with improved complexity.
problem Nonconvex optimization problems in machine learning.
method SPIDER-ADMM, a stochastic ADMM method using a new differential estimator.
result Achieves optimal IFO complexity of O(n+n1/2ε−1) for finding an ε-approximate stationary point. Tripod spiders' energy control analyzed for Hooke and Coulomb potentials.
problem Control of tripod spiders' energy configurations.
method Morse theory for Hooke potential, stationary charges for Coulomb energy.
result For positive charges in a regular triangle, the domain of robust control is non-void.
A new EM algorithm improves inference from large datasets.
problem Efficient inference in latent variable models with large datasets.
method Introduces SPIDER-EM, a novel EM algorithm using SPIDER estimator.
result Finite-time complexity bounds for smooth non-convex likelihood.
Study bounds variance modulation function for K-spider distributions.
problem Bounding variance modulation function for K-spider distributions.
method Used folded moments and total probabilities of spider legs.
result Gave an interval for the variance modulation function.
Study spider mechanism configuration spaces using squared distance function.
problem Understand configuration spaces of spider mechanisms.
method Use Morse theory of squared distance function from body to fixed point.
result List and describe critical manifolds of squared distance function as products of polygon spaces.
The topology of SU(3)-representation varieties of the fundamental groups of planar webs so that the meridians are sent to matrices with trace equal to −1 are explored, and compared to data coming from spider evaluation of the webs. Corresponding to an evaluation of a web as a spider is a rooted tree. We associate t…
Spider category comparison proves equivalence to Sikora's quotient category.
problem Comparing skein theories of SLn. method Proved equivalence between spider category and Sikora's quotient category.
result Spider category Sp(SLn) is equivalent to Sikora's quotient category. Study transitions between tableau and spider bases for Specht modules.
problem Transitioning between tableau and spider bases for Specht modules.
method Combinatorial path model to study transitioning matrix from tableau basis to spider basis.
result Positive entries in the transitioning matrix for upper-triangular portion.
Improved zeroth-order algorithms for nonconvex optimization with reduced complexity and improved performance.
problem Designing efficient zeroth-order algorithms for nonconvex optimization with reduced function query complexities and improved convergence rates.
method Proposed new algorithms ZO-SVRG-Coord-Rand and ZO-SPIDER-Coord, developed new analyses, and addressed issues of function query complexities and stepsize generation.
result New algorithms outperform existing methods in terms of function query complexities and convergence rates.
Spider GAN accelerates GAN training with a new approach.
problem Stable training of Generative adversarial networks (GANs).
method Spider GAN leverages a novel approach to identify closely related datasets (friendly neighborhoods) and uses a new measure (signed inception distance) to accelerate GAN training.
result Spider GAN achieves faster convergence and state-of-the-art FID values with one-fifth of the training iterations.
A distributed algorithm reduces communication complexity for non-convex optimization.
problem Efficiently solving non-convex optimization problems in a distributed setting.
method Parallel Restarted SPIDER algorithm, incorporating SPIDER gradient estimator.
result Achieves optimal communication complexity O(ε−1) and optimal computation complexity. Given a point (the "spider") on a rectangular box, we would like to find the minimal distance along the surface to its opposite point (the "fly" - the reflection of the spider across the center of the box). Without loss of generality, we can assume that the box has dimensions 1×a×b with the spider on one …
SARAH and SPIDER are two recently developed stochastic variance-reduced algorithms, and SPIDER has been shown to achieve a near-optimal first-order oracle complexity in smooth nonconvex optimization. However, SPIDER uses an accuracy-dependent stepsize that slows down the convergence in practice, and cannot handle objec…
We show that the A2 clasps in the Karoubi envelope of A2 spider satisfy the recursive formula of the two-variable Chebyshev polynomials of the second kind associated with a root system of type A2. The A2 spider is a diagrammatic description of the representation category for Uq(sl3) and the $…
New scheme adapts batch size for faster variance-reduced algorithms.
problem Slowness of variance-reduced algorithms due to large batch size.
method Eliminates backtracking line search, adapts batch size via history stochastic gradients.
result Significantly reduces overall complexity for SVRG and SARAH/SPIDER.
Self-Organizing Maps (SOM) are popular unsupervised artificial neural network used to reduce dimensions and visualize data. Visual interpretation from Self-Organizing Maps (SOM) has been limited due to grid approach of data representation, which makes inter-scenario analysis impossible. The paper proposes a new way to …
System tackles indeterminacies in automated audio captioning.
problem Word selection and sentence length indeterminacies in automated audio captioning.
method Solves caption generation and sub-indeterminacy problems through multi-task learning to estimate keywords and sentence length.
result Model achieved 20.7 SPIDEr score, significantly outperforming baseline.
Improved optimization technique reduces training complexity for non-convex problems.
problem Training non-convex optimization problems with exploding gradients.
method Employed variance reduction technique (SPIDER) with carefully designed learning rate.
result Improved stochastic gradient complexity to O(ε−3) for ε-stationary solutions. Paper develops momentum schemes with variance reduction for non-convex composition optimization.
problem Lack of convergence guarantee and efficient momentum design in existing algorithms.
method Develops various momentum schemes with SPIDER-based variance reduction.
result Achieves near-optimal sample complexity and linear convergence rate.
Improved variance reduction for Riemannian non-convex optimization with adaptive batch size.
problem Optimizing non-convex functions on Riemannian manifolds.
method Batch size adaptation in R-SVRG, R-SRG, and R-SPIDER.
result Achieves lower total complexities for various non-convex functions.
SPIDER uses deep neural networks for streaming tensor factorization.
problem Lack of effective approach for deep tensor factorization of streaming data.
method Bayesian neural networks with spike-and-slab prior, Taylor expansions, moment matching, and EPI framework.
result Effective incremental updates for latent factors and NN weights.
We define and study the category of symmetric sl2-webs. This category is a combinatorial description of the category of all finite dimensional quantum sl2-modules. Explicitly, we show that (the additive closure of) the symmetric sl2-spider is (braided monoidally) equivalent to …
We study natural bases for two constructions of the irreducible representation of the symmetric group corresponding to [n,n,n]: the {\em reduced web} basis associated to Kuperberg's combinatorial description of the spider category; and the {\em left cell basis} for the left cell construction of Kazhdan and Lusztig. I…
We develop a theory of confluence of graphs. We describe an algorithm for proving that a given system of reduction rules for abstract graphs and graphs in surfaces is locally confluent. We apply this algorithm to show that each simple Lie algebra of rank at most 2, gives rise to a confluent system of reduction rules of…
Zeroth-order (a.k.a, derivative-free) methods are a class of effective optimization methods for solving complex machine learning problems, where gradients of the objective functions are not available or computationally prohibitive. Recently, although many zeroth-order methods have been developed, these approaches still…
Let G be a simple algebraic group. Labelled trivalent graphs called webs can be used to product invariants in tensor products of minuscule representations. For each web, we construct a configuration space of points in the affine Grassmannian. Via the geometric Satake correspondence, we relate these configuration spaces…
Paper proposes faster method to find local minima in nonconvex optimization.
problem Escaping saddle points and finding local minima in nonconvex optimization.
method LENA (Last stEp shriNkAge) framework for faster perturbed stochastic gradient methods.
result LENA finds (ε,εH)-approximate local minima within ildeO(ε−3+εH−6) evaluations. The sl_3 spider is a diagrammatic category used to study the representation theory of the quantum group U_q(sl_3). The morphisms in this category are generated by a basis of non-elliptic webs. Khovanov- Kuperberg observed that non-elliptic webs are indexed by semistandard Young tableaux. They establish this bijection v…
New method samples manifolds efficiently using Dirichlet distribution.
problem Sampling on complex manifolds efficiently.
method Data-driven Dirichlet sampling on manifolds.
result Efficient sampling respects manifold structure with low computational effort.
When translating natural language questions into SQL queries to answer questions from a database, we would like our methods to generalize to domains and database schemas outside of the training set. To handle complex questions and database schemas with a neural encoder-decoder paradigm, it is critical to properly encod…
We reconsider the su(3) link homology theory defined by Khovanov in math.QA/0304375 and generalized by Mackaay and Vaz in math.GT/0603307. With some slight modifications, we describe the theory as a map from the planar algebra of tangles to a planar algebra of (complexes of) `cobordisms with seams' (actually, a `canopo…
Freya PAGE optimizes nonconvex optimization with heterogeneous, asynchronous workers.
problem Optimizing nonconvex finite-sum problems with varying worker processing times.
method Freya PAGE, a parallel method robust to stragglers and adaptive to slow computations.
result Freya PAGE offers improved time complexity guarantees compared to previous methods.
Improves text-to-SQL models by selecting the best SQL query from beam output.
problem Simplifying database query writing for natural language questions.
method Discriminative re-ranker using BERT fine-tuned classifier.
result Achieved top 4 score on Spider leaderboard.
Regularization and data augmentation can be class-dependent, leading to poor performance on some classes.
problem Class-dependent effects of regularization and data augmentation.
method Evaluation of regularization and data augmentation techniques on Imagenet and INaturalist datasets.
result Regularization and data augmentation can lead to significant performance drops on some classes.
Improved analysis for nonconvex SGD methods with flexible sampling.
problem Finding approximately stationary points of nonconvex functions with gradient evaluations.
method Generalized SPIDER and PAGE algorithms with flexible sampling mechanisms.
result Sharper complexity bounds for optimal SGD methods in smooth nonconvex settings.
New estimators outperform maximum likelihood without hyper-parameter estimation.
problem Improving system identification performance without hyper-parameter estimation.
method Developed generalized Bayes and closed-form biased estimators using excess MSE.
result New estimators have comparable performance to empirical-Bayes-based regularized estimator.
New estimator reduces kernel mean estimation error.
problem Kernel mean estimation in reproducing kernel Hilbert spaces.
method Corrupt data with known distributions and estimate kernel mean under the corrupted distribution.
result The marginalized kernel mean estimator achieves lower estimation error.
Dual Bayesian Affine Estimators for Wiener-type state-space models
problem Estimating parameters in Wiener-type state-space models
method Fixed-point architecture combining two affine estimators
result Dual basis-parameter estimator achieves comparable parameter MSE to purely affine estimator
Enhances gradient estimates for Hermitian Monge-Ampère equations.
problem Improving estimates for Hermitian Monge-Ampère equations.
method Improves gradient estimates using Evans-Krylov and third derivatives estimates.
result Enhanced estimates for second and third order derivatives.
Paper proposes robust estimators for GANs under Wasserstein contamination.
problem Robust estimation of distributions under contamination.
method Wasserstein GAN-based estimators for location, covariance, and regression.
result Proposed estimators are minimax optimal in many scenarios.
New framework converts offline to online estimation using black-box offline estimators.
problem Convert offline estimation algorithms to online estimation algorithms.
method Oracle-Efficient Online Estimation (OEOE) framework.
result Achieves near-optimal online estimation error via black-box offline estimators.
Proposes variational autoencoder for efficient MMSE estimation.
problem Efficient parameterized MMSE estimation for noisy observations.
method Variational autoencoder models data distribution, approximates MMSE.
result Proposed estimator performs well compared to state-of-the-art.
Paper improves Fisher information estimation methods.
problem Estimating Fisher information for location parameters.
method Revisits and improves Bhattacharya estimator, introduces clipped estimator.
result Clipped estimator shows superior convergence rates in Gaussian noise.
Proposes a robust estimator for RD designs.
problem Estimating treatment effects in RD designs.
method Doubly robust estimator combining two estimators.
result Enhances robustness of treatment effect estimators.
New estimator reduces variance in discrete random variables.
problem Estimating gradients for discrete random variables with reduced variance.
method Sampling without replacement and Rao-Blackwellization.
result Our estimator is the most consistent gradient estimator across different entropy settings.