arXiv research

In this paper, we propose a simple variant of the original SVRG, called variance reduced stochastic gradient descent (VR-SGD). Unlike the choices of snapshot and starting points in SVRG and its proximal variant, Prox-SVRG, the two vectors of VR-SGD are set to the average and last iterate of the previous epoch, respecti…

We prove contractibility of VR complexes for integer lattices up to dimension 5.

In this paper, we propose a simple variant of the original stochastic variance reduction gradient (SVRG), where hereafter we refer to as the variance reduced stochastic gradient descent (VR-SGD). Different from the choices of the snapshot point and starting point in SVRG and its proximal variant, Prox-SVRG, the two vec…

New method reduces density estimation variance for multivariate data.

VR methods improve SGD for faster machine learning.

Given a sample of points $X$ in a metric space $M$ and a scale $r>0$, the Vietoris-Rips simplicial complex $\mathrm{VR}(X;r)$ is a standard construction to attempt to recover $M$ from $X$ up to homotopy type. A deficiency of this approach is that $\mathrm{VR}(X;r)$ is not metrizable if it is not locally finite, and thu…

Researchers identify valid auxiliary functions for extreme value distributions and their max-domains of attraction.

Paper formalizes and analyzes a new bound for variational inference.

Deep learning predicts VR head movements for better 360-degree videos.

Variance reduction (VR) methods boost the performance of stochastic gradient descent (SGD) by enabling the use of larger, constant stepsizes and preserving linear convergence rates. However, current variance reduced SGD methods require either high memory usage or an exact gradient computation (using the entire dataset)…

VR game data for P300 BCI with raccoon vs demon stimuli.

Researchers decompose Forman-Ricci curvature for efficient computation in VR complexes.

Statistical characteristics of deep network representations, such as sparsity and correlation, are known to be relevant to the performance and interpretability of deep learning. When a statistical characteristic is desired, often an adequate regularizer can be designed and applied during the training phase. Typically, …

Paper improves Bayesian inference in federated learning with new algorithm VR-FALD*.

This paper compares gradient estimators in importance-weighted VI and justifies the superiority of DREP over REP.

AdaSVRG combines adaptive gradient with SVRG for robust optimization.

We inspect Vietoris-Rips complexes $VR_t(X)$ of certain metric spaces $X$ using a new generalization of Bestvina-Brady discrete Morse theory. Our main result is a pair of metric criteria on $X$, called the Morse Criterion and Link Criterion, that allow us to deduce information about the homotopy types of certain $VR_t(…

VR-ConfTr reduces noise in CP training, leading to more stable and efficient model performance.

This paper introduces the variational Rényi bound (VR) that extends traditional variational inference to Rényi's alpha-divergences. This new family of variational methods unifies a number of existing approaches, and enables a smooth interpolation from the evidence lower-bound to the log (marginal) likelihood that is co…

Stochastic variance reduction algorithms have recently become popular for minimizing the average of a large, but finite number of loss functions. The present paper proposes a Riemannian stochastic quasi-Newton algorithm with variance reduction (R-SQN-VR). The key challenges of averaging, adding, and subtracting multipl…

New homology theory for metric spaces, proving stability and anticipating topological changes.

Homotopy equivalence shown between complex and thickened versions of manifolds.

Improved sampling accuracy in SG-MCMC methods via non-uniform gradient subsampling.

Virtual reality (VR) offers immersive visualization and intuitive interaction. We leverage VR to enable any biomedical professional to deploy a deep learning (DL) model for image classification. While DL models can be powerful tools for data analysis, they are also challenging to understand and develop. To make deep le…

Study finds super-efficiency correlates more strongly with stock market valuation than ROA in Chinese banks.

We present a large scale data set, OpenEDS: Open Eye Dataset, of eye-images captured using a virtual-reality (VR) head mounted display mounted with two synchronized eyefacing cameras at a frame rate of 200 Hz under controlled illumination. This dataset is compiled from video capture of the eye-region collected from 152…

New privacy method for eye tracking data reduces correlations and maintains accuracy.

Improved inference for models with continuous latent variables.

We study the convergence properties of the VR-PCA algorithm introduced by \cite{shamir2015stochastic} for fast computation of leading singular vectors. We prove several new results, including a formal analysis of a block version of the algorithm, and convergence from random initialization. We also make a few observatio…

We describe and analyze a simple algorithm for principal component analysis and singular value decomposition, VR-PCA, which uses computationally cheap stochastic iterations, yet converges exponentially fast to the optimal solution. In contrast, existing algorithms suffer either from slow convergence, or computationally…

Generative modeling of 3D shapes has become an important problem due to its relevance to many applications across Computer Vision, Graphics, and VR. In this paper we build upon recently introduced 3D mesh-convolutional Variational AutoEncoders which have shown great promise for learning rich representations of deformab…

VR-GHAL method solves stochastic fixed-point equations with high probability.

New method identifies whether equity return predictability is due to magnitude shrinkage or directional reversal.

We propose an L-BFGS optimization algorithm on Riemannian manifolds using minibatched stochastic variance reduction techniques for fast convergence with constant step sizes, without resorting to linesearch methods designed to satisfy Wolfe conditions. We provide a new convergence proof for strongly convex functions wit…

Hypernom is a virtual reality game. The cells of a regular 4D polytope are radially projected to S^3, the sphere in 4D space, then stereographically projected to 3D space where they are viewed in the headset. The orientation of the headset is given by an element of the group SO(3), which is also a space that is double …

This paper introduces persistent equivariant cohomology and applies it to circle actions.

Study analyzes profitability and efficiency of Chinese banks, finding state-owned banks superior.

We study the stochastic Riemannian gradient algorithm for matrix eigen-decomposition. The state-of-the-art stochastic Riemannian algorithm requires the learning rate to decay to zero and thus suffers from slow convergence and sub-optimal solutions. In this paper, we address this issue by deploying the variance reductio…

With the large rising of complex data, the nonconvex models such as nonconvex loss function and nonconvex regularizer are widely used in machine learning and pattern recognition. In this paper, we propose a class of mini-batch stochastic ADMMs (alternating direction method of multipliers) for solving large-scale noncon…

The paper proposes an AI and IIoT framework for improved maintenance.

New PG methods tackle nonconvex optimization with auto-conditioned stepsizes.

The study finds variations in ownership structure and efficiency across sectors in Malaysia.

Deep learning framework detects emotions from EEG data.

This research improves EEG-based MI-BCI systems to be more resilient to emotional arousal.

Prediction markets can shape political behavior through persistent signals, not just forecast accuracy.

We study the cross-correlation matrix $C_{ij}$ of inventory variations of the most active individual and institutional investors in an emerging market to understand the dynamics of inventory variations. We find that the distribution of cross-correlation coefficient $C_{ij}$ has a power-law form in the bulk followed by …

Graph convolutional network (GCN) has been successfully applied to many graph-based applications; however, training a large-scale GCN remains challenging. Current SGD-based algorithms suffer from either a high computational cost that exponentially grows with number of GCN layers, or a large space requirement for keepin…