arXiv research

New method improves convergence for smooth games.

New algorithm reduces regret in graphical bilinear bandits.

Improved SEG method converges to Nash equilibrium in bilinear games.

We consider differentiable games where the goal is to find a Nash equilibrium. The machine learning community has recently started using variants of the gradient method (GD). Prime examples are extragradient (EG), the optimistic gradient method (OG) and consensus optimization (CO), which enjoy linear convergence in cas…

The extragradient method accelerates convergence in complex game dynamics.

We study a wide class of non-convex non-concave min-max games that generalizes over standard bilinear zero-sum games. In this class, players control the inputs of a smooth function whose output is being applied to a bilinear zero-sum game. This class of games is motivated by the indirect nature of the competition in Ge…

Min-max formulations have attracted great attention in the ML community due to the rise of deep generative models and adversarial methods, while understanding the dynamics of gradient algorithms for solving such formulations has remained a grand challenge. As a first step, we restrict to bilinear zero-sum games and giv…

Improved convergence rates for saddle-point optimization algorithms.

Improves training GANs by escaping limit cycles.

We use matrix iteration theory to characterize acceleration in smooth games. We define the spectral shape of a family of games as the set containing all eigenvalues of the Jacobians of standard gradient dynamics in the family. Shapes restricted to the real line represent well-understood classes of problems, like minimi…

Proposes CoPO, a new policy optimization method for competitive games.

Transforms game optimization dynamics into frequency domain for precise hyperparameter analysis.

Negative momentum accelerates convergence in minimax games but at a suboptimal rate.

Computing Nash equilibrium (NE) of multi-player games has witnessed renewed interest due to recent advances in generative adversarial networks. However, computing equilibrium efficiently is challenging. To this end, we introduce the Gradient-based Nikaido-Isoda (GNI) function which serves: (i) as a merit function, vani…

New framework recovers reward and rationality parameters from game behavior.

This work finds mixed equilibria in machine learning problems using measures and simultaneous gradient ascent-descent.

New ODE models show saddle-point optimization methods converge differently, with last-iterate convergence for OGDA.

OMWU shows last iterate convergence in convex-concave games.

We examine two different techniques for parameter averaging in GAN training. Moving Average (MA) computes the time-average of parameters, whereas Exponential Moving Average (EMA) computes an exponentially discounted sum. Whilst MA is known to lead to convergence in bilinear settings, we provide the -- to our knowledge …

Training generative adversarial networks (GANs) often suffers from cyclic behaviors of iterates. Based on a simple intuition that the direction of centripetal acceleration of an object moving in uniform circular motion is toward the center of the circle, we present the Simultaneous Centripetal Acceleration (SCA) method…

This paper presents a novel unifying framework of bilinear LSTMs that can represent and utilize the nonlinear interaction of the input features present in sequence datasets for achieving superior performance over a linear LSTM and yet not incur more parameters to be learned. To realize this, our unifying framework allo…

Generalization of twistor spinors to Kähler manifolds which are called Kählerian twistor spinors are considered. We find the differential equation satisfied by the bilinear forms of Kählerian twistor spinors. We show that the bilinear form equation reduces to Kählerian conformal Killing-Yano equation under special cond…

Algorithm identifies correct hypothesis from alternatives in bandit problems.

Owing to their connection with generative adversarial networks (GANs), saddle-point problems have recently attracted considerable interest in machine learning and beyond. By necessity, most theoretical guarantees revolve around convex-concave (or even linear) problems; however, making theoretical inroads towards effici…

Algorithm identifies bilinear dynamical systems from noisy data.

In this paper, we extend Su-Zhang's Cheeger-Mueller type theorem for symmetric bilinear torsions to manifolds with boundary in the case that the Riemannian metric and the non-degenerate symmetric bilinear form are of product structure near the boundary. Our result also extends Bruening-Ma's Cheeger-Mueller type theorem…

Bilinear MLPs offer a new way to interpret deep learning models without complex nonlinearities.

Identifies bilinear systems from a single trajectory with optimal sample complexity.

Generalizes Riemann's results on flat coordinates for non-symmetric bilinear forms.

The theory of harmonic symmetric bilinear forms on a Riemannian manifold is an analogue of the theory of harmonic exterior differential forms on this manifold. To show this, we must consider every symmetric bilinear form on a Riemannian manifold as a one-form with values in the cotangent bundle of this manifold. In thi…

Paper reduces sample complexity for bilinear systems identification to nearly constant.

This note provides a neat and enjoyable expansion and application of the magnificent Ordentlich-Cover theory of "universal portfolios." I generalize Cover's benchmark of the best constant-rebalanced portfolio (or 1-linear trading strategy) in hindsight by considering the best bilinear trading strategy determined in hin…

Enhances knot invariants using bilinear forms on vector spaces.

Constructs a bilinear form from a quasimorphism on symplectic manifold groups.

In this paper, we propose to employ a bank of modality-dedicated Convolutional Neural Networks (CNNs), fuse, train, and optimize them together for person classification tasks. A modality-dedicated CNN is used for each modality to extract modality-specific features. We demonstrate that, rather than spatial fusion at the…

We define a type of biquandle which is a generalization of symplectic quandles. We use the extra structure of these bilinear biquandles to define new knot and link invariants and give some examples.

A parsimonious model reduces over-parameterization in skewed matrix variate mixtures.

Study learns linear system dynamics from noisy bilinear data.

We use the Jones-Wenzl idempotents to construct a basis of Temperley-Lieb algebra TL_n. This allows a short calculation for a Gram determinant of Lickorish's bilinear form on the Temperley-Lieb algebra.

Non-bilinear observations make optimal control harder, showing non-convex costs and non-affine optimal controllers.

This thesis is concerned with the theory of invariant bilinear differential pairings on parabolic geometries. It introduces the concept formally with the help of the jet bundle formalism and provides a detailed analysis. More precisely, after introducing the most important notations and definitions, we first of all giv…

Unified bounds for sketched bilinear forms in machine learning and statistics.

BiN normalizes financial time-series for better forecasting.

Proposes a low-rank bilinear pooling model for link prediction in knowledge graphs.

Study dynamics of alternating minimization for bilinear regression under large system limits.

Vector-valued neural learning has emerged as a promising direction in deep learning recently. Traditionally, training data for neural networks (NNs) are formulated as a vector of scalars; however, its performance may not be optimal since associations among adjacent scalars are not modeled. In this paper, we propose a n…

In this paper the notion of an M-th order invariant bilinear differential pairing is introduced and a formal definition is given. If the manifold has an AHS structure, then various first order pairings are constructed. This yields a classification of all first order invariant bilinear differential pairings on homogeneo…

We are interested in approximation of a multivariate function $f(x_1,\dots,x_d)$ by linear combinations of products $u^1(x_1)\cdots u^d(x_d)$ of univariate functions $u^i(x_i)$, $i=1,\dots,d$. In the case $d=2$ it is a classical problem of bilinear approximation. In the case of approximation in the $L_2$ space the bili…