arXiv research

A new algorithm tackles bilevel optimization with multiple inner minima.

The paper challenges the belief that more inner iterations at test time improve performance in implicit deep learning.

The paper solves the Andreadakis problem for specific groups using inner automorphisms.

We study one extremal problem on the product of power of generalized inner radii of non-overlapping domains in $\mathbb{C}^{n}$.

In this paper, we mainly study eigenvalue problems of p-Laplacian on domains with an interior hole. Firstly we prove Faber-Krahn-type inequalities, and Cheng-type eigenvalue comparison theorems on manifolds. Secondly, we prove a comparison theorem for eigenvalues with inner Dirichlet and outer Neumann boundary in minim…

A new stochastic method tackles bi-level optimization problems in deep learning.

We present and discuss some open problems formulated by participants of the International Workshop "Knots, Braids, and Auto\-mor\-phism Groups" held in Novosibirsk, 2014. Problems are related to palindromic and commutator widths of groups; properties of Brunnian braids and two-colored braids, corresponding to an amalga…

New approach to bilevel optimization for machine learning using functional methods.

This work speeds up hyperparameter selection for non-smooth convex models using implicit differentiation.

In this paper, we develop a loop group description of harmonic maps $\mathcal{F}: M \rightarrow G/K$ ``of finite uniton type", from a Riemann surface $M$ into inner symmetric spaces of compact or non-compact type. This develops work of Uhlenbeck, Segal, and Burstall-Guest to non-compact inner symmetric spaces. To be mo…

We present the first provably sublinear time algorithm for approximate \emph{Maximum Inner Product Search} (MIPS). Our proposal is also the first hashing algorithm for searching with (un-normalized) inner product as the underlying similarity measure. Finding hashing schemes for MIPS was considered hard. We formally sho…

A new pruning method improves neural network efficiency and accuracy.

Study on combustion theory solutions, proving nondegeneracy and stability in limit.

The variance reduction class of algorithms including the representative ones, SVRG and SARAH, have well documented merits for empirical risk minimization problems. However, they require grid search to tune parameters (step size and the number of iterations per inner loop) for optimal performance. This work introduces `…

This paper explores the non-convex composition optimization in the form including inner and outer finite-sum functions with a large number of component functions. This problem arises in some important applications such as nonlinear embedding and reinforcement learning. Although existing approaches such as stochastic gr…

In this paper, we consider the convex and non-convex composition problem with the structure $\frac{1}{n}\sum\nolimits_{i = 1}^n {{F_i}( {G( x )} )}$, where $G( x )=\frac{1}{n}\sum\nolimits_{j = 1}^n {{G_j}( x )}$ is the inner function, and $F_i(\cdot)$ is the outer function. We explore the variance reduction based met…

Study higher rank inner products and their tilings to describe tori degenerations.

Paper improves adversarial training using a learned optimizer.

Characterizes quandles with abelian inner automorphisms.

New spectral functionals for Dirac operators with inner fluctuations computed.

A graph (digraph) $G=(V,E)$ with a set $T\subseteq V$ of terminals is called inner Eulerian if each nonterminal node $v$ has even degree (resp. the numbers of edges entering and leaving $v$ are equal). Cherkassky and Lovász showed that the maximum number of pairwise edge-disjoint $T$-paths in an inner Eulerian graph $G…

This paper constructs quandles with abelian inner automorphism groups from graphs, proving their homogeneity.

We consider the problem of designing locality sensitive hashes (LSH) for inner product similarity, and of the power of asymmetric hashes in this context. Shrivastava and Li argue that there is no symmetric LSH for the problem and propose an asymmetric LSH based on different mappings for query and database points. Howev…

Equivalent tests for SGD batch size selection found.

New algorithm for linear bandits tackles Optimal Transport problems.

Recently it was shown that the problem of Maximum Inner Product Search (MIPS) is efficient and it admits provably sub-linear hashing algorithms. Asymmetric transformations before hashing were the key in solving MIPS which was otherwise hard. In the prior work, the authors use asymmetric transformations which convert th…

Fewer data weight updates lead to faster convergence in machine learning models.

A new sliced IGW distance for Gromov-Wasserstein alignment.

Groups with specific properties have vanishing $\ell^2$-Betti numbers.

The author reviews his results on locally compact homogeneous spaces with inner metric, in particular, homogeneous manifolds with inner metric. The latter are isometric to homogeneous (sub-)Finslerian manifolds; under some additional conditions they are isometric to homogeneous (sub)-Riemannian manifolds. The class $Ω$…

Researchers prove inner product recovery is impossible in latent space models.

Minwise hashing (Minhash) is a widely popular indexing scheme in practice. Minhash is designed for estimating set resemblance and is known to be suboptimal in many applications where the desired measure is set overlap (i.e., inner product between binary vectors) or set containment. Minhash has inherent bias towards sma…

RSGDA improves convergence rates for nonconvex-strongly concave optimization.

In this paper, we determine the automorphism group of the $p$-cones ($p\neq 2$) in dimension greater than two. In particular, we show that the automorphism group of those $p$-cones are the positive scalar multiples of the generalized permutation matrices that fix the main axis of the cone. Next, we take a look at a pro…

This paper addresses the nearest neighbor search problem under inner product similarity and introduces a compact code-based approach. The idea is to approximate a vector using the composition of several elements selected from a source dictionary and to represent this vector by a short code composed of the indices of th…

Paper proposes a new method to optimize feature coordinates for better image classification.

We classify homotopes of classical symmetric spaces (studied in Part I of this work). Our classification uses the fibered structure of homotopes: they are fibered as symmetric spaces, with flat fibers, over a non-degenerate base; the base spaces correspond to inner ideals in Jordan pairs. Using that inner ideals in cla…

ES-Single uses ES to estimate gradients in unrolled graphs, reducing variance and improving performance.

Study of Gaussian distributions using entropic Gromov-Wasserstein and inner product Gromov-Wasserstein.

GPU-accelerates multiuser detection for 5G URLLC systems.

We study hamiltonian actions of compact groups in the presence of compatible involutions. We show that the lagrangian fixed point set on the symplectically reduced space is isomorphic to the disjoint union of the involutively reduced spaces corresponding to involutions on the group strongly inner to the given one. Our …

A core capability of intelligent systems is the ability to quickly learn new tasks by drawing on prior experience. Gradient (or optimization) based meta-learning has recently emerged as an effective approach for few-shot learning. In this formulation, meta-parameters are learned in the outer loop, while task-specific m…

New method solves complex optimization problems with real-time learning.

ANIL adapts only a subset of parameters, reducing computational cost.

Study bounds the index of minimal submanifolds using energy measures and Yang-Mills-Higgs equations.

We propose a quantization based approach for fast approximate Maximum Inner Product Search (MIPS). Each database vector is quantized in multiple subspaces via a set of codebooks, learned directly by minimizing the inner product quantization error. Then, the inner product of a query to a database vector is approximated …

Introduces PPMM algorithm for nonconvex robust regression problems.

New method solves complex constrained optimization problems.