arXiv research

The paper analyzes deep neural networks' expressivity and training, revealing critical expressivity issues.

Study reveals different types of critical points in shallow neural networks.

The paper connects reflection groups to maps with specific dynamical properties.

We reformulate the option framework as two parallel augmented MDPs. Under this novel formulation, all policy optimization algorithms can be used off the shelf to learn intra-option policies, option termination conditions, and a master policy over options. We apply an actor-critic algorithm on each augmented MDP, yieldi…

QAM uses adjoint matching to optimize continuous-action RL policies efficiently.

Formula for critical points of chi fields on manifolds.

ADAC uses analogous policies to improve RL exploration without sacrificing stability.

Study reveals Transformer's expressive power and mechanisms.

Using the method of Witten deformation, we express the basic index of a transversal Dirac operator over a Riemannian foliation as the sum of integers associated to the critical leaf closures of a given foliated bundle map.

This work explores the relationship between expressivity and generalization in GNNs.

A new aggregation strategy improves GNN performance and learning dynamics.

Given any n-tuple of complex numbers, one can canonically define a polynomial of degree n+1 that has the entries of this n-tuple as its critical points. In 2002, Beardon, Carne, and Ng studied a map $θ\colon \mathbb{C}^n\to \mathbb{C}^n$ which outputs the critical values of the canonical polynomial constructed from the…

New insights into matrix factorization show strict saddles have bounded eigenvalues.

When agents interact with a complex environment, they must form and maintain beliefs about the relevant aspects of that environment. We propose a way to efficiently train expressive generative models in complex environments. We show that a predictive algorithm with an expressive generative model can form stable belief-…

Any compact manifold with positive scalar curvature has an associated asymptotically flat metric constructed using the Green's function of the conformal Laplacian, and the mass of this metric is an important geometric invariant. An explicit expression for the mass of the product of spheres $S^2 \times S^2$, both with t…

Study on Gaussian random fields' singularities on manifolds.

Deep Reinforcement Learning (DRL) algorithms for continuous action spaces are known to be brittle toward hyperparameters as well as \cut{being}sample inefficient. Soft Actor Critic (SAC) proposes an off-policy deep actor critic algorithm within the maximum entropy RL framework which offers greater stability and empiric…

Given a knot K in an Euclidean space E and a finite dimensional space V of smooth functions on K, we express the expected number of critical points of a random function in V in terms of an integral-geometric invariant of K and V. When V consists of the restrictions to K of homogeneous polynomials of degree d on E, this…

CoRMF uses RNNs to solve Ising models efficiently by ordering critical edges.

This paper improves convergence bounds for AC and NAC algorithms with function approximation.

Critical volatility triggers log-normal to power-law transitions in interconnected systems.

Stem uses diffusion models to infer gene expression from H&E images.

Introduces a restricted Chen-Nagano variational principle for the Einstein-Hilbert functional.

The paper studies a flow equation on even-dimensional manifolds, proving convergence under critical conditions.

A new method synthesizes expressions from characteristics using GAN for healthcare.

At critical coupling, the interactions of Ginzburg-Landau vortices are determined by the metric on the moduli space of static solutions. The asymptotic form of the metric for two well separated vortices is shown here to be expressible in terms of a Bessel function. A straightforward extension gives the metric for N vor…

Search-based methods for hard combinatorial optimization are often guided by heuristics. Tuning heuristics in various conditions and situations is often time-consuming. In this paper, we propose NeuRewriter that learns a policy to pick heuristics and rewrite the local components of the current solution to iteratively i…

New framework analyzes SGD dynamics in large samples and dimensions.

We consider a 3-dimensional smooth manifold $M$ equipped with an arbitrary, \textit{a priori} non-integrable, distribution (plane field) ${\cal D}$ and a vector field $T$ transverse to ${\cal D}$. Using a 1-form $ω$ such that ${\cal D} = \ker\,ω$ and $ω(T)=1$ we construct a 3-form analogous to that defining the Godbill…

MoEs can efficiently model complex tasks with low-dimensionality and sparsity.

Model-free expression for SSR derived in terms of characteristic function.

New algorithm reduces bias in off-policy reinforcement learning.

We define and examine the notion of a Killing section of a Riemannian Lie algebroid as a natural generalisation of a Killing vector field. We show that the various expression for a vector field to be Killing naturally generalise to the setting of Lie algebroids. As an application we examine the internal symmetries of a…

The expression for the variation of the area functional of the second fundamental form of a hypersurface in a Euclidean space involves the so-called "mean curvature of the second fundamental form". Several new characteristic properties of (hyper)spheres, in which the mean curvature of the second fundamental form occurs…

In this paper, we study the sensitivity of the spectral clustering based community detection algorithm subject to a Erdos-Renyi type random noise model. We prove phase transitions in community detectability as a function of the external edge connection probability and the noisy edge presence probability under a general…

Quantum algorithms for CVaR portfolio optimization face trade-offs between hardware coherence and expressibility.

To address the challenge of backpropagating the gradient through categorical variables, we propose the augment-REINFORCE-swap-merge (ARSM) gradient estimator that is unbiased and has low variance. ARSM first uses variable augmentation, REINFORCE, and Rao-Blackwellization to re-express the gradient as an expectation und…

This study explores star-shaped regularizers learned from critic-based losses.

The critical locus of the loss function of a neural network is determined by the geometry of the functional space and by the parameterization of this space by the network's weights. We introduce a natural distinction between pure critical points, which only depend on the functional space, and spurious critical points, …

NO-BEARS algorithm speeds up gene network inference from transcriptomic data.

We consider a $(2q+1)$-dimensional smooth manifold $M$ equipped with a $(q+1)$-dimensional, a priori non-integrable, distribution ${\cal D}$ and a $q$-vector field ${\bf T}=T_1\wedge\ldots\wedge T_q$, where $\{T_i\}$ are linearly independent vector fields transverse to~${\cal D}$. Using a $q$-form $ω$ such that ${\cal …

This work addresses two main issues of the standard Kernel Entropy Component Analysis (KECA) algorithm: the optimization of the kernel decomposition and the optimization of the Gaussian kernel parameter. KECA roughly reduces to a sorting of the importance of kernel eigenvectors by entropy instead of by variance as in K…

This study reviews and evaluates clustering methods for single-cell RNA-seq data.

The method integrates survival constraints into NMF for identifying survival-associated gene clusters.

The study analyzes how large language models form and express investor risk profiles.

Study spherical curves with curvature dependent on distance to a great circle.

We describe a limitation in the expressiveness of the predictive uncertainty estimate given by mean-field variational inference (MFVI), a popular approximate inference method for Bayesian neural networks. In particular, MFVI fails to give calibrated uncertainty estimates in between separated regions of observations. Th…

It was proved by Graham and Witten in 1999 that conformal invariants of submanifolds can be obtained via volume renormalization of minimal surfaces in conformally compact Einstein manifolds. The conformal invariant of a submanifold $Σ$ is contained in the volume expansion of the minimal surface which is asymptotic to $…