The paper analyzes deep neural networks' expressivity and training, revealing critical expressivity issues.
problem Critical expressivity issues in deep neural networks.
method Quantitative analysis using Hilbert space and Hermite polynomials for feature mapping and activation function design.
result Deep neural networks evolve to the edge of chaos, but expressivity depends on overcoming convergence.
Study reveals different types of critical points in shallow neural networks.
problem Optimization challenges in two-layer ReLU networks with symmetry.
method Symmetry analysis, bifurcation theory, and geometric group actions.
result Different types of spurious minima have distinct loss behavior.
The paper connects reflection groups to maps with specific dynamical properties.
problem Understanding the relationship between reflection groups and anti-rational maps.
method Established a correspondence using planar graphs.
result Complete answers to geometric mating problems for anti-rational maps.
We reformulate the option framework as two parallel augmented MDPs. Under this novel formulation, all policy optimization algorithms can be used off the shelf to learn intra-option policies, option termination conditions, and a master policy over options. We apply an actor-critic algorithm on each augmented MDP, yieldi…
QAM uses adjoint matching to optimize continuous-action RL policies efficiently.
problem Efficient optimization of expressive diffusion or flow-matching policies with respect to a Q-function.
method QAM leverages adjoint matching to bypass the numerical instability of backpropagation through multi-step denoising processes.
result QAM consistently outperforms prior approaches on hard, sparse reward tasks in offline and offline-to-online RL.
Formula for critical points of chi fields on manifolds.
problem Computing critical points of chi fields on general manifolds.
method Semi-analytic formula using Kac-Rice argument and Hessian matrix representation.
result Expression for expected value of critical points in high-threshold limit.
ADAC uses analogous policies to improve RL exploration without sacrificing stability.
problem Improving RL exploration without compromising stability and expressiveness.
method Disentangled actor-critic approach with analogous pairs of actors and critics.
result Empirical evaluation shows ADAC outperforms alternatives in challenging exploration tasks.
Study reveals Transformer's expressive power and mechanisms.
problem Understanding the approximation properties of Transformer for sequence modeling.
method Systematic study of Transformer's components and their combined effects, establishing approximation rates.
result Reveals roles of critical parameters in Transformer, such as number of layers and attention heads.
Using the method of Witten deformation, we express the basic index of a transversal Dirac operator over a Riemannian foliation as the sum of integers associated to the critical leaf closures of a given foliated bundle map.
This work explores the relationship between expressivity and generalization in GNNs.
problem Understanding the trade-off between expressivity and generalization in GNNs.
method Introducing a novel framework that connects GNN generalization to the variance in graph structures they can capture.
result Theoretical findings align with empirical results, offering a deeper understanding of how expressivity enhances GNN generalization.
A new aggregation strategy improves GNN performance and learning dynamics.
problem Improving expressivity and learning dynamics of GNNs.
method Proposes a variance-preserving aggregation function (VPA) for GNNs.
result VPA leads to increased predictive performance and improved learning dynamics.
Given any n-tuple of complex numbers, one can canonically define a polynomial of degree n+1 that has the entries of this n-tuple as its critical points. In 2002, Beardon, Carne, and Ng studied a map θ:Cn→Cn which outputs the critical values of the canonical polynomial constructed from the…
New insights into matrix factorization show strict saddles have bounded eigenvalues.
problem Understanding the nature of critical points in matrix factorization.
method Analyzing orbits of critical points under the general linear group and identifying canonical points.
result Minimum eigenvalue of strict saddles is not uniformly bounded below zero.
When agents interact with a complex environment, they must form and maintain beliefs about the relevant aspects of that environment. We propose a way to efficiently train expressive generative models in complex environments. We show that a predictive algorithm with an expressive generative model can form stable belief-…
Any compact manifold with positive scalar curvature has an associated asymptotically flat metric constructed using the Green's function of the conformal Laplacian, and the mass of this metric is an important geometric invariant. An explicit expression for the mass of the product of spheres S2×S2, both with t…
Study on Gaussian random fields' singularities on manifolds.
problem Understanding singularities of Gaussian random fields on manifolds.
method Computed expected values of singularities under various conditions.
result Explicit formulae for singularities under different constraints.
Deep Reinforcement Learning (DRL) algorithms for continuous action spaces are known to be brittle toward hyperparameters as well as \cut{being}sample inefficient. Soft Actor Critic (SAC) proposes an off-policy deep actor critic algorithm within the maximum entropy RL framework which offers greater stability and empiric…
Given a knot K in an Euclidean space E and a finite dimensional space V of smooth functions on K, we express the expected number of critical points of a random function in V in terms of an integral-geometric invariant of K and V. When V consists of the restrictions to K of homogeneous polynomials of degree d on E, this…
CoRMF uses RNNs to solve Ising models efficiently by ordering critical edges.
problem Solving Ising models efficiently and accurately.
method Criticality-ordered spin sequence and RNNs for mean-field factorization.
result Proves tighter error bounds than naive mean-field.
This paper improves convergence bounds for AC and NAC algorithms with function approximation.
problem Improving convergence bounds for actor-critic algorithms with function approximation.
method Non-asymptotic analysis of AC and NAC algorithms with compatible function approximation.
result Eliminates the term ε_critic from the error bounds while maintaining best known sample complexities.
Critical volatility triggers log-normal to power-law transitions in interconnected systems.
problem Understanding the transition from log-normal to power-law distributions in interconnected systems.
method Analyzing an infinite option-on-option chain model, deriving a critical volatility threshold.
result A critical volatility threshold of approximately 250.66% for unconditional cases, dropping to 125.3% with selective survival.
Stem uses diffusion models to infer gene expression from H&E images.
problem Inference of gene expression from H&E stained images is time-consuming and expensive.
method Conditional diffusion generative model to infer gene expression.
result Stem achieves state-of-the-art performance in spatial gene expression prediction.
Introduces a restricted Chen-Nagano variational principle for the Einstein-Hilbert functional.
problem Deriving critical metrics for the Einstein-Hilbert functional on compact Riemannian manifolds.
method Restricts the variational problem to an infinite-dimensional subspace.
result Derives a novel structural characterization of critical metrics.
The paper studies a flow equation on even-dimensional manifolds, proving convergence under critical conditions.
problem Proving convergence of the prescribed Q-curvature flow equation in critical cases. method Analyzes the flow equation on arbitrary even-dimensional closed Riemannian manifolds, proving convergence under specific geometric hypotheses.
result Proves convergence of the flow equation when the integral of Q equals (n−1)!Vol(Sn), extending previous results. A new method synthesizes expressions from characteristics using GAN for healthcare.
problem Synthesizing expressions from given characteristics in high-dimensional space.
method Generative Adversarial Network (GAN) based selective ensemble learning.
result The proposed SE-CTES method effectively handles deterministic and stochastic patterns.
At critical coupling, the interactions of Ginzburg-Landau vortices are determined by the metric on the moduli space of static solutions. The asymptotic form of the metric for two well separated vortices is shown here to be expressible in terms of a Bessel function. A straightforward extension gives the metric for N vor…
Geometric study of linear neural networks identifies pure and spurious critical points.
problem Understanding the landscape of loss functions in linear neural networks.
method Geometric properties of functional spaces and parameterization analysis.
result Different phenomena cause the absence of bad local minima in linear networks, depending on the architecture and loss function.
Search-based methods for hard combinatorial optimization are often guided by heuristics. Tuning heuristics in various conditions and situations is often time-consuming. In this paper, we propose NeuRewriter that learns a policy to pick heuristics and rewrite the local components of the current solution to iteratively i…
New framework analyzes SGD dynamics in large samples and dimensions.
problem Analyzing stochastic gradient descent in large-scale settings.
method Inspired by random matrix theory, new framework for fixed stepsize and finite sum settings.
result SGD dynamics become deterministic in the large sample and dimensional limit, governed by a Volterra integral equation.
We consider a 3-dimensional smooth manifold M equipped with an arbitrary, \textit{a priori} non-integrable, distribution (plane field) D and a vector field T transverse to D. Using a 1-form ω such that D=kerω and ω(T)=1 we construct a 3-form analogous to that defining the Godbill…
MoEs can efficiently model complex tasks with low-dimensionality and sparsity.
problem Understanding the theoretical foundations of MoEs for complex tasks.
method Systematic study of MoEs with two structural priors: low-dimensionality and sparsity.
result MoEs can approximate functions on low-dimensional manifolds and exhibit exponential structured tasks.
Model-free expression for SSR derived in terms of characteristic function.
problem Calculating the skew-stickiness-ratio (SSR) in financial markets.
method Model-free expression using characteristic function, focusing on diffusion and affine forward variance cases.
result General formula for SSR simplifies and becomes particularly tractable in affine forward variance cases, with a limit of H+3/2 for short-term limit. New algorithm reduces bias in off-policy reinforcement learning.
problem Challenges in designing off-policy reinforcement learning algorithms.
method Doubly robust off-policy actor-critic (DR-Off-PAC) with a single timescale structure.
result Establishes the first overall sample complexity analysis for a single time-scale off-policy AC algorithm.
We define and examine the notion of a Killing section of a Riemannian Lie algebroid as a natural generalisation of a Killing vector field. We show that the various expression for a vector field to be Killing naturally generalise to the setting of Lie algebroids. As an application we examine the internal symmetries of a…
The expression for the variation of the area functional of the second fundamental form of a hypersurface in a Euclidean space involves the so-called "mean curvature of the second fundamental form". Several new characteristic properties of (hyper)spheres, in which the mean curvature of the second fundamental form occurs…
In this paper, we study the sensitivity of the spectral clustering based community detection algorithm subject to a Erdos-Renyi type random noise model. We prove phase transitions in community detectability as a function of the external edge connection probability and the noisy edge presence probability under a general…
Quantum algorithms for CVaR portfolio optimization face trade-offs between hardware coherence and expressibility.
problem Quantum algorithmic resilience for CVaR portfolio optimization
method WS-QAOA vs. HE-VQNN
result WS-QAOA provides exact theoretical mapping but suffers from hardware decoherence, while HE-VQNN preserves hardware coherence but lacks expressibility.
The paper studies variations of the Godbillon--Vey invariant for certain foliations.
problem Investigating the Godbillon--Vey invariant for transversely parallelizable foliations.
method Constructing a (2q+1)-form analogous to the Godbillon--Vey class and expressing it in terms of ω and ${f T}$ for a compatible Riemannian metric. result Characterizing critical pairs of $(ω,{f T})$ and finding sufficient conditions for critical foliations.
To address the challenge of backpropagating the gradient through categorical variables, we propose the augment-REINFORCE-swap-merge (ARSM) gradient estimator that is unbiased and has low variance. ARSM first uses variable augmentation, REINFORCE, and Rao-Blackwellization to re-express the gradient as an expectation und…
This study explores star-shaped regularizers learned from critic-based losses.
problem Understanding the structure of regularizers learned from critic-based losses.
method Optimizing critic-based loss functions over star-shaped regularizers.
result Derives exact expressions for optimal regularizers in certain cases.
NO-BEARS algorithm speeds up gene network inference from transcriptomic data.
problem Constructing accurate gene regulatory networks from transcriptomic data.
method NO-BEARS algorithm, based on NOTEARS, with new constraint and polynomial regression loss.
result Significantly reduced computational time and improved accuracy in inferring gene regulatory networks.
This work addresses two main issues of the standard Kernel Entropy Component Analysis (KECA) algorithm: the optimization of the kernel decomposition and the optimization of the Gaussian kernel parameter. KECA roughly reduces to a sorting of the importance of kernel eigenvectors by entropy instead of by variance as in K…
This study reviews and evaluates clustering methods for single-cell RNA-seq data.
problem Identifying and characterizing novel cell types from single-cell RNA-seq data.
method Review and performance comparison of clustering methods.
result Performance comparison experiments on two datasets.
The method integrates survival constraints into NMF for identifying survival-associated gene clusters.
problem Understanding and interpreting high-dimensional biological data for disease markers.
method Cox proportional hazards regression integrated with NMF via proportional hazards non-negative matrix factorization.
result The method can uncover survival-associated gene clusters in cancer gene expression data.
The study analyzes how large language models form and express investor risk profiles.
problem Understanding how large language models (LLMs) form and express investor risk profiles.
method Examined three LLMs (GPT, Gemini, and Llama) and assessed their responses to a standardized risk questionnaire under varying prompts.
result LLMs generally form long-term investment profiles, but they exhibit different risk tolerance levels.
Study spherical curves with curvature dependent on distance to a great circle.
problem Understanding spherical curves with curvature dependent on distance to a great circle.
method Introducing spherical angular momentum, characterizing known curves, finding new families, and obtaining arc length parametrizations.
result New families of spherical curves with intrinsic equations in elementary or Jacobi elliptic functions.
We describe a limitation in the expressiveness of the predictive uncertainty estimate given by mean-field variational inference (MFVI), a popular approximate inference method for Bayesian neural networks. In particular, MFVI fails to give calibrated uncertainty estimates in between separated regions of observations. Th…
It was proved by Graham and Witten in 1999 that conformal invariants of submanifolds can be obtained via volume renormalization of minimal surfaces in conformally compact Einstein manifolds. The conformal invariant of a submanifold Σ is contained in the volume expansion of the minimal surface which is asymptotic to $…