EUNNs improve RNN performance and efficiency.
problem Gradient explosion/vanishing and long-term correlations in RNNs.
method Unitary matrices with tunable representation capacity and O(1) computational complexity. result EUNNs significantly outperform other RNNs and LSTMs in performance and training speed.
Full-capacity uRNNs improve performance over restricted-capacity ones.
problem Vanishing and exploding gradient issues in recurrent neural networks.
method Optimized full-capacity unitary recurrence matrices over all unitary matrices.
result Significantly improved performance compared to LSTMs and restricted-capacity uRNNs.
GORU combines unitary and gated RNNs for better long-term memory management.
problem Learning to effectively manage long-term memory in neural networks.
method Extending unitary RNNs with a gating mechanism to forget irrelevant information.
result GORU outperforms LSTMs, GRUs, and Unitary RNNs on long-term dependency tasks.
New unitary RNN architecture using complex Cayley transform outperforms existing methods.
problem Vanishing or exploding gradient problem in RNNs.
method Developed a unitary RNN architecture based on a complex scaled Cayley transform.
result scuRNN achieves comparable or better results than existing unitary RNNs.
UGConvs improve CNN accuracy with unitary transforms.
problem Improving CNN accuracy with richer representations.
method UGConvs combine group convolutions with unitary transforms.
result HadaNets achieve similar accuracy to circulant networks with lower complexity.
Quantum neural networks generalize better due to flatter parameter space.
problem Generalization in quantum neural networks.
method Mapped feature data to a quantum state, applied unitary evolution, and measured for classification.
result Quantum neural networks have better generalization than classical networks.
A major challenge in the training of recurrent neural networks is the so-called vanishing or exploding gradient problem. The use of a norm-preserving transition operator can address this issue, but parametrization is challenging. In this work we focus on unitary operators and describe a parametrization using the Lie al…
A new method for optimizing neural networks with orthogonal constraints.
problem Optimizing neural networks with orthogonal constraints.
method Parametrization using the exponential map to transform constrained optimization into unconstrained.
result Faster, more accurate, and stable convergence in RNNs with orthogonal recurrent weights.
scoRNN improves RNN performance with simpler orthogonal weight matrices.
problem Vanishing and exploding gradients in RNNs.
method Parametrizing orthogonal recurrent weight matrices with a scaled Cayley transform.
result scoRNN achieves superior results with fewer parameters than other unitary RNNs.
URNNs are as expressive as general RNNs with ReLU activations.
problem Expressiveness of URNNs compared to general RNNs.
method Input-output equivalence between URNNs and contractive RNNs with ReLU activations.
result URNNs are as expressive as general RNNs with ReLU activations.
Novel neural network approximates exact distance for robust classification.
problem Adversarial attacks on neural networks in safety-critical systems.
method Signed Distance Classifiers (SDCs) and Unitary-Gradient Neural Network.
result Approximates exact distance from classification boundary for certifiable predictions.
Recurrent neural networks (RNNs) are notoriously difficult to train. When the eigenvalues of the hidden to hidden weight matrix deviate from absolute value 1, optimization becomes difficult due to the well studied issue of vanishing and exploding gradients, especially when trying to learn long-term dependencies. To cir…
Quantum neural networks need both data-dependent and trainable unitaries for effective geometric deformation.
problem Quantum neural networks lack the geometric flexibility of classical networks due to limitations in state reachability.
method Viewing quantum states as embedded manifolds, we analyze infinitesimal unitary actions and introduce the CLA maps and aCLS criterion.
result Geometric flexibility in quantum neural networks requires a joint dependence on data and trainable weights.
Quantum neural networks converge to Gaussian processes as they grow.
problem Understanding the convergence of quantum neural networks to Gaussian processes.
method Analyzing Haar random unitary and orthogonal deep QNNs, considering input states, measurement observables, and non-independence of unitary matrix entries.
result Quantum neural networks outputs converge to Gaussian processes in the limit of large Hilbert space dimension.
ENRNN uses eigenvalue normalization for short-term memory in RNNs.
problem Vanishing/exploding gradient problem and long-term dependency modeling.
method Eigenvalue normalization of recurrent matrix to simulate short-term memory.
result ENRNN outperforms existing RNN variants in experiments.
QCNNs avoid barren plateaus, making them trainable.
problem Exponentially vanishing gradients in QNNs.
method Graph-based method to analyze Haar-distributed unitaries.
result QCNNs do not exhibit barren plateaus, implying trainability.
Guarantees recovery of compressible signals from adversarial noise.
problem Recovering compressible signals from noise and adversarial attacks.
method Extends adversarial defense framework to ℓ0, ℓ2, and ℓ∞ norms. result Recovery guarantees for various signal recovery methods under different noise types.
Quantum models learn unitary actions on entangled states from product states.
problem Generalization to out-of-distribution data in quantum machine learning.
method Proved out-of-distribution generalization for learning unitary actions.
result Learned unitary actions on entangled states from product states.
We introduce general scattering transforms as mathematical models of deep neural networks with l2 pooling. Scattering networks iteratively apply complex valued unitary operators, and the pooling is performed by a complex modulus. An expected scattering defines a contractive representation of a high-dimensional probabil…
RUM improves RNN's long-term memory by using unitary matrices.
problem Limited capacity of RNN to manipulate long-term memory.
method Proposes Rotational Unit of Memory (RUM) with unitary matrices.
result RUM learns long-term dependencies and improves state-of-the-art results.
Quantum neural networks approximate periodic functions more efficiently.
problem Approximating periodic functions with quantum neural networks.
method Using Jackson's inequality to construct a QNN that approximates a trigonometric polynomial of the function.
result Quantum neural networks can achieve better approximation results with fewer parameters for smoother functions.
Develops neural networks for reductive Lie groups, enhancing symmetry respect.
problem Symmetry respect in neural networks for reductive Lie groups.
method General equivariant neural network architecture for any reductive Lie Group G.
result Demonstrates generality and performance in top quark decay tagging and shape recognition.
The paper provides theoretical guarantees for optimized sampling in compressed sensing, showing error vanishes with more measurements.
problem Theoretical and practical improvements in compressed sensing with optimized sampling schemes.
method Theoretical analysis and empirical experiments with optimized sampling schemes for subsampled unitary matrices.
result The error caused by measurement noise vanishes with an increasing number of measurements for optimized sampling schemes, assuming Gaussian noise.
Develops a framework for designing quantum neural networks that respect symmetries.
problem Trainability and generalization issues in quantum neural networks.
method Equivariant quantum neural networks (EQNN) for any symmetry group.
result Efficient construction of equivariant layers for EQNNs, including QCNNs.
We formulate the unitary rational orbifold conformal field theories in the algebraic quantum field theory framework. Under general conditions, we show that the orbifold of a given unitary rational conformal field theories generates a unitary modular category. Many new unitary modular categories are obtained. We also sh…
Contact group retracts to unitary subgroup.
problem Understanding contact structures on 3-sphere.
method Proving deformation retraction to unitary subgroup.
result Group of contactomorphisms retracts to U(2).
Zeta functions for non-unitary twists are shown to have analytic continuation.
problem Analytic continuation of zeta functions for non-unitary twists.
method Analytic continuation for compact locally-symmetric spaces with non-unitary twists.
result Zeta functions admit analytic continuation as meromorphic functions.
Revises super unitary representations by relaxing Hilbert space definition.
problem Left-regular representations of super Lie groups are not super unitary.
method Weaken super Hilbert space definition and introduce a new metric.
result Left-regular representations of all super Lie groups are super unitary.
Method quantifies spectral ergodicity in deep learning networks.
problem Understanding the success of deep learning architectures.
method Combines TM and KL divergence metrics to analyze random matrix ensembles.
result Spectral ergodicity increases with network size, suggesting its importance.
Study on determinants of unitary Brownian motion and their asymptotic laws.
problem Understanding determinants of unitary Brownian motion and their behavior over time.
method Using Stiefel fibration and skew-product decomposition of the Stiefel Brownian motion.
result Prove asymptotic laws for determinants of block entries of unitary Brownian motion.
Paper refines RNN training by analyzing smoothness and attractors.
problem Exploding and vanishing gradients in RNNs.
method Refined concept of exploding gradients using cost function smoothness.
result RNNs need to learn attractors to fully use their power.
We construct a new family of toric manifolds generating the unitary bordism ring. Each manifold in the family is the complex projectivisation of the sum of a line bundle and a trivial bundle over a complex projective space. We also construct a family of special unitary quasitoric manifolds which contains polynomial gen…
A fast method for learning MZI parameters in optical neural networks.
problem Time-consuming learning of MZI parameters in optical neural networks.
method Customized complex-valued derivatives and a chain rule for Wirtinger derivatives, incorporated into a function module.
result 20 times faster learning compared to conventional AD in MNIST task.
QRNN uses quantum neurons to learn sequences efficiently.
problem Efficiently learning sequences with quantum computing.
method Parametrized quantum neurons and amplitude amplification.
result QRNN outperforms classical RNNs on sequence learning tasks.
New structure on unitary group of Hilbert space.
problem No specific problem stated; constructing a new structure.
method Constructing a Banach Poisson-Lie group structure.
result Banach Poisson-Lie group structure on unitary group of Hilbert space.
Neural network implementation of Brenier's polar factorization for vector fields.
problem Implementing Brenier's polar factorization theorem for vector fields using neural networks.
method Parameterizing the convex function u as an input convex neural network and estimating the measure-preserving map M. result Practical neural implementation of Brenier's polar factorization theorem.
Study circle actions on unitary manifolds with discrete fixed points.
problem Understanding circle actions on compact unitary manifolds with discrete fixed points.
method Prove relationships between weights at fixed points and derive results regarding the first equivariant Chern class and Hirzebruch χy-genus. result Derive a multigraph encoding fixed point data, leading to new insights into unitary S1-manifolds. This paper tackles quantum machine learning by embedding nonlinear functions in topographic representations.
problem Challenges of nonlinear processes in quantum machine learning.
method Topographic representation of information for quantum machine learning.
result Nonlinear functions can be embedded in unitary processes.
The study shows ergodicity of unitary frame flows on Kähler manifolds with specific curvature conditions.
problem Ergodicity of unitary frame flows on Kähler manifolds with negative holomorphic sectional curvature.
method Analysis of the unitary frame flow on the principal U(m)-bundle of unitary frames.
result For even-dimensional Kähler manifolds with negative λ(m)-pinched holomorphic sectional curvature, the unitary frame flow is ergodic and mixing.
Improved sampling strategy reduces Fourier measurements for neural network signals.
problem Efficiently sampling signals from neural networks with random Fourier matrices.
method Model-adapted sampling strategy with improved sample complexity.
result Reduced sample complexity from O(kdnα∞²) to O(kdα²₂) measurements.
Shared classical randomness improves quantum generative models' output distributions.
problem Improving generative performance of shallow unitary quantum models.
method Introducing stochasticity into unitary quantum models via shared classical randomness.
result Shared classical randomness allows shallow unitary quantum models to represent a strictly larger family of distributions.
Study asymptotics of unitary matrix elements in quantum mechanics.
problem Asymptotic behavior of unitary matrix elements in quantum mechanics.
method Uses Berezin-Toeplitz quantization and symplectic geometry.
result Recover asymptotics of Wigner's d-matrix elements for spin representations.
Under-parameterized networks can either copy or average teacher weights, leading to universal optimal solutions.
problem Approximating a teacher network with an under-parameterized student network.
method Analyzing shallow neural networks with erf activation function and unitary teacher weights, proving copy-average configurations are critical points and finding the optimal solution.
result The optimal solution for under-parameterized networks has a universal structure, whether copying or averaging teacher neurons.
New unitary representations for mapping class groups without almost invariant vectors.
problem Understanding unitary representations of mapping class groups.
method Space of measured foliations and Teichmüller space.
result None of the representations has almost invariant vectors.
Convexity proven for sums of angles of unitary paths.
problem Proving convexity of sums of eigenvalues of unitary matrices.
method Analyzing paths of unitary matrices and their angles, using operator norms.
result Sum of first m angles of unitary path is convex.
Study circumcenters in Finsler unitary groups with optimal convexity bounds.
problem Existence and convexity of circumcenters in Finsler unitary groups.
method Analysis of distance functions and p-Schatten norm on Lie algebra.
result Existence of circumcenters for sets with radius < π/2 in several metrics.
This paper introduces a submanifold of the moduli space of unitary representations of the fundamental group of a punctured sphere with fixed local monodromy. The submanifold is defined via products of involutions through Lagrangian subspaces. We show that the moduli space of Lagrangian representations is a Lagrangian s…
Defines unitary setting for quantum mechanics, explaining time evolution.
problem Completing quantum theory by defining unitary time evolution.
method Introduces geometric space with north and south poles, defines unitary time evolution as vector field flow.
result Unitary time evolution is explained as Lie group-Lie group algebra correspondence.