Finite singular times for symmetric network curvature flow.
problem Formation of singularities in network curvature flow.
method Curvature flow of networks with symmetric initial data and two triple junctions.
result The set of singular times is finite.
New neural networks respect symmetries in symmetric tensors, improving efficiency and generalization.
problem Learning from symmetric tensors efficiently and respecting their inherent symmetries.
method Developed two characterizations of linear permutation equivariant functions between symmetric power spaces of R^n.
result These functions are highly data efficient compared to standard MLPs and generalize well to different sizes of symmetric tensors.
The paper analyzes tensor recovery from symmetric rank-one measurements using information theory.
problem Recovering tensors with low symmetric rank from symmetric rank-one measurements.
method Covering numbers argument, Carbery-Wright inequality, orthogonal polynomials, Fano's inequality.
result Near-optimal sample complexity bounds for log-concave distributions.
Symmetric critical points lead to symmetry breaking in neural networks.
problem Understanding symmetry in critical points of invariant functions.
method Analyzing the symmetry of critical points and their neighbors in invariant nonconvex functions.
result Symmetric critical points in invariant nonconvex functions are generically followed by symmetry breaking adjacent points.
New neural network models learn symmetric functions of varying input sizes.
problem Learning symmetric functions with varying input sizes.
method Functional perspective on neural networks, treating symmetric functions as functions over probability measures.
result Established approximation and generalization bounds for shallow architectures that extend across input sizes.
Novel neural network approach on hyperbolic and SPD spaces.
problem Developing neural networks on symmetric spaces of noncompact type.
method Unified formulation of distance from a point to a hyperplane.
result Closed-form expression for point-to-hyperplane distance in higher-rank spaces.
New deep learning methods solve symmetric PDEs efficiently.
problem Solving nonlinear symmetric PDEs in high dimensions.
method Design of PointNet and DeepSet neural networks.
result DeepSet networks provide more accurate solutions and gradients.
New algorithms learn simple staged trees from data, improving model fit.
problem Complex conditional independences in categorical data vectors.
method Structural learning algorithms for simple staged trees, coalescing the underlying tree.
result Data-learned simple staged trees often outperform Bayesian networks in model fit.
Symmetrizes loss functions to improve neural network robustness against noisy labels.
problem Designing robust loss functions for noisy labels in neural networks.
method Symmetrization of multi-class loss functions, focusing on cross-entropy and unhinged loss.
result The multi-class unhinged loss is the unique convex symmetric loss under suitable assumptions.
Deep networks can memorize random labels; symmetric loss improves this.
problem Deep networks can memorize random labels, ignoring standard regularization.
method Empirical studies with MNIST and CIFAR-10 datasets, formal definition of robustness.
result Symmetric loss function improves network's ability to resist memorization.
Bayesian networks are simplified for categorical variables using staged trees and asymmetry-labeled DAGs.
problem Representing non-symmetric conditional independences in Bayesian networks.
method Formalized relationship between Bayesian networks and staged trees, introduced asymmetry-labeled DAGs, and developed an algorithm to learn staged trees.
result A novel algorithm for learning staged trees that captures non-symmetric independences.
Symmetric CNNs improve sequential recommendation and protein structure prediction.
problem Improving prediction accuracy in sequential recommendation and protein structure inference.
method Developed a CNN architecture that preserves symmetry in convolutional layers, using parameterized convolutional kernels.
result Symmetric structured CNNs achieve better performance with fewer parameters.
New algorithm samples Bayesian neural networks for improved calibration.
problem Improving calibration of Bayesian neural networks.
method Symmetric Minibatch Splitting-UBU (SMS-UBU) algorithm.
result SMS-UBU provides better calibration performance than standard methods.
Proposes SGELU for neural networks to improve performance and convergence.
problem Improving neural network performance and convergence.
method Integrates Gaussian Error Linear Unit (GELU) with symmetrical characteristics to create SGELU.
result SGELU outperforms GELU and LiSHT in MNIST classification and auto-encoder tasks.
New method for inferring network topology from partial data.
problem Inferring network topology from limited node data.
method Vector autoregressive model and Gaussian mixture algorithm.
result The proposed method converges to the network combination matrix in probability.
Paper introduces Deep Sets for Symmetric Elements (DSS) layers for learning sets of symmetric elements.
problem Learning sets of symmetric elements is underexplored.
method Characterized equivariant layers, showed DSS layers are universal approximators, and demonstrated their effectiveness.
result DSS layers improve set-learning architectures across various data types.
We give a new algorithm for learning a two-layer neural network under a general class of input distributions. Assuming there is a ground-truth two-layer network y=Aσ(Wx)+ξ, where A,W are weight matrices, ξ represents noise, and the number of neurons in the hidden layer is no larger than the input or outp…
Deep neural networks with heavy-tailed weights converge to stable distributions.
problem Understanding the convergence of heavy-tailed weights in infinitely-wide neural networks.
method Analyzing infinitely-wide multi-layer perceptrons with i.i.d. symmetric α-stable weight distributions. result The vector of pre-activation values converges to i.i.d. symmetric α-stable distributions. We study the robustness to symmetric label noise of GNNs training procedures. By combining the nonlinear neural message-passing models (e.g. Graph Isomorphism Networks, GraphSAGE, etc.) with loss correction methods, we present a noise-tolerant approach for the graph classification task. Our experiments show that test a…
Proposes Siegel neural networks for improved classification tasks.
problem Classification on Siegel spaces is underexplored.
method Uses quotient structure and vector-valued distance notation.
result Demonstrates state-of-the-art performance in radar and node classification.
New method uses symmetric splitting for efficient HMC inference in large neural networks.
problem Efficient inference for Bayesian neural networks with large datasets.
method Introduces a symmetric integration scheme for Hamiltonian Monte Carlo (HMC) that does not rely on stochastic gradients.
result Symmetric splitting leads to more efficient HMC inference over large data sets.
S-SGD adds symmetrical noise to weights to avoid sharp minima in deep learning.
problem SGD does not always converge to a flat minimum, leading to poor generalization.
method Symmetrical weight noise injection in SGD.
result S-SGD outperforms conventional SGD and weight-noise injection methods in large batch training.
Stable processes emerge as limits of deep neural networks with symmetric stable distributions.
problem Understanding the behavior of deep neural networks as they become infinitely wide.
method Analyzing fully connected feed-forward deep neural networks with symmetric stable distributions and showing the limit as a stable process.
result The infinite wide limit of the network is a stable process with multivariate stable distributions.
Study shows directional convergence for neural networks under spherical symmetry.
problem Learning linear predictors with neural networks under spherically symmetric data.
method Analysis of gradient flow and gradient descent for two-layer and deep linear networks.
result Directional convergence guarantees with exact convergence rate for specific network architectures.
We introduce SARR for symmetric object pose estimation, improving CNN performance.
problem Ambiguities in symmetric object orientations hinder deep learning pose estimation.
method Numeric rotation representation using symmetry-derived trigonometric identities.
result SARR enables standard CNNs to achieve state-of-the-art performance.
Deep networks can handle noisy labels up to a certain threshold.
problem Deep learning's robustness to noisy labels.
method Applying classical statistical theory and universal consistency of DNNs.
result Certain DNNs can tolerate massive symmetric label noise up to the information-theoretic threshold.
Proposes a symmetric graph autoencoder for unsupervised learning.
problem Graph representation learning without labeled data.
method Symmetric graph convolutional autoencoder with Laplacian sharpening and signed graphs.
result Outperforms state-of-the-art algorithms in clustering, link prediction, and visualization tasks.
Variational inference struggles with weight symmetries in neural networks, leading to biased posteriors.
problem Weight space symmetries in neural networks cause multimodal posteriors, challenging variational inference.
method Developed a symmetrization mechanism to create permutation invariant variational posteriors.
result Symmetrized variational posteriors have a better fit to the true posterior and improved predictive performance.
Olshausen and Field (OF) proposed that neural computations in the primary visual cortex (V1) can be partially modeled by sparse dictionary learning. By minimizing the regularized representation error they derived an online algorithm, which learns Gabor-filter receptive fields from a natural image ensemble in agreement …
Group-invariant neural networks improve approximation accuracy for symmetric functions.
problem Improving approximation accuracy for symmetric functions using neural networks.
method Investigates the generalization error of group-invariant neural networks within the Barron framework.
result Group invariance introduces a factor δ that can significantly improve approximation accuracy when it is small.
Transformers tend to learn more symmetric functions in sequence data.
problem Understanding inductive bias in Transformers with infinitely over-parameterized models.
method Analyzing Transformers in the Gaussian process limit, using representation theory of the symmetric group.
result Transformers are biased towards more permutation symmetric functions, and this can be quantitatively predicted.
New method for mixed memberships using symmetrized Laplacian inverse matrix.
problem Mixed memberships in community detection.
method Spectral clustering on symmetrized Laplacian inverse matrix.
result Mixed-SLIM methods outperform state-of-the-art methods.
Deep neural networks favor symmetric structures, enabling multilevel symmetries.
problem Understanding and optimizing deep neural networks.
method Formulating DNN training as convex Lasso problems with geometric algebra.
result Deep networks inherently favor symmetric structures, enabling multilevel symmetries.
Equivariant flows sample symmetric multi-body systems like proteins.
problem Sampling symmetric multi-body systems like proteins with strong interactions.
method Developed equivariant flows that respect the symmetries of the energy function.
result Equivariant flows can sample new configurations not possible with non-equivariant flows.
The paper develops mathematical models for neural networks using non-compact symmetric spaces.
problem Developing mathematical models for neural networks using non-compact symmetric spaces.
method Introducing layers modeled as non-compact symmetric spaces, each mapped onto the next by solvable group homomorphisms.
result Group theoretical construction of separators for all non-compact symmetric spaces and uniformization of specific surfaces.
E2M predicts metric space outputs using deep learning.
problem Predicting non-Euclidean outputs like distributions and matrices.
method Weighted Fréchet means over learned weights.
result E2M achieves state-of-the-art performance across various outputs.
We prove lognormal distribution for symmetric perceptron model, solving key conjectures.
problem Understanding the performance of learning algorithms in neural networks.
method Lognormal distribution characterization and small graph conditioning method.
result Established lognormal distribution and several conjectures for the symmetric perceptron model.
Protein function prediction is the important problem in modern biology. In this paper, the un-normalized, symmetric normalized, and random walk graph Laplacian based semi-supervised learning methods will be applied to the integrated network combined from multiple networks to predict the functions of all yeast proteins …
Convolutional neural networks handle rotated image symmetries without dimensionality issues.
problem Binary image classification with rotational symmetry.
method Least squares plug-in classifiers based on convolutional neural networks under rotationally symmetric assumptions.
result Convolutional neural networks can circumvent the curse of dimensionality in binary image classification with rotational symmetry.
Study on optimal ReLU networks with weight decay for interpolation.
problem Interpolating data with radially symmetric distributions using shallow ReLU networks.
method Weight decay regularization in infinite neuron, infinite data limit; analysis of growth rates.
result Existence and growth rates of unique radially symmetric minimizers with weight decay.
Stable unactivated neurons reduce expressiveness in ReLU networks.
problem Reducing expressiveness in ReLU neural networks due to stably unactivated neurons.
method Investigated the probability of neurons being stably unactivated in ReLU networks with symmetric weight and bias distributions.
result Proved the probability of a neuron being stably unactivated in the second hidden layer of a ReLU network.
We consider convex symmetric lens-shaped networks in R^2 that evolve under curve shortening flow. We show that the enclosed convex domain shrinks to a point in finite time. Furthermore, after appropriate rescaling the evolving networks converge to a self-similarly shrinking network, which we prove to be unique in an ap…
Algorithm designs neural group actions for symmetric transformations.
problem Designing neural networks for symmetric transformations.
method Develops Neural Group Actions (NGAs) for finite groups, enforcing volume-preserving constraints.
result Demonstrates NGAs for the quaternion group Q8 can learn quantum gate transformations. The paper extends hypothesis testing to non-diagonalizable matrices, improving network statistics inference.
problem Testing on non-diagonalizable matrices for network statistics.
method Generalizes Wald and t-tests to non-symmetric matrices, controlling convergence rates.
result Improved inference on network statistics from directed networks.
We extend neural networks with fractional and mixed activation functions for better function approximation.
problem Limitations in approximating higher-order smooth functions in complex spaces.
method Incorporating fractional exponents in activation functions and defining new density functions.
result Improved accuracy and broader applicability of neural network approximation theory.
Researchers extend ResNets to Riemannian manifolds, improving performance over existing methods.
problem Learning on Riemannian manifolds, especially for hierarchical graphs and manifold-valued data.
method Geometrically principled extension of ResNets to general Riemannian manifolds.
result Riemannian ResNets outperform existing manifold neural networks in relevant metrics and training dynamics.
RotEqNet preserves rotation symmetry in fluid systems using high-order tensors.
problem Lack of rotational symmetry in machine learning models for fluid systems.
method Introduces RotEqNet, a network that guarantees rotation-equivariance for high-order tensors.
result RotEqNet reduces errors and maintains rotation-equivariance in fluid systems.
New algorithm eliminates symmetry requirement for training neural networks on resistive device arrays.
problem Training accuracy on resistive device arrays depends on device switching symmetry.
method Developed 'Tiki-Taka' algorithm to minimize unintentional cost term due to device asymmetry.
result Achieves same accuracy with non-symmetric devices as with symmetric devices.