The paper verifies deep neural networks' ability to approximate functions on spheres.
problem Theoretical verification of deep neural networks' performance on spherical functions.
method Spherical analysis using reproducing kernels and convolutional factorizations.
result Rates of uniform approximation for functions in Sobolev spaces and additive ridge forms.
New method solves PDEs on spheres using physics-informed convolutional neural networks.
problem Solving PDEs on surfaces, especially spheres, with high accuracy and efficiency.
method Physics-informed convolutional neural networks (PICNN) with theoretical analysis and approximation results.
result Established fast convergence rates for PICNN solving PDEs on spheres.
Graph-based CNN for spherical data with equivariance.
problem Efficiently learning from non-uniformly distributed spherical data.
method Discretized sphere as graph, graph convolutions, equivariance using Defferrard's graph neural network.
result Good performance on rotation-invariant learning problems.
Simpler neural network for spherical images using Clebsch-Gordan transforms.
problem Learning spherical images rotation invariantly.
method Clebsch-Gordan transform for nonlinearity, avoiding repeated Fourier transforms.
result Improved performance compared to previous methods.
Convolutional Neural Networks (CNNs) have become the method of choice for learning problems involving 2D planar images. However, a number of problems of recent interest have created a demand for models that can analyze spherical images. Examples include omnidirectional vision for drones, robots, and autonomous cars, mo…
DeepSphere improves spherical CNNs by balancing efficiency and rotation equivariance.
problem Designing efficient and rotation-equivariant convolutional layers for spherical data.
method Graph-based approach to represent spherical data, focusing on the number of vertices and neighbors.
result DeepSphere achieves state-of-the-art performance and demonstrates efficiency and flexibility.
Sparse connectivity improves generalization in neural networks below the Edge of Stability.
problem Generalization guarantees for fully-connected networks fail at the Edge of Stability.
method Analyzed sparse connectivity's impact on generalization in two-layer ReLU networks.
result Sparse connectivity changes the effective constraint, leading to non-vacuous generalization bounds.
DELIMIT PyTorch enhances deep learning for diffusion imaging.
problem Applying deep learning to spherical diffusion imaging data.
method Added spherical harmonic interpolation and local convolution layers to PyTorch.
result Deep learning can now be applied conveniently to diffusion imaging data.
Generalizes CNNs on homogeneous spaces like Euclidean and spherical surfaces.
problem Classifying and understanding equivariant CNNs on homogeneous spaces.
method Develops a theory for equivariant maps between field spaces of given types.
result Equivariant kernels correspond to the most general kind of equivariant linear maps.
This paper uses a geometric approach to understand how normalization layers affect neural network optimization.
problem Understanding the effect of normalization layers on optimization in neural networks.
method Introduces a spherical framework to study optimization dynamics of neural networks with normalization layers from a geometric perspective.
result Derives the first effective learning rate expression of Adam and shows that SGD with NLs is equivalent to a constrained variant of Adam.
Designs neural networks on matrix manifolds for improved performance in tasks like human action recognition.
problem Designing neural networks for tasks on non-Euclidean manifolds.
method Develops fully-connected and convolutional layers for SPD manifolds, and MLR on SPSD manifolds.
result Demonstrates improved performance in human action recognition and node classification tasks.
We describe convolutional networks using harmonic functions.
problem Understanding the function space and smoothness of convolutional networks.
method Using reproducing kernel Hilbert spaces and functional ANOVA decomposition.
result Convolutional networks can be decomposed into a sum of elementary functions.
Group equivariant neural networks simplify complex tasks with group representation theory.
problem Challenging tasks requiring input transformations like rotations.
method Group representation theory, non-commutative harmonic analysis, differential geometry.
result A neural network is group equivariant if and only if it has a convolutional structure.
Gauge equivariant CNNs improve image segmentation and climate pattern analysis.
problem Improving image segmentation and climate pattern analysis on non-Euclidean surfaces.
method Developed gauge equivariant convolutional neural networks (gCNNs) for icosahedral surfaces.
result Significant improvements in image segmentation and climate pattern analysis.
CeCNN predicts SE and AL from UWF images, improving myopia screening.
problem Predicting axial length and spherical equivalence from UWF fundus images.
method Copula-enhanced Convolutional Neural Network (CeCNN) for multiresponse regression.
result CeCNN improves prediction of SE and AL compared to baseline CNNs.
New method uses spherical convolutional Wasserstein distance to validate climate models.
problem Ensuring the accuracy of global climate models.
method Spherical convolutional Wasserstein distance to measure model differences.
result Phase 6 models show modest improvements in realistic climatologies.
Study shows directional convergence for neural networks under spherical symmetry.
problem Learning linear predictors with neural networks under spherically symmetric data.
method Analysis of gradient flow and gradient descent for two-layer and deep linear networks.
result Directional convergence guarantees with exact convergence rate for specific network architectures.
Paper reduces Hausdorff Distance in medical image segmentation.
problem Reduction of Hausdorff Distance in medical image segmentation.
method Three novel loss functions for training CNNs to estimate and reduce HD.
result Approximately 18-45% reduction in HD without degrading other performance metrics.
Paper proposes SMFN for high-res spherical video super-resolution.
problem Super-resolution of 360-degree panoramic videos is expensive and challenging.
method Deformable convolutions, mixed attention mechanism, dual learning strategy, weighted mean square error loss function.
result The proposed SMFN method improves super-resolution of equatorial regions in 360-degree videos.
ChebLieNet uses Lie groups to create invariant spectral graph networks.
problem Handling anisotropic data in graph neural networks.
method Develops anisotropic convolutional layers on Lie groups with Riemannian metrics.
result Demonstrates the effectiveness of balancing equivariance and invariance.
The Funk-Minkowski transform and spherical convolution reconstruct functions and vector fields on the sphere.
problem Reconstructing functions and vector fields on the sphere using Funk-Minkowski transform and Hilbert type spherical convolution.
method Inversion formula for Funk-Minkowski transform and Helmholtz-Hodge decomposition solution using spherical convolution.
result Complete reconstruction of functions and vector fields on the sphere.
Paper explores neural network approximations on sphere domains.
problem Approximating functionals on sphere domains using neural networks.
method Encoder-decoder framework with spherical harmonics for infinite-dimensional domain.
result Approximation rates of neural networks with different encoder structures.
ConvNets can be translated into CKNs that perform similarly.
problem The distinction between ConvNets and kernel-based methods.
method Translation of ConvNets into CKNs using a new gradient algorithm.
result CKNs perform as well as ConvNets, supporting the translation.
Develops methods for learning similarity metrics and group-equivariant representations.
problem Learning discriminative representations for comparing objects, especially when limited computational resources are available.
method Proposes new formulations for metric learning, including extensions for kNN regression and asymmetric similarity learning. Introduces a computationally inexpensive approach for estimating metrics using gradient estimates. Develops SO(3)-equivariant neural networks for spherical data.
result Demonstrates improved k-NN accuracy and regression performance through novel metric learning formulations.
VC dimensions of group CNNs are infinite for certain kernels and groups.
problem Estimating the generalization capacity of group convolutional neural networks.
method Identifying precise VC dimension estimates for simple sets of group CNNs.
result Two-parameter families of convolutional neural networks have an infinite VC dimension for infinite groups and certain kernels.
Characterization of lung nodules as benign or malignant is one of the most important tasks in lung cancer diagnosis, staging and treatment planning. While the variation in the appearance of the nodules remains large, there is a need for a fast and robust computer aided system. In this work, we propose an end-to-end tra…
BiGraphNet generalizes graph neural networks for more efficient operations.
problem Fragmented graph neural network architectures hinder optimization.
method Explicitly separates input and output nodes, enabling new efficient operations.
result BiGraphNet accelerates and scales computations in hierarchical networks.
Convolutional neural networks converge quickly with gradient descent.
problem Learning efficient image classifiers with over-parameterized networks.
method Gradient descent for training over-parametrized CNNs with global average-pooling.
result Gradient descent quickly reduces the misclassification risk of CNNs.
Proposes a method to constrain singular values of convolutional kernels in neural networks.
problem Avoiding exploding/vanishing gradient problems and improving generalizability in neural networks.
method Introduces a penalty function to constrain singular values of convolutional kernels around 1, and derives an algorithm for optimization.
result Demonstrates the effectiveness of the method through numerical examples.
We found a 'Goldilocks zone' in neural network loss landscapes that correlates with good initialization.
problem Understanding and optimizing neural network loss landscapes for better initialization.
method Random and low-dimensional hypersurfaces to evaluate the Hessian of loss functions.
result The Goldilocks zone is a region of unusually high convexity and positive curvature, correlated with good network performance.
New algorithm learns convolutional neural networks with overlapping patches.
problem Learning convolutional neural networks with overlapping patches.
method Algorithm draws from isotonic regression and landscape analysis.
result Algorithm works for general class of patches, including common computer vision structures.
Paper examines Gaussian process perspective of CNNs.
problem Understanding when and why CNNs perform well.
method Casts CNNs in a Gaussian process framework.
result Gains insights into CNN performance and assumptions.
Convolutional neural networks improve image classification accuracy.
problem Improving accuracy in image classification.
method Analyzing the convergence rate of misclassification risk for image classifiers.
result A rate of convergence independent of image dimension proves the effectiveness of CNNs.
Deep neural network predicts traffic flow on city maps.
problem Short-term traffic flow prediction on high-resolution city maps.
method UNet-based deep convolutional neural network with densely connected layers.
result Best performance on the Traffic4cast challenge 2019.
Capsule networks are vulnerable to adversarial attacks, similar to convolutional neural networks.
problem Vulnerability of capsule networks to adversarial attacks.
method Compared capsule networks to convolutional neural networks using various adversarial attacks.
result Capsule networks are vulnerable to adversarial attacks, similar to convolutional neural networks.
This work proposes hyperbolic deep convolutional neural networks for better pattern recognition.
problem The limitations of Euclidean deep convolutional neural networks in capturing intricate patterns.
method Developed Hyperbolic DCNN based on Poincaré Disc, analyzing expansive convolution in non-Euclidean space.
result Hyperbolic convolutional architecture outperforms Euclidean ones in pattern recognition tasks.
Proves DCNNs with expansive convolution are strongly universally consistent.
problem Theoretical consistency of deep convolutional neural networks (DCNNs).
method Empirical risk minimization on DCNNs with expansive convolution (with zero-padding).
result DCNNs with expansive convolution are strongly universally consistent.
Deep CNNs can approximate any function, proving their efficiency.
problem Understanding the approximation and generalization ability of deep learning methods.
method Demonstrated universality of deep convolutional neural networks (CNNs).
result Deep CNNs can approximate any continuous function to arbitrary accuracy.
This research studies affine invariance in continuous-domain convolutional neural networks.
problem Recognizing patterns and features under affine transformations in continuous domains.
method Introduces a new criterion for assessing affine invariance, embeds images into the affine Lie group, and analyzes convolution over this group.
result Extends the scope of geometrical transformations that deep-learning pipelines can handle.
L-CNNs learn gauge invariant quantities on lattices.
problem Learning gauge invariant quantities on lattices.
method Novel convolutional layer preserving gauge equivariance and forming Wilson loops.
result L-CNNs can approximate any gauge covariant function on the lattice.
Convolutional neural networks can regularize inverse problems without training.
problem Solving inverse problems like image recovery from limited data.
method Fixed or parameterized convolutional networks with few parameters.
result Untrained convolutional networks can recover images from few measurements.
L-CNNs maintain gauge symmetry on non-Abelian lattice theories.
problem Applying convolutional neural networks to non-Abelian lattice gauge theories while preserving gauge symmetry.
method Developed a geometric formulation of L-CNNs that are equivariant under global symmetries and gauge transformations.
result Convolutional operations in L-CNNs are a specific case of gauge-equivariant neural networks on SU(N) principal bundles. RecNets use RNNs to process image channels in a compact, recurrent way.
problem Creating efficient neural network architectures for computer vision.
method Introducing RecNets with CRC layers that simulate recurrent processing of image channels.
result RecNets achieve superior size-accuracy trade-off compared to other compact models.
Coordinate-independent convolutions on manifolds avoid reference frame ambiguity.
problem Applying convolutions on non-Euclidean manifolds without reference frame ambiguity.
method Developed coordinate-independent and gauge-equivariant convolutions on Riemannian manifolds.
result Coordinate-independent convolutions are equivariant under local gauge transformations.
Proposes a new convolutional neural network for non-grid data.
problem Limited applicability of standard CNNs to non-grid structured data.
method Introduces Parametric Continuous Convolution (PCC) with learnable kernel functions.
result Significant improvement in point cloud segmentation and lidar motion estimation.
New PTC convolution preserves properties of Euclidean convolutions on manifolds.
problem Lack of generalizable convolutions on curved domains with desirable properties.
method Parallel transport convolution (PTC) on Riemannian manifolds.
result PTC preserves compactly supported filters and directionality.
New bounds for CNNs show better generalization than previous models.
problem Improving understanding of CNNs' generalization ability.
method Proposed tighter generalization bounds for CNNs by exploiting the sparse and permutation structure of weight matrices and spectral norms of convolution operations.
result Theoretical and experimental results show tighter bounds for CNNs than existing bounds.
Improved CNN with general image processing kernels reduces training time and achieves high accuracy.
problem Training time and accuracy of CNNs.
method Used 41 general-purpose kernels for the first layer of CNNs.
result GFNN reduces training time by 30% and achieves 99.56% accuracy.