Proposes α-integration pooling for CNNs to improve performance.
problem Finding optimal pooling method for CNNs is challenging.
method Introduces α-integration pooling with a trainable parameter α. result Demonstrates α-integration pooling outperforms other pooling methods in image recognition. Convolutional neural networks converge quickly with gradient descent.
problem Learning efficient image classifiers with over-parameterized networks.
method Gradient descent for training over-parametrized CNNs with global average-pooling.
result Gradient descent quickly reduces the misclassification risk of CNNs.
GSP improves global average pooling for deep metric learning by learning weights and selecting semantic entities.
problem Improving global average pooling for deep metric learning.
method Generalized Sum Pooling (GSP) method that learns weights and selects semantic entities.
result GSP improves metric learning performance on 4 popular benchmarks.
In most convolution neural networks (CNNs), downsampling hidden layers is adopted for increasing computation efficiency and the receptive field size. Such operation is commonly so-called pooling. Maximation and averaging over sliding windows (max/average pooling), and plain downsampling in the form of strided convoluti…
A modified VDCNN model reduces size and latency for mobile platforms.
problem Memory and processing constraints on mobile platforms.
method Temporal Depthwise Separable Convolutions and Global Average Pooling.
result The squeezed model (SVDCNN) is 10x-20x smaller with minimal accuracy loss.
Tensor regression networks achieve high compression rate of neural networks while having slight impact on performances. They do so by imposing low tensor rank structure on the weight matrices of fully connected layers. In recent years, tensor regression networks have been investigated from the perspective of their comp…
Enhanced CNN kernels improve image classification accuracy.
problem Improving classification accuracy of CNNs.
method Local Average Pooling and random image patch representation.
result 89% accuracy on CIFAR-10, matching AlexNet performance.
The paper reformulates U-Nets as wavelet-based models and applies this to hierarchical VAEs.
problem Theoretical understanding and regularization properties of U-Nets and their relationship to wavelets.
method Formulating a multi-resolution framework to identify U-Nets as finite-dimensional truncations of infinite-dimensional models, proving average pooling corresponds to projection, and identifying HVAEs as discretizations of multi-resolution diffusion processes.
result HVAEs learn a time representation allowing for improved parameter efficiency through weight-sharing.
XceptionTime improves hand gesture recognition accuracy using novel deep learning.
problem Improving hand gesture recognition from sparse sEMG signals.
method Depthwise separable convolutions, adaptive pooling, non-linear normalization.
result Significantly improved accuracy (5.71% improvement) in hand gesture recognition.
Improves text-dependent speaker verification using neural network supervectors and AUC optimization.
problem Enhance performance in text-dependent speaker verification systems.
method Proposes a supervector generation method and AUC optimization for neural networks.
result Improves system performance through novel alignment techniques and AUC optimization.
Proposes a new pooling operator for CNNs to handle spatially varying information.
problem Need to treat spatial locations in non-uniform manner for better image classification.
method Introduces an extended pooling operator that can learn different weights for each pixel location.
result The proposed pooling operator improves generalization and robustness in image classification tasks.
New kernels boost RNN performance on non-time-series data.
problem Improving performance of RNNs on non-time-series data.
method Extended RNN kernels to complex architectures, developed fast GPU implementation.
result RNN-based classifiers outperform baselines on 90 non-time-series datasets.
We propose a simple but strong baseline for time series classification from scratch with deep neural networks. Our proposed baseline models are pure end-to-end without any heavy preprocessing on the raw data or feature crafting. The proposed Fully Convolutional Network (FCN) achieves premium performance to other state-…
New kernels from neural networks show better performance than traditional methods.
problem Improving neural network performance on small datasets.
method Developed algebraic operations to create compositional kernels from neural network architectures.
result Compositional kernels achieve higher accuracy than neural tangent kernels and neural networks on small datasets.
Adaptive masked proxies improve few-shot segmentation efficiency.
problem Efficiently segmenting objects with limited labeled data in robotics.
method Constructs segmentation weights from few labelled samples using multi-resolution average pooling and masked embeddings.
result Outperforms state-of-the-art in few-shot semantic segmentation on PASCAL-5i.
We seek to improve deep neural networks by generalizing the pooling operations that play a central role in current architectures. We pursue a careful exploration of approaches to allow pooling to learn and to adapt to complex and variable patterns. The two primary directions lie in (1) learning a pooling function via (…
Convolution and pooling improve kernel methods in image classification.
problem Understanding the interplay between approximation and generalization in convolutional architectures.
method Characterized RKHS of kernels with convolution, pooling, and downsampling, computed generalization error.
result Convolution and pooling operations trade off approximation with generalization power.
Capsule network improves polyp diagnosis accuracy.
problem Low accuracy of optical biopsy methods for polyps.
method D-Caps architecture with CAP method.
result 43% improvement over previous state-of-the-art.
New CNN initialization scheme derived from modern architectures.
problem Stability of CNN model parameters initialization.
method Derived new initialization scheme from modern CNN architectures.
result New initialization method outperforms de facto standard schemes.
The study characterizes conditions for trainability and generalization in deep neural networks.
problem Understanding the conditions for deep neural networks to be trainable and generalize well.
method Analysis of Neural Tangent Kernel (NTK) for wide and deep networks.
result Large regions of hyperparameter space exist where networks can memorize training data but fail to generalize.
Unified framework for U-Net design and analysis.
problem Understudied design and architecture of U-Nets.
method Theoretical results, Multi-ResNets, function constraints encoding.
result Competitive and superior performance in various tasks.
Global covariance pooling improves deep CNNs' representation and generalization.
problem Capturing richer statistics of deep features for better representation and generalization.
method Integrates global covariance pooling into deep CNNs, addressing challenges with robust covariance estimation and geometry exploitation.
result Proposes MPN-COV Pooling and a Gaussian embedding network, achieving state-of-the-art performance.
Sound event detection (SED) methods are tasked with labeling segments of audio recordings by the presence of active sound sources. SED is typically posed as a supervised machine learning problem, requiring strong annotations for the presence or absence of each sound source at every time instant within the recording. Ho…
3D ConvNets improved with Project & Excite for medical imaging segmentation.
problem Improving segmentation performance in 3D medical imaging.
method Proposed Project & Excite (PE) modules for 3D F-CNNs, extending 2D recalibration methods.
result Project & Excite modules boost segmentation performance up to 0.3 in Dice Score.
Spatial smoothing improves BNNs' accuracy, uncertainty, and robustness without increasing computational cost.
problem Large ensembles in BNNs increase computational cost and reduce performance.
method Spatial smoothing adds blur layers to convolutional neural networks to ensemble neighboring feature map points.
result Spatial smoothing improves BNNs' performance with fewer ensembles and enhances robustness.
BNAS improves neural architecture search with a scalable, fast, and efficient approach.
problem Efficiently searching for optimal neural architectures with high performance and low training time.
method Designing a broad scalable architecture (BCNN) with reinforcement learning and parameter sharing, and developing two variants.
result Significantly reduces training time and achieves state-of-the-art performance on CIFAR-10 and ImageNet.
Tiled Squeeze-and-Excite improves channel attention with local spatial context.
problem Improving channel attention mechanisms in neural networks.
method Proposes tiled squeeze-and-excite (TSE) framework for channel attention.
result Local context of 7 rows or columns is sufficient for matching global context performance.
LipKernel adds robustness to CNNs by enforcing Lipschitz bounds.
problem Improving robustness of CNNs in real-time applications.
method Dissipative layers parameterized by LMIs and 2-D Roesser model.
result Orders of magnitude faster run-time compared to state-of-the-art methods.
New models exploit invariance to reduce model complexity.
problem Reducing model complexity for tasks with inherent invariances.
method Invariant random features and kernel methods.
result Exploiting invariance saves a dα factor in model complexity. Deep neural-kernel models combine neural networks and kernel machines for scalable large datasets.
problem Combining neural networks and kernel machines for efficient large-scale learning.
method Hybrid neural-kernel architecture using explicit feature mapping and pooling layers.
result The deep neural-kernel models are effective and scalable on benchmark datasets.
We propose a random convolutional neural network to generate a feature space in which we study image classification and retrieval performance. Put briefly we apply random convolutional blocks followed by global average pooling to generate a new feature, and we repeat this k times to produce a k-dimensional feature spac…
Geometric triangulations can be transformed by bistellar moves.
problem Transforming geometric triangulations of different manifolds.
method Using bistellar moves, a type of local change to triangulations.
result Geometric triangulations of compact manifolds can be connected by bistellar moves.
Geometric GNNs improve graph discrimination through GWL.
problem Discriminating geometric graphs embedded in Euclidean space.
method Proposed a geometric version of the Weisfeiler-Leman test (GWL) for geometric graphs.
result Characterized the expressive power of geometric GNNs based on physical symmetries.
Geometric Algebra Transformer (GATr) handles various geometric data types efficiently.
problem Lack of a single architecture for diverse geometric data types.
method GATr uses projective geometric algebra, equivariant to E(3), and is a Transformer architecture.
result GATr outperforms non-geometric and equivariant baselines in various geometric tasks.
Geometric methods study 3-manifold splittings.
problem Studying Heegaard splittings of 3-manifolds.
method Geometric approaches.
result Recent advances in geometric methods.
The differential geometric aspects of Geometric Phases are reviewed.
Expanding on previous work, this note generalizes geometric structures results.
problem Generalizing geometric structures results.
method Generalization to a class of geometric structures including integrable almost-complex structures.
result Main results generalized to a broader class of geometric structures.
tf_geometric simplifies graph deep learning in TensorFlow.
problem Efficient graph deep learning in TensorFlow.
method Kernel libraries and infrastructures for GNNs.
result tf_geometric supports various graph tasks and provides efficient GNN models.
Geometric Bass martingales linked to Brownian motion and geometric Brownian motion.
problem Modeling continuous martingales with prescribed initial and terminal distributions.
method Developed geometric Bass martingales and established their properties.
result Explicit bijection and representation of geometric Bass martingales.
Study geometric bounds on generalized Ricci flow.
problem No specific problem stated; focuses on bounds.
method Analogous geometric quantities and bounds proven.
result Geometric and analytic bounds established.
Researchers geometrically define asymptotic coordinates in General Relativity.
problem Understanding the asymptotic behavior of relativistic initial data sets.
method Geometrization of asymptotic flatness and analysis of geometric invariants.
result Geometrically defined asymptotic coordinates for mass, energy, momentum, and angular momentum.
GDB bridges geometric states with improved accuracy and generality.
problem Challenges in predicting geometric state evolution in complex systems.
method Geometric Diffusion Bridge (GDB) framework using equivariant diffusion bridges.
result GDB surpasses existing methods in accurately bridging geometric states.
New findings show some hyperbolic 3-manifolds can't be geometrically bounded.
problem Understanding which cusped hyperbolic 3-manifolds can be geometrically bounded.
method Analyzing embeddings and geometric properties of hyperbolic 3-manifolds.
result Some cusped hyperbolic 3-manifolds cannot be geometrically bounded.
Study extremal trajectories of a rolling disk using geometric control theory.
problem Optimizing the motion of a vertical rolling disk.
method Geometric control theory and symmetries of geometric structures.
result Demonstrated computations in Maple for extremal trajectories.
Estimates small eigenvalues for geometrically finite manifolds.
problem Estimating small eigenvalues of Schrödinger operators.
method Geometrically finite manifolds, Riemannian vector bundles.
result Estimates the number of small eigenvalues.
Geometric symbols help compute heat invariants.
problem Computing heat invariants efficiently.
method Geometric symbol calculus of pseudodifferential operators.
result Efficient computation of heat invariants.
We show that for a strongly convergent sequence of geometrically finite Kleinian groups with geometrically finite limit, the Cannon-Thurston maps of limit sets converge uniformly. If however the algebraic and geometric limits differ, as in the well known examples due to Kerckhoff and Thurston, then provided the geometr…
In this paper we study a collection of jet geometrical concepts, we refer to d-tensors, relativistic time dependent semisprays, harmonic curves and nonlinear connections on the 1-jet space J1(R;M), necessary to the construction of a Miron's-like geometrization for Lagrangians depending on a relativistic time. The geome…