Convolutional neural networks (CNNs) have achieved breakthrough performances in a wide range of applications including image classification, semantic segmentation, and object detection. Previous research on characterizing the generalization ability of neural networks mostly focuses on fully connected neural networks (F…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “convolution bounds”
New bounds for quantile aggregation unify and clarify existing methods.
problem Analytical bounds for quantile aggregation with dependence uncertainty.
method Using inf-convolution of quantile-based risk measures, establish new analytical bounds called convolution bounds.
result Convolution bounds are the best available and provide sharp results in many cases.
The paper bounds the complexity of GCNs using Rademacher complexity.
problem Understanding the sample complexity of GCNs.
method Derived tight upper and lower bounds of Rademacher complexity for GCN models.
result The derived bounds depend on the largest eigenvalue of the graph filter and the degree distribution.
We prove bounds on the generalization error of convolutional networks. The bounds are in terms of the training loss, the number of parameters, the Lipschitz constant of the loss and the distance from the weights to the initial weights. They are independent of the number of pixels in the input, and the height and width …
New bounds improve deep learning performance efficiently.
problem Improving generalization and robustness of deep learning models.
method Deriving four provable upper bounds on spectral norm of convolution layers, differentiable and efficient.
result Minimum of four bounds is a tight, differentiable and efficient upper bound on spectral norm.
We describe convolutional networks using harmonic functions.
problem Understanding the function space and smoothness of convolutional networks.
method Using reproducing kernel Hilbert spaces and functional ANOVA decomposition.
result Convolutional networks can be decomposed into a sum of elementary functions.
We establish linear regret bounds for convex smooth losses using Fenchel-Young losses.
problem Establishing linear regret bounds for convex smooth losses.
method Constructing a convex smooth surrogate loss using Fenchel-Young losses generated by the convolutional negentropy.
result We derive a smooth loss with a linear surrogate regret bound.
LipKernel adds robustness to CNNs by enforcing Lipschitz bounds.
problem Improving robustness of CNNs in real-time applications.
method Dissipative layers parameterized by LMIs and 2-D Roesser model.
result Orders of magnitude faster run-time compared to state-of-the-art methods.
New method tightens Lipschitz bounds for CNNs efficiently.
problem Lipschitz regularization of Convolutional Neural Networks (CNNs).
method Using Toeplitz matrix theory, introduces a tight and computationally efficient upper bound for convolutional layers.
result Developed an algorithm to train Lipschitz regularized CNNs.
Extended Convolution Bounds on the Fréchet Problem: Robust Risk Aggregation and Risk Sharingq-fin.RM
Paper provides new bounds for risk aggregation and sharing.
problem Quantitative risk management and robust risk aggregation with dependence uncertainty.
method Established new inequality for RVaR, derived extended convolution bounds, and analyzed risk sharing for averaged quantiles.
result Extended convolution bounds for robust risk aggregation and risk sharing, providing sharpness conditions and explicit expressions.
Study learns convolution operators on compact Abelian groups using regularization.
problem Learning convolution operators on compact Abelian groups.
method Regularization-based approach with ridge regression estimator.
result Characterizes the accuracy of the estimator in terms of finite sample bounds.
A Frobenius norm regularization method for convolutional kernels to avoid unstable gradient problemcs.LG
Convolutional neural network is a very important model of deep learning. It can help avoid the exploding/vanishing gradient problem and improve the generalizability of a neural network if the singular values of the Jacobian of a layer are bounded around in the training process. We propose a new penalty function for…
New bounds for neural networks ensure robustness and accuracy.
problem Ensuring robustness of neural networks by computing Lipschitz constants.
method Analyzed and proposed new bounds for and norms, using explicit and implicit methods for convnets.
result One of the new bounds is optimal and more accurate than existing ones.
A new FFT method for Heston model option pricing with explicit error bounds.
problem Efficiently pricing European options in the Heston model with high accuracy.
method Convolution-FFT method leveraging a continuously differentiable joint characteristic function.
result Explicit error bounds for FFT-based convolution method in Heston model.
Improved singular value approximation for convolutional layers.
problem Improving accuracy of singular value approximation for linear convolutional layers.
method Developed a new spectral density matrix method for singular value approximation with improved accuracy and reduced computational complexity.
result Obtained moderate improvement in singular value distribution compared to circular approximation.
Deep convolutional networks can be understood through kernel methods, providing insights into their inductive bias.
problem Understanding the functional space and inductive bias of deep convolutional networks.
method Using kernel methods to analyze simple hierarchical kernels with convolution and pooling layers.
result The RKHS consists of additive models of interaction terms between patches, and pooling layers encourage spatial similarities.
Convolutional neural networks converge quickly with gradient descent.
problem Learning efficient image classifiers with over-parameterized networks.
method Gradient descent for training over-parametrized CNNs with global average-pooling.
result Gradient descent quickly reduces the misclassification risk of CNNs.
Though Convolutional Neural Networks (CNNs) have surpassed human-level performance on tasks such as object classification and face verification, they can easily be fooled by adversarial attacks. These attacks add a small perturbation to the input image that causes the network to misclassify the sample. In this paper, w…
Recently the generalization error of deep neural networks has been analyzed through the PAC-Bayesian framework, for the case of fully connected layers. We adapt this approach to the convolutional setting.
Lipschitz constraints under L2 norm on deep neural networks are useful for provable adversarial robustness bounds, stable training, and Wasserstein distance estimation. While heuristic approaches such as the gradient penalty have seen much practical success, it is challenging to achieve similar practical performance wh…
Convolutional neural network is an important model in deep learning. To avoid exploding/vanishing gradient problems and to improve the generalizability of a neural network, it is desirable to have a convolution operation that nearly preserves the norm, or to have the singular values of the transformation matrix corresp…
A new method speeds up spectral normalization for neural nets.
problem Efficiently controlling the spectral norm of convolutional layers.
method Depthwise separable convolutions with spectral normalization.
result Significant reduction in computational and memory costs.
Improved robustness of 1D CNNs for heart arrhythmia classification.
problem Improving the robustness of 1D CNNs for classification tasks.
method Parameterization using Cayley transform and controllability Gramian for Lipschitz-bounded CNNs.
result Improved robustness of trained Lipschitz-bounded 1D CNNs for heart arrhythmia classification.
Convolutional analysis operator learning (CAOL) enables the unsupervised training of (hierarchical) convolutional sparsifying operators or autoencoders from large datasets. One can use many training images for CAOL, but a precise understanding of the impact of doing so has remained an open question. This paper presents…
Wiatowski and Bölcskei, 2015, proved that deformation stability and vertical translation invariance of deep convolutional neural network-based feature extractors are guaranteed by the network structure per se rather than the specific convolution kernels and non-linearities. While the translation invariance result appli…
The study estimates the expressiveness of GCNs with bounds on the number of linear regions.
problem Characterizing the expressiveness of graph convolutional networks (GCNs).
method Estimates the number of linear regions for one-layer and multi-layer GCNs.
result GCNs with multiple layers have exponentially more expressivity per parameter than one-layer GCNs.
Study on VC dimension of GCNNs with input resolution effects.
problem Understanding the generalization capabilities of GCNNs.
method Derived upper and lower bounds for VC dimension, analyzed factors affecting it.
result Extended previous results on VC dimension of GCNNs, providing insights into input resolution dependence.
New method enforces orthogonality in convolutional layers for improved robustness.
problem Improving adversarial robustness in deep learning models.
method Applying the Cayley transform to skew-symmetric convolutions in the Fourier domain.
result The proposed method preserves orthogonality and enhances adversarial robustness compared to existing techniques.
The paper defines and analyzes set-valued stochastic integrals for Lévy processes.
problem Defining and analyzing set-valued stochastic integrals for Lévy processes.
method Extending classical definitions to convoluted integrals with square-integrable kernels, and proving properties of set-valued convoluted stochastic integrals.
result Set-valued convoluted stochastic integrals can be explosive and take extended vector values.
Verifying robustness of neural network classifiers has attracted great interests and attention due to the success of deep neural networks and their unexpected vulnerability to adversarial perturbations. Although finding minimum adversarial distortion of neural networks (with ReLU activations) has been shown to be an NP…
Inspired by convolutional neural networks on 1D and 2D data, graph convolutional neural networks (GCNNs) have been developed for various learning tasks on graph data, and have shown superior performance on real-world datasets. Despite their success, there is a dearth of theoretical explorations of GCNN models such as t…
Exact bounds derived for neural network outputs with noisy inputs.
problem Bounding the output distribution of neural networks with random inputs.
method Applying ReLU NNs to derive bounds for general NNs, then using these to find exact error guarantees.
result Exact upper and lower bounds for the output distribution of neural networks with random inputs.
Paper analyzes GCNN sensitivity to probabilistic graph perturbations.
problem Investigating how GCNNs handle probabilistic graph errors.
method Establishes error bounds and linear relationships between GSO perturbations and GCNN outputs.
result GCNNs maintain stability under graph edge perturbations if GSO errors are bounded.
Improved generalization bounds for CNNs using Rademacher complexity.
problem Establishing non-vacuous generalization bounds for deep learning models.
method Rademacher complexity framework with novel contraction lemmas for high-dimensional mappings.
result Enhanced generalization bounds for a broader class of activation functions.
Paper analyzes the free energy of CNNs with skip connections in Bayesian learning.
problem Dependency of CNNs with skip connections on the number of parameters.
method Examines the Bayesian free energy of CNNs with and without skip connections.
result The upper bound of free energy of Bayesian CNN with skip connections does not depend on overparametrization.
The paper calculates bounds on the local Lipschitz constants of neural network layers.
problem Understanding the Lipschitz constants of neural network layers for robustness analysis.
method Analytical approach to determine upper bounds on local Lipschitz constants of affine-ReLU functions.
result The method produces tighter bounds than the standard conservative bound, especially for small perturbations.
Popular deep neural networks (DNNs) spend the majority of their execution time computing convolutions. The Winograd family of algorithms can greatly reduce the number of arithmetic operations required and is present in many DNN software frameworks. However, the performance gain is at the expense of a reduction in float…
Graph neural networks can be adapted to new graphs with a limit object called graphon NNs.
problem Transferability of graph neural networks across different graphs.
method Introduced graphon NNs as limit objects of GNNs and proved a bound on the difference between GNN and graphon-NN outputs.
result The bound on the difference between GNN and graphon-NN outputs vanishes with growing number of nodes if the graph convolutional filters are bandlimited.
Convolutional networks predict turbulence from wall quantities.
problem Predicting turbulence fields from wall-shear-stress components and wall pressure.
method Two CNN models: FCN and FCN-POD, trained on DNS data.
result FCN and FCN-POD models outperform EPOD in predicting turbulence fields.
A new convolutional spectral kernel network learns hierarchical and local features.
problem Lack of deep learning in non-stationary spectral kernels.
method Introduces convolutional filters and deep architectures into non-stationary spectral kernels, derives generalization error bounds, and introduces regularizers.
result Validated the effectiveness of the convolutional spectral kernel network on real-world datasets.
Local convolutions bias neural networks towards high-frequency adversarial examples.
problem High-frequency adversarial examples in neural networks.
method Analysis of different linear and nonlinear architectures, focusing on the impact of local convolution operations.
result Local convolutions induce an implicit bias towards high frequency features, leading to high-frequency adversarial examples.
Group Invariance, Stability to Deformations, and Complexity of Deep Convolutional Representationsstat.ML
The success of deep convolutional architectures is often attributed in part to their ability to learn multiscale and invariant representations of natural signals. However, a precise study of these properties and how they affect learning guarantees is still missing. In this paper, we consider deep convolutional represen…
Max-convolution is an important problem closely resembling standard convolution; as such, max-convolution occurs frequently across many fields. Here we extend the method with fastest known worst-case runtime, which can be applied to nonnegative vectors by numerically approximating the Chebyshev norm $\| \cdot \|_\infty…
A new neural network model reduces features in high-dimensional sequential data.
problem Exponential growth in features of truncated signature transform in high-dimensional data.
method Proposes a neural network model inspired by Convolutional Neural Networks to address feature growth.
result Reduces the number of features efficiently in a data-dependent way.
We show generalisation error bounds for deep learning with two main improvements over the state of the art. (1) Our bounds have no explicit dependence on the number of classes except for logarithmic factors. This holds even when formulating the bounds in terms of the -norm of the weight matrices, where previous bo…
Normalization layers are widely used in deep neural networks to stabilize training. In this paper, we consider the training of convolutional neural networks with gradient descent on a single training example. This optimization problem arises in recent approaches for solving inverse problems such as the deep image prior…
New method solves PDEs on spheres using physics-informed convolutional neural networks.
problem Solving PDEs on surfaces, especially spheres, with high accuracy and efficiency.
method Physics-informed convolutional neural networks (PICNN) with theoretical analysis and approximation results.
result Established fast convergence rates for PICNN solving PDEs on spheres.
Combinatorial optimization problems are typically tackled by the branch-and-bound paradigm. We propose a new graph convolutional neural network model for learning branch-and-bound variable selection policies, which leverages the natural variable-constraint bipartite graph representation of mixed-integer linear programs…