We seek to improve deep neural networks by generalizing the pooling operations that play a central role in current architectures. We pursue a careful exploration of approaches to allow pooling to learn and to adapt to complex and variable patterns. The two primary directions lie in (1) learning a pooling function via (…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “Pooling Operations”
In this work we compute lower Lipschitz bounds of pooling operators for as well as pooling operators preceded by half-rectification layers. These give sufficient conditions for the design of invertible neural network layers. Numerical experiments on MNIST and image patches confirm tha…
Graph Neural Networks (GNNs), which generalize deep neural networks to graph-structured data, have drawn considerable attention and achieved state-of-the-art performance in numerous graph related tasks. However, existing GNN models mainly focus on designing graph convolution operations. The graph pooling (or downsampli…
We propose a novel graph pooling operation using cliques as the unit pool. As this approach is purely topological, rather than featural, it is more readily interpretable, a better analogue to image coarsening than filtering or pruning techniques, and entirely nonparametric. The operation is implemented within graph con…
Develops a novel global pooling framework using optimal transport.
problem Sub-optimal performance in global pooling operations.
method Regularized Optimal Transport (ROT) for generalized global pooling.
result ROTP layers can improve performance in various machine learning scenarios.
Sound event detection (SED) methods are tasked with labeling segments of audio recordings by the presence of active sound sources. SED is typically posed as a supervised machine learning problem, requiring strong annotations for the presence or absence of each sound source at every time instant within the recording. Ho…
MultiRocket boosts TSC speed and accuracy with pooling and transformations.
problem Efficient time series classification with high accuracy.
method Multiple pooling operators and transformations applied to raw and differenced series.
result MultiRocket outperforms MiniRocket and is competitive with state-of-the-art methods in terms of accuracy and speed.
Convolution and pooling improve kernel methods in image classification.
problem Understanding the interplay between approximation and generalization in convolutional architectures.
method Characterized RKHS of kernels with convolution, pooling, and downsampling, computed generalization error.
result Convolution and pooling operations trade off approximation with generalization power.
Proposes SimPool for graph pooling using structural similarity features.
problem Challenges in graph pooling due to lack of spatial locality.
method Integrates structural similarity features with a revised pooling layer to propose SimPool.
result SimPool produces node cluster assignments resembling CNN's locality preserving pooling.
In most convolution neural networks (CNNs), downsampling hidden layers is adopted for increasing computation efficiency and the receptive field size. Such operation is commonly so-called pooling. Maximation and averaging over sliding windows (max/average pooling), and plain downsampling in the form of strided convoluti…
Graph neural networks, which generalize deep neural network models to graph structured data, have attracted increasing attention in recent years. They usually learn node representations by transforming, propagating and aggregating node features and have been proven to improve the performance of many graph related tasks…
Advanced methods of applying deep learning to structured data such as graphs have been proposed in recent years. In particular, studies have focused on generalizing convolutional neural networks to graph data, which includes redefining the convolution and the downsampling (pooling) operations for graphs. The method of …
In modern computer vision tasks, convolutional neural networks (CNNs) are indispensable for image classification tasks due to their efficiency and effectiveness. Part of their superiority compared to other architectures, comes from the fact that a single, local filter is shared across the entire image. However, there a…
New 3D protein analysis methods improve accuracy.
problem Lack of suitable learning algorithms for protein data.
method Intrinsic-Extrinsic Convolution and Pooling for 3D protein structures.
result Outperforms state-of-the-art methods on protein analysis tasks.
We introduce a simple and effective method for regularizing large convolutional neural networks. We replace the conventional deterministic pooling operations with a stochastic procedure, randomly picking the activation within each pooling region according to a multinomial distribution, given by the activities within th…
In graph neural networks (GNNs), pooling operators compute local summaries of input graphs to capture their global properties, and they are fundamental for building deep GNNs that learn hierarchical representations. In this work, we propose the Node Decimation Pooling (NDP), a pooling operator for GNNs that generates c…
Spectral clustering (SC) is a popular clustering technique to find strongly connected communities on a graph. SC can be used in Graph Neural Networks (GNNs) to implement pooling operations that aggregate nodes belonging to the same cluster. However, the eigendecomposition of the Laplacian is expensive and, since cluste…
PiNet improves graph classification efficiency and accuracy.
problem Graph level classification challenges.
method Attention-based pooling mechanism for graph convolution operations.
result Superior performance and high sample efficiency.
Graphon pooling preserves spectral properties in GNNs, reducing overfitting.
problem Unclear pooling and sampling strategies in GNNs that alter graph structure.
method Modeling graph layers as elements of a sequence converging to a graphon.
result Graphon pooling GNNs reduce overfitting and improve performance.
Polygonal meshes provide an efficient representation for 3D shapes. They explicitly capture both shape surface and topology, and leverage non-uniformity to represent large flat regions as well as sharp, intricate features. This non-uniformity and irregularity, however, inhibits mesh analysis efforts using neural networ…
Global pooling, such as max- or sum-pooling, is one of the key ingredients in deep neural networks used for processing images, texts, graphs and other types of structured data. Based on the recent DeepSets architecture proposed by Zaheer et al. (NIPS 2017), we introduce a Set Aggregation Network (SAN) as an alternative…
This paper explores vulnerabilities in hierarchical graph pooling neural networks for graph classification.
problem Vulnerability of hierarchical graph pooling neural networks in graph classification tasks.
method Proposes an adversarial attack framework using a surrogate model to generate adversarial samples.
result Adversarial samples can fool hierarchical GNN-based graph classification models, demonstrating their vulnerability.
Most of the banks' operational risk internal models are based on loss pooling in risk and business line categories. The parameters and outputs of operational risk models are sensitive to the pooling of the data and the choice of the risk classification. In a simple model, we establish the link between the number of ris…
Haar Graph Poolingcs.LG
Deep Graph Neural Networks (GNNs) are useful models for graph classification and graph-based regression tasks. In these tasks, graph pooling is a critical ingredient by which GNNs adapt to input graphs of varying size and structure. We propose a new graph pooling operation based on compressive Haar transforms -- HaarPo…
Proposes a graph pooling method leveraging node proximity for hierarchical graph representation learning.
problem Efficiently exploiting the geometry of graph data for hierarchical representation learning.
method Combines node proximity with kernel representation of topology and node features for adaptive node signal similarities evaluation.
result Achieves state-of-the-art performance on graph classification benchmark datasets.
With the advent of data science, the analysis of network or graph data has become a very timely research problem. A variety of recent works have been proposed to generalize neural networks to graphs, either from a spectral graph theory or a spatial perspective. The majority of these works however focus on adapting the …
Max-pooling architectures are theoretically analyzed and shown to be globally optimized and generalize well.
problem Theoretical understanding and optimization of max-pooling in deep learning architectures.
method Theoretical analysis of a convolutional max-pooling architecture, focusing on a pattern detection problem.
result Max-pooling architectures can be globally optimized and generalize well, even for highly over-parameterized models.
We reduce training time in convolutional networks (CNNs) with a method that, for some of the mini-batches: a) scales down the resolution of input images via downsampling, and b) reduces the forward pass operations via pooling on the convolution filters. Training is performed in an interleaved fashion; some batches unde…
Neural Architecture Search (NAS) is an emerging topic in machine learning and computer vision. The fundamental ideology of NAS is using an automatic mechanism to replace manual designs for exploring powerful network architectures. One of the key factors of NAS is to scale-up the search space, e.g., increasing the numbe…
The paper addresses instability in CNNs' first layer by proving max pooling's shift invariance.
problem Instability in CNNs' first layer, leading to sensitivity to small input shifts.
method Establishing conditions for max pooling's shift invariance and deriving a measure of stability.
result Max pooling approximates a nearly shift-invariant complex modulus under certain conditions.
Efficiently combines probabilistic predictions using kernel embeddings.
problem Combining multiple probabilistic predictions to improve forecast accuracy.
method Embed predictions into RKHS, optimizes kernel-based scoring rules, finds efficient implementation.
result Generalised linear pool outperforms traditional linear pool in operational wind speed forecasts.
Backtesting Framework for Concentrated Liquidity Market Makers on Uniswap V3 Decentralized Exchangeq-fin.MF
Backtesting framework for CLMMs on Uniswap V3 reduces reward estimation error.
problem Estimating rewards for CLMMs in Uniswap V3 liquidity pools.
method Parametric model for liquidity distribution, historical data analysis.
result Error in reward estimation less than 1% for each pool.
The paper integrates multiple Gaussian process predictions using Monte Carlo sampling.
problem Accurate prediction of variables using multiple models.
method Log-linear pooling of Gaussian process predictions, combined with Monte Carlo sampling.
result The log-linear pooling method improves prediction accuracy compared to linear pooling.
Many of our core assumptions about how neural networks operate remain empirically untested. One common assumption is that convolutional neural networks need to be stable to small translations and deformations to solve image recognition tasks. For many years, this stability was baked into CNN architectures by incorporat…
New MHSNs extract multiscale features from complex data for robust classification.
problem Signal classification and domain classification on complex data.
method Layered structure with multiscale basis dictionaries, pooling operations, and invariant features.
result High-accuracy classification with fewer parameters than traditional graph neural networks.
Interpool solves interoperability issues by minting, exchanging, and burning tokens within a single liquidity pool.
problem Lack of proper interoperability in blockchain use cases.
method Interpool operates as a standalone liquidity pool that mints, exchanges, and burns tokens, optimizing the order of transactions in the mempool.
result Interpool transforms front-running issues into a solution that ensures ultimate liquidity through a burning procedure, enabling trustless design.
We introduce general scattering transforms as mathematical models of deep neural networks with l2 pooling. Scattering networks iteratively apply complex valued unitary operators, and the pooling is performed by a complex modulus. An expected scattering defines a contractive representation of a high-dimensional probabil…
PAN uses path integrals for graph convolution and pooling, improving GNN performance.
problem Designing efficient graph convolution and pooling for graph neural networks.
method Path integral based graph convolution and pooling using learnable weights for path lengths.
result PAN achieves state-of-the-art performance on various graph classification/regression tasks.
Pyramidal GNN combines RC and pooling for efficient graph embeddings.
problem Efficiently embedding graphs while maintaining accuracy.
method Alternates RC layers with pooling to reduce complexity.
result Formally shows how pooling reduces complexity and speeds convergence.
According to the Loss Distribution Approach, the operational risk of a bank is determined as 99.9% quantile of the respective loss distribution, covering unexpected severe events. The 99.9% quantile can be considered a tail event. As supported by the Pickands-Balkema-de Haan Theorem, tail events exceeding some high thr…
WavPool improves deep neural networks with wavelet-based pooling.
problem Improving efficiency and performance of deep neural networks.
method Introducing WavPool, a wavelet-transform-based pooling layer.
result WavPool outperforms existing network architectures by 10% on CIFAR-10.
Convolutional neural networks (CNNs) have achieved remarkable performance in many applications, especially in image recognition tasks. As a crucial component of CNNs, sub-sampling plays an important role for efficient training or invariance property, and max-pooling and arithmetic average-pooling are commonly used sub-…
Graph U-Netscs.LG
We consider the problem of representation learning for graph data. Convolutional neural networks can naturally operate on images, but have significant challenges in dealing with graph data. Given images are special cases of graphs with nodes lie on 2D lattices, graph embedding tasks have a natural correspondence with i…
The study quantifies how many objects can be linearly classified under all views.
problem Understanding the expressivity of group-equivariant representations.
method Generalization of Cover's Function Counting Theorem to quantify separable dichotomies.
result The fraction of separable dichotomies is determined by the fixed space dimension of the group action.
Global catastrophe risk pools increase financial resilience by diversifying risk and including more countries.
problem Low- to middle-income countries rely heavily on foreign aid for recovery from extreme weather events, which is slow and uncertain.
method Developed a method to form global catastrophe risk pools that maximize risk diversification and select countries with low bilateral correlations or low shares in the pool risk.
result Global pooling increases risk diversification, lowers countries' shares in the pool risk, and increases the number of countries benefiting from risk pooling.
New image classifier uses hierarchical max-pooling with local pooling.
problem Improving image classification accuracy with variable spatial relationships.
method Introduces a hierarchical max-pooling model with additional local pooling for convolutional neural networks.
result Demonstrates improved performance in estimating image features.
Are Neural Architecture Search Benchmarks Well Designed? A Deeper Look Into Operation Importancecs.LG
This study analyzes NAS benchmarks and finds that only a subset of operations is crucial for generating high-performing architectures.
problem NAS benchmarks lack generability and provide skewed performance distributions, leading to unreliable comparisons.
method Empirical analysis of widely used NAS benchmarks (101, 201, TransNAS-Bench-101) focusing on operation importance and generability.
result Only a subset of operations is necessary to generate architectures close to the upper-bound performance range, and convolution layers have the highest impact.
Two-stage recommender systems show better performance when components interact rather than operate independently.
problem Two-stage recommender systems are often treated as sums of their parts, ignoring interactions between components.
method Used synthetic and real-world data to demonstrate interactions between ranker and nominators. Derived a generalization lower bound and proposed a Mixture-of-Experts approach to learn optimal item pools.
result Independent nominator training can lead to performance on par with random recommendations, highlighting the importance of interactions.