RPN reduces communication costs in federated learning.
problem Efficient communication in federated learning with complex models.
method Residual Pooling Network (RPN) for compression.
result RPN achieves similar performance with reduced data transmission.
Simpler CNN model with spatial attention and temporal pooling outperforms complex models.
problem Emotion recognition from videos with small face deformations and identity variations.
method Spatial attention mechanism and temporal softmax pooling applied to a pre-trained CNN.
result The approach achieves higher accuracy than state-of-the-art methods on the EmotiW dataset.
Faster CNN training with downsampling and pooling.
problem Reducing training time in CNNs.
method Interleaved training with downsampling and pooling.
result Up to 23% reduction in training time with minimal loss.
An emerging design principle in deep learning is that each layer of a deep artificial neural network should be able to easily express the identity transformation. This idea not only motivated various normalization techniques, such as \emph{batch normalization}, but was also key to the immense success of \emph{residual …
The paper introduces a diagnostic method to detect grokking transitions in models before test accuracy improves.
problem Detecting the transition from training to generalization in machine learning models.
method Summarize task-dependent observables as empirical distributions, map them to Wasserstein/quantile coordinates, and analyze using Hankel dynamic mode decomposition.
result The diagnostic method achieves AUROC \(\approx\) 0.93 for grokking-vs-non-grokking discrimination at the run level.
MLP residual networks implement a selective coarse-graining procedure governed by the spectral structure of the input distribution.
problem Understanding the coarse-graining procedure in MLP residual networks
method Analyzing a pure MLP residual stack on synthetic Markov chain sequences
result MLP residual networks implement a selective coarse-graining procedure governed by the spectral structure of the input distribution
Framework scores DeFi users based on liquidity and trading behavior.
problem Distinguishing between liquidity provision and active trading in DeFi.
method Rule-based decomposition, deep residual neural network, pool-level context.
result Deep residual neural network improves user scoring and risk assessment.
CSP improves time-series forecasting without training, outperforming DeepNPTS in speed and accuracy.
problem Improving probabilistic time-series forecasting without training.
method Mixing empirical and residual draws around a seasonal naive forecast.
result CSP significantly outperforms DeepNPTS on CRPS, normalized mean quantile loss, and coverage metrics.
Convolutional neural networks (CNNs) have achieved remarkable performance in many applications, especially in image recognition tasks. As a crucial component of CNNs, sub-sampling plays an important role for efficient training or invariance property, and max-pooling and arithmetic average-pooling are commonly used sub-…
Global catastrophe risk pools increase financial resilience by diversifying risk and including more countries.
problem Low- to middle-income countries rely heavily on foreign aid for recovery from extreme weather events, which is slow and uncertain.
method Developed a method to form global catastrophe risk pools that maximize risk diversification and select countries with low bilateral correlations or low shares in the pool risk.
result Global pooling increases risk diversification, lowers countries' shares in the pool risk, and increases the number of countries benefiting from risk pooling.
In most convolution neural networks (CNNs), downsampling hidden layers is adopted for increasing computation efficiency and the receptive field size. Such operation is commonly so-called pooling. Maximation and averaging over sliding windows (max/average pooling), and plain downsampling in the form of strided convoluti…
New image classifier uses hierarchical max-pooling with local pooling.
problem Improving image classification accuracy with variable spatial relationships.
method Introduces a hierarchical max-pooling model with additional local pooling for convolutional neural networks.
result Demonstrates improved performance in estimating image features.
Convolutional Neural Networks (CNNs) have been recently introduced in the domain of session-based next item recommendation. An ordered collection of past items the user has interacted with in a session (or sequence) are embedded into a 2-dimensional latent matrix, and treated as an image. The convolution and pooling op…
We seek to improve deep neural networks by generalizing the pooling operations that play a central role in current architectures. We pursue a careful exploration of approaches to allow pooling to learn and to adapt to complex and variable patterns. The two primary directions lie in (1) learning a pooling function via (…
Optimizes diversification in catastrophe risk pooling using asymptotic analysis.
problem Maximizing diversification benefit from catastrophic events in insurance pools.
method Asymptotic analysis to solve high-dimensional optimization problem.
result Derives an asymptotically optimal pool that approximates practical optimal pool.
We decompose the squared price-of-risk premium into three components: intervention-stable premium, confounding wedge, and information loss.
problem Decomposing the squared price-of-risk premium into its components
method Identifying an order-three obstruction to aggregation across portfolios
result The decomposition is estimable and detectable with a permutation-calibrated screen
An efficient algorithm identifies labels from sparse pooled data.
problem Identify unknown labels from condensed measurements of items.
method Design an efficient algorithm based on novel pooling scheme.
result Resolves statistical-to-computational gap in minimum pools needed.
This paper investigates the impact of dark pools on price discovery (the efficiency of prices on stock exchanges to aggregate information). Assets are traded in either an exchange or a dark pool, with the dark pool offering better prices but lower execution rates. Informed traders receive noisy and heterogeneous signal…
This research simplifies lending pools in decentralized finance for better understanding and security.
problem Complexity and lack of executable models make lending pools hard to understand and predict.
method Developed a formal model to reflect common features of lending pools and proved general properties.
result Proved correct handling of funds and described vulnerabilities and attacks.
Graph Neural Network (GNN) research has concentrated on improving convolutional layers, with little attention paid to developing graph pooling layers. Yet pooling layers can enable GNNs to reason over abstracted groups of nodes instead of single nodes. To close this gap, we propose a graph pooling layer relying on the …
New regularizer improves neural network robustness and generalization.
problem Ineffective weight decay for networks with homogeneous activation functions.
method Proposes an invariant regularizer to penalize intrinsic weight norms.
result Improves generalization and adversarial robustness on various datasets.
Develops a novel global pooling framework using optimal transport.
problem Sub-optimal performance in global pooling operations.
method Regularized Optimal Transport (ROT) for generalized global pooling.
result ROTP layers can improve performance in various machine learning scenarios.
We consider interactive algorithms in the pool-based setting, and in the stream-based setting. Interactive algorithms observe suggested elements (representing actions or queries), and interactively select some of them and receive responses. Pool-based algorithms can select elements at any order, while stream-based algo…
In this work we compute lower Lipschitz bounds of ℓp pooling operators for p=1,2,∞ as well as ℓp pooling operators preceded by half-rectification layers. These give sufficient conditions for the design of invertible neural network layers. Numerical experiments on MNIST and image patches confirm tha…
ASAP improves graph pooling for hierarchical graph representations.
problem Pooling in graphs fails to effectively capture substructure or scale to large graphs.
method ASAP uses self-attention and modified GNN to capture node importance and learn sparse soft cluster assignments.
result Combining ASAP with GNN architectures leads to state-of-the-art results on graph classification benchmarks.
We propose a novel graph pooling operation using cliques as the unit pool. As this approach is purely topological, rather than featural, it is more readily interpretable, a better analogue to image coarsening than filtering or pruning techniques, and entirely nonparametric. The operation is implemented within graph con…
HGP-SL pools and learns graph structure for hierarchical representation learning.
problem Graph pooling is overlooked in GNN models, limiting hierarchical representation learning.
method Integrates graph pooling and structure learning into a unified module.
result HGP-SL improves graph classification performance on benchmarks.
TL-ANDI distills context from source data to improve transfer learning for TFMs.
problem Limited transfer learning due to context-size constraints and distribution shifts.
method TL-ANDI uses posterior-aware distillation to construct a compact source context and locally distills labels.
result Improves transfer performance by addressing context-size and distribution shifts.
Proposes a graph pooling method leveraging node proximity for hierarchical graph representation learning.
problem Efficiently exploiting the geometry of graph data for hierarchical representation learning.
method Combines node proximity with kernel representation of topology and node features for adaptive node signal similarities evaluation.
result Achieves state-of-the-art performance on graph classification benchmark datasets.
EarnMore uses masked stock representations to train RL agents for customizable stock pools efficiently.
problem Training RL agents for customizable stock pools (CSPs) is computationally expensive and unstable.
method EarnMore introduces a mechanism to mask out stocks outside the target pool, learns meaningful stock representations, and uses a re-weighting mechanism to focus on favorable stocks.
result EarnMore significantly outperforms state-of-the-art baselines in profit metrics with over 40% improvement.
Global pooling, such as max- or sum-pooling, is one of the key ingredients in deep neural networks used for processing images, texts, graphs and other types of structured data. Based on the recent DeepSets architecture proposed by Zaheer et al. (NIPS 2017), we introduce a Set Aggregation Network (SAN) as an alternative…
Graph neural networks, which generalize deep neural network models to graph structured data, have attracted increasing attention in recent years. They usually learn node representations by transforming, propagating and aggregating node features and have been proven to improve the performance of many graph related tasks…
We explore the application of end-to-end stateless temporal modeling to small-footprint keyword spotting as opposed to recurrent networks that model long-term temporal dependencies using internal states. We propose a model inspired by the recent success of dilated convolutions in sequence modeling applications, allowin…
Study characterizes Uniswap v3 liquidity pools using transaction graphs and identifies ideal trading conditions.
problem Computational expense in analyzing the full Uniswap v3 ecosystem.
method Extracted and analyzed a sub-universe of liquidity pools, using transaction graphs and graph2vec algorithm.
result Identified seven clusters of liquidity takers with similar trading preferences and introduced an ideal crypto law.
Optimal rebalancing strategy improves AMM pool performance by 25%.
problem Optimizing the sequence of weights in dynamic AMM pools to minimize rebalancing costs.
method Using optimal interpolation and a cheap-to-compute approximation to achieve nearly optimal rebalancing.
result Approximately-optimal weight changes lead to significant increases in pool performance (up to 25%) under various conditions.
High-fee pools attract more liquidity but execute less volume; low-fee pools have more stable LPs.
problem Optimal liquidity supply and execution on decentralized exchanges with fixed gas costs.
method Analysis of Uniswap data to compare high- and low-fee pools.
result Fragmented liquidity leads to more LPs and competition, improving overall market efficiency.
Study optimal liquidation strategies in lit and dark pools with and without regulation.
problem Optimal liquidation strategies in dark and lit pools with execution uncertainty.
method Design optimal make-take fee policies, solve HJB-Fokker-Planck systems, use BSDEs.
result Explicit solutions for optimal strategies in both competitive and regulated markets.
Dual model predicts electricity spot prices using neural networks and wavelets.
problem Forecasting hourly electricity spot prices.
method Dual generalized long memory modelling with k-factor GARMA and G-GARCH models, using LLWNN and PSO for variance prediction.
result The hybrid k-factor GARMA-LLWNN model outperforms other methods in forecasting accuracy.
We present a theoretically grounded approach to train deep neural networks, including recurrent networks, subject to class-dependent label noise. We propose two procedures for loss correction that are agnostic to both application domain and network architecture. They simply amount to at most a matrix inversion and mult…
SpaPool combines dense and sparse techniques for efficient graph pooling.
problem Efficiently processing large graphs in graph neural networks.
method Adaptive clustering of graph vertices into clusters.
result SpaPool outperforms existing methods on small-scale graphs.
The paper discusses a pooling mechanism to induce subsampling in graph structured data and introduces it as a component of a graph convolutional neural network. The pooling mechanism builds on the Non-Negative Matrix Factorization (NMF) of a matrix representing node adjacency and node similarity as adaptively obtained …
Improved forecasting of investment dynamics across heterogeneous panels using a two-stage model.
problem Forecasting investment dynamics in heterogeneous panels with varying dynamics.
method Two-stage architecture: global pooled AR(1) for shared persistence, local models for residual dynamics.
result Significant improvement in out-of-sample R2 from 0.630 to 0.677, with a gain of 0.047. This research improves capital efficiency and impermanent loss in cryptocurrency markets using multi-token trading pools.
problem Poor impermanent loss and capital efficiency in automated market makers.
method Analysis and construction of a multi-token token proactive market maker (MPMM).
result MPMM shows better impermanent loss and capital efficiency than comparable market makers.
Study on pooled annuity funds and how initial savings affect income stability.
problem Analyzing the stability of income payments in pooled annuity funds.
method Examining the influence of initial savings on income fluctuations and developing a criterion for pooling funds.
result Identification of a term, the 'implied number of homogeneous members', linking initial savings to income fluctuations.
We propose a max-pooling based loss function for training Long Short-Term Memory (LSTM) networks for small-footprint keyword spotting (KWS), with low CPU, memory, and latency requirements. The max-pooling loss training can be further guided by initializing with a cross-entropy loss trained network. A posterior smoothin…
Proposes SimPool for graph pooling using structural similarity features.
problem Challenges in graph pooling due to lack of spatial locality.
method Integrates structural similarity features with a revised pooling layer to propose SimPool.
result SimPool produces node cluster assignments resembling CNN's locality preserving pooling.
Recently, with convolutional neural networks gaining significant achievements in many challenging machine learning fields, hand-crafted neural networks no longer satisfy our requirements as designing a network will cost a lot, and automatically generating architectures has attracted increasingly more attention and focu…
Paper proposes a new method for more accurate group testing of infected patients.
problem Identifying infected patients efficiently with reduced tests and corrected errors.
method Adaptive design of pools based on Bayesian posterior prediction using belief propagation algorithm.
result The proposed method results in more accurate identification of infected patients.