Paper tackles fixed-size representation learning for variable-sized signatures.
problem Learning feature representations for signatures of varying sizes.
method Modified Spatial Pyramid Pooling to learn fixed-sized representations from variable-sized signatures.
result Comparable performance to state-of-the-art on GPDS dataset, removing size constraint.
The softmax content-based attention mechanism has proven to be very beneficial in many applications of recurrent neural networks. Nevertheless it suffers from two major computational limitations. First, its computations for an attention lookup scale linearly in the size of the attended sequence. Second, it does not enc…
Paper proposes FOFE for efficient WSD.
problem Word sense disambiguation (WSD) problem.
method Fixed-size ordinally forgetting encoding (FOFE) combined with FFNN.
result FOFE-based FFNN achieves comparable performance to state-of-the-art at lower cost.
Local nonparametric meta-learning improves meta-generalization across tasks.
problem Meta-learning struggles with global inductive biases and out-of-distribution tasks.
method Proposes a local, nonparametric meta-learning algorithm using meta-trained local learning rules.
result Improved meta-generalization and state-of-the-art results in robotics benchmarks.
Deep neural networks can approximate complex functions through repeated compositions of a fixed-size ReLU network.
problem Understanding the expressive power of deep neural networks through function compositions.
method Demonstrated the surprising expressive power of repeated compositions of a single fixed-size ReLU network.
result Repeated compositions of a single fixed-size ReLU network can approximate 1-Lipschitz continuous functions on [0,1]d with an error O(r−1/d). New activation functions achieve arbitrary-accuracy Sobolev approximation by fixed-size neural networks.
problem Approximation of Sobolev functions by neural networks
method Elementary Universal Activation Function and Differentiable Universal Activation Functions
result Arbitrary-accuracy Sobolev approximation by fixed-size neural networks
Capsule Neural Networks classify graphs from categorical features and relationships.
problem Graph classification in scientific domains, especially with varying graph sizes and features.
method Explicit tensor representations, Capsule Network for classification.
result Capsule Network model performs competitively with state-of-the-art models.
New findings show equivalent inclusion probabilities for fixed-size and varying-size DPPs as set size increases.
problem Computational challenges in fixed-size DPPs and their equivalence to varying-size DPPs.
method Analysis of inclusion probabilities and maximum likelihood estimators for k-DPPs and DPPs. result Inclusion probabilities of k-DPPs and DPPs converge as set size grows, suggesting equivalent models. Improves GCN by sampling neighbors and features for better node representation.
problem GCN's aggregation process treats all neighbors and features equally, leading to suboptimal node representations.
method Introduces a new convolution operation on feature maps constructed from a fixed node bandwidth, then passes to a standard GCN.
result Outperforms competing methods in semi-supervised node classification tasks.
Enhances graph classification by adding virtual nodes to represent latent graph aspects.
problem Graph classification challenges in representing latent graph aspects not directly available from attributes and connectivity.
method Introducing virtual nodes bidirectionally connected to all existing nodes, then using Column Network for representation.
result Virtual Column Network (VCN) outperforms existing methods in bioactivity prediction and vulnerability detection.
Formula found for minimum ARI between clusterings of fixed sizes.
problem Understanding the lowest possible agreement between clusterings.
method Explicit formula derivation for minimum ARI.
result A specific pair of clusterings achieving the minimum ARI is provided.
PiNet learns graph representations invariant to node permutations.
problem Graph classification and representation learning invariant to node permutations.
method Differentiable node attention pooling, permutation invariant graph neural network.
result Significant accuracy improvement in isomorphic graph classification with limited training data.
New model learns graph features for classification.
problem Graph classification with structural information loss.
method Transform graphs into vertex grids, apply vertex convolution.
result Model preserves structural information on local vertices.
EMDE efficiently estimates manifold densities for diverse recommendation systems.
problem Efficiently estimating manifold densities for multi-modal recommendation systems.
method EMDE (Efficient Manifold Density Estimator) framework for arbitrary vector representations.
result Established new state-of-the-art results in top-k and session-based recommendation settings.
Unsupervised model learns word and context embeddings from character sequences.
problem Learning meaningful word and context embeddings from unlabeled data.
method Character-aware neural architecture that jointly learns word and context embeddings.
result Compact encoders achieve high performance in downstream tasks.
MAGIC generates image collages from set templates using attention and set representations.
problem Generating image collages from set templates is challenging for classical models.
method Memory Attentive Generation of Image Collages (MAGIC) using Set-Transformer layers and set-pooling.
result MAGIC can generate image collages from set templates in one forward pass.
Improved batched SH algorithm maintains original performance.
problem Maintaining performance in batched multi-armed bandits.
method Simple batch version of Sequential Halving algorithm.
result Batching does not degrade SH algorithm's performance.
New neural network models learn symmetric functions of varying input sizes.
problem Learning symmetric functions with varying input sizes.
method Functional perspective on neural networks, treating symmetric functions as functions over probability measures.
result Established approximation and generalization bounds for shallow architectures that extend across input sizes.
Expectation maximization (EM) has recently been shown to be an efficient algorithm for learning finite-state controllers (FSCs) in large decentralized POMDPs (Dec-POMDPs). However, current methods use fixed-size FSCs and often converge to maxima that are far from optimal. This paper considers a variable-size FSC to rep…
A new method converts complex data patterns into simpler vectors for machine learning.
problem Complex persistence diagrams are hard to use in machine learning.
method Persistence codebooks, adapting BoW, VLAD, and FV for PDs.
result Persistence codebooks improve machine learning performance and speed.
Bounds on VC dimension for 1NN classifiers with fixed prototype sets.
problem No theoretical results for VC dimension of 1NN classifiers with fixed size prototype sets.
method Collected and used relevant theoretical results to provide explicit lower and upper bounds.
result Explicit lower and upper bounds for VC dimension of 1NN classifiers with fixed prototype set size.
Machine learning improves jet charge classification.
problem Classifying jets according to their electric charge.
method Convolutional, recurrent, and recursive neural networks, including distance within the jet and clustering history.
result Significant improvement in jet charge extraction over traditional methods.
We discuss our recent work [4] in which gravitational radiation was studied by evaluating the Wang-Yau quasi-local mass of surfaces of fixed size at the infinity of both axial and polar perturbations of the Schwarzschild spacetime, à la Chandrasekhar [1].
Study compares tessellation strategies for taxi demand-supply forecasting models.
problem Improving taxi demand-supply forecasting using neural networks.
method Compared Voronoi tessellation and Geohash tessellation for LSTM models.
result Variable-sized polygon tessellation yields superior performance in LSTM models.
In this paper, we study the global geometry of complete, constant mean curvature hypersurfaces embedded in n-manifolds. More precisely, we give conditions that imply properness of such surfaces and prove the existence of fixed size one-sided regular neighborhoods for certain constant mean curvature hypersurfaces in cer…
New method speeds up sparse graph neural networks training on dense hardware.
problem Training sparse graph neural networks is slow on custom hardware.
method Inspired by sparse matrix optimization, developed techniques for dense hardware.
result Sparse graph neural networks trained in 13 minutes on 512-core TPUv2 Pod.
gLSTM improves graph neural networks by increasing storage capacity to prevent over-squashing.
problem Over-squashing in GNNs collapses information from a large receptive field into a single vector, creating an information bottleneck.
method Introduced a new synthetic task to measure over-squashing and adapted ideas from sequence modeling to develop gLSTM, a novel GNN architecture with improved capacity.
result gLSTM architecture demonstrates strong performance on synthetic and real-world graph benchmarks, mitigating over-squashing.
Improved taxi demand-supply forecasts using graph-based LSTM.
problem Accurate taxi demand-supply forecasting with complex spatial and temporal patterns.
method Investigated impact of spatial partitioning techniques (Voronoi vs. Geohash) on LSTM network performance.
result GraphLSTM offers competitive performance against ConvLSTM, at lower complexity, across real-world data sets.
A scalable online method for Gaussian processes that improves decision-making in various applications.
problem Scalability issues with Gaussian processes for online decision-making.
method Online variational conditioning (OVC) for SVGPs.
result OVC enables efficient online learning and decision-making with SVGPs.
Develops BASGCN for graph classification with improved feature learning.
problem Graph classification with information loss and imprecise representation.
method Transforms graphs into grid structures and defines a new spatial graph convolution operation.
result Reduces information loss and improves feature representation compared to existing models.
Reinforcement Patching optimizes dynamic sequence patching for efficient time series forecasting.
problem Efficiently learning data-adaptive representations for long-horizon sequence data, especially continuous sequences.
method Reinforcement Patching (ReinPatch) uses reinforcement learning to optimize dynamic patching policies and sequence backbones.
result ReinPatch achieves compelling performance in time-series forecasting compared to state-of-the-art methods.
Introduces Spectral Attention for better long-range time series forecasting.
problem Challenges in capturing long-range dependencies in time series forecasting.
method Spectral Attention mechanism that preserves temporal correlations and long-range dependencies.
result Achieves state-of-the-art results on 11 real-world time series datasets.
WEGL embeds graphs in a vector space for faster machine learning.
problem Efficiently embedding graphs for machine learning tasks.
method Wasserstein distance for node embedding similarity, Monge maps for graph representation.
result State-of-the-art classification performance with superior computational efficiency.
Proposes a new QSGCNN model for graph classification.
problem Information loss and imprecise representation in existing GCN models.
method Quantum Spatial Graph Convolutional Neural Network (QSGCNN) model.
result Demonstrates effectiveness on benchmark graph classification datasets.
StrGNN detects anomalies in dynamic graphs by analyzing subgraphs and temporal features.
problem Detecting anomalies in dynamic graphs with structural changes.
method StrGNN is an end-to-end model that uses structural subgraphs and temporal features for anomaly detection.
result StrGNN effectively detects anomalies in dynamic graphs, as shown by extensive experiments.
MR-GNN predicts interactions between structured entities using multi-resolution and dual graph neural networks.
problem Predicting interactions between structured entities, especially considering features in substructures of different sizes and interactions between entities.
method MR-GNN uses a multi-resolution architecture and dual graph-state L-STMs to extract features from different neighborhoods and pairwise graphs, respectively.
result MR-GNN improves prediction accuracy compared to state-of-the-art methods.
Bayesian optimization improves efficiency with semi-supervised learning.
problem Efficiently find global optima of expensive functions.
method Density ratio estimation combined with semi-supervised learning.
result Improved accuracy in identifying global optima with unlabeled data.
This paper improves RNN memory capacity for long sequences through learning associative memory update rules.
problem Challenges in RNNs remembering long sequences.
method Jointly learns memory update rule with task objective and uses multiple associative memories.
result Improves memory capacity for long sequence encoding.
In this paper, we focus on online representation learning in non-stationary environments which may require continuous adaptation of model architecture. We propose a novel online dictionary-learning (sparse-coding) framework which incorporates the addition and deletion of hidden units (dictionary elements), and is inspi…
We introduce a deep learning architecture for structure-based virtual screening that generates fixed-sized fingerprints of proteins and small molecules by applying learnable atom convolution and softmax operations to each compound separately. These fingerprints are further transformed non-linearly, their inner-product …
Deep neural nets predict aircraft flight paths from weather data.
problem Accurate prediction of aircraft trajectories for aviation efficiency.
method Deep generative convolutional recurrent neural network with tree-based matching.
result Model accurately predicts aircraft flight paths from weather data.
The discrete Nahm equations, a system of matrix valued difference equations, arose in the work of Braam and Austin on half-integral mass hyperbolic monopoles. We show that the discrete Nahm equations are completely integrable in a natural sense: to any solution we can associate a spectral curve and a holomorphic line-b…
OMD monitors stock market dynamics through matrix trajectories and reveals crisis patterns.
problem Understanding and predicting stock market crises and sector rotations.
method Applying OMD to S\&P 500 returns over three crises, analyzing distance matrices and their spectra.
result Market dynamics show coherent changes during crises, with distinct sector leadership.
This paper considers binomial approximation of continuous time stochastic processes. It is shown that, under some mild integrability conditions, a process can be approximated in mean square sense and in other strong metrics by binomial processes, i.e., by processes with fixed size binary increments at sampling points. …
OMD monitors stock market dynamics through matrix trajectories, revealing crisis patterns and sector rotations.
problem Understanding and predicting stock market dynamics during crises.
method Applying OMD to S&P 500 returns over three crises, analyzing distance matrices and their spectra.
result Market dynamics show coherent changes during crises, with sector-specific patterns and volatility clustering.
Paper develops efficient method for probability estimation.
problem Estimating probabilities with high efficiency.
method Adaptive Monte Carlo estimation using truncated inverse binomial sampling.
result Proposed method is orders of magnitude more efficient.
Optimal transport embedding learns feature sets efficiently.
problem Learning on sets of features with long-range dependencies and few labeled data.
method Parametrized fixed-size embedding that aggregates features according to optimal transport plan.
result Achieves state-of-the-art results on protein fold recognition and chromatin profiles.
There are two important statements regarding the Trautman-Bondi mass [1,8,5] at null infinity: one is the positivity [7,6], and the other is the Bondi mass loss formula [1], which are both global in nature. The positivity of the quasi-local mass can potentially lead to a local description at null infinity. This is conf…