GP-DRF model handles variable-sized input and learns deep features.
problem Scaling deep Gaussian processes for variable-sized data.
method GP-DRF model combining GPs and DRF layers for efficient inference.
result GP-DRF outperforms standard GP and DRF models across various datasets.
Paper introduces new bounds linking data compressibility to generalization error.
problem Establishing data-dependent generalization bounds.
method Variable-size compressibility framework linking generalization error to compression rate of input data.
result New bounds depend on empirical data measure, subsuming existing PAC-Bayes and intrinsic dimension bounds.
We introduce a new neural architecture to learn the conditional probability of an output sequence with elements that are discrete tokens corresponding to positions in an input sequence. Such problems cannot be trivially addressed by existent approaches such as sequence-to-sequence and Neural Turing Machines, because th…
New hypergraph neural network learns variable-sized hyperedges.
problem Learning representations for non-uniform hypergraphs with variable cardinalities.
method Developed a hypergraph neural network exploiting incidence structure.
result Significant improvement in accuracy on real-world hypergraph datasets.
Janossy pooling averages permutation-sensitive functions over all sequences to create invariant functions.
problem Creating deep, invariant functions for variable-size inputs.
method Janossy pooling: average permutation-sensitive functions over all reorderings.
result Improved performance over state-of-the-art methods.
Study compares tessellation strategies for taxi demand-supply forecasting models.
problem Improving taxi demand-supply forecasting using neural networks.
method Compared Voronoi tessellation and Geohash tessellation for LSTM models.
result Variable-sized polygon tessellation yields superior performance in LSTM models.
We address a largely open problem of multilabel classification over graphs. Unlike traditional vector input, a graph has rich variable-size substructures which are related to the labels in some ways. We believe that uncovering these relations might hold the key to classification performance and explainability. We intro…
We present a memory augmented neural network for natural language understanding: Neural Semantic Encoders. NSE is equipped with a novel memory update rule and has a variable sized encoding memory that evolves over time and maintains the understanding of input sequences through read}, compose and write operations. NSE c…
Two Bayesian optimization methods tackle dynamic design spaces with mixed variables.
problem Optimizing complex systems with varying numbers and types of variables and constraints.
method Two Bayesian optimization approaches: budget allocation and kernel function.
result Both methods converge faster and more consistently than standard approaches.
Expectation maximization (EM) has recently been shown to be an efficient algorithm for learning finite-state controllers (FSCs) in large decentralized POMDPs (Dec-POMDPs). However, current methods use fixed-size FSCs and often converge to maxima that are far from optimal. This paper considers a variable-size FSC to rep…
Proposes a new optimization-based method for aggregating sets in neural networks.
problem Limited representational power of existing aggregation methods.
method Equilibrium Aggregation: an optimization-based approach.
result Equilibrium Aggregation outperforms existing methods in various tasks.
Deep Sets improve reinforcement learning for autonomous driving with variable inputs.
problem Optimal decision making in autonomous driving with varying number of objects.
method Employed Deep Sets for high-level decision making in reinforcement learning.
result Deep Sets outperform other approaches in performance and generalization.
Sequences have become first class citizens in supervised learning thanks to the resurgence of recurrent neural networks. Many complex tasks that require mapping from or to a sequence of observations can now be formulated with the sequence-to-sequence (seq2seq) framework which employs the chain rule to efficiently repre…
RS-Del provides robustness for sequence classifiers against edit distance attacks.
problem Certifying robustness of discrete sequence classifiers against edit distance attacks.
method Randomized deletion (RS-Del) for discrete sequence classifiers, focusing on edit distance-bounded adversaries.
result Achieved a certified accuracy of 91% at an edit distance radius of 128 bytes on malware detection.
Improved taxi demand-supply forecasts using graph-based LSTM.
problem Accurate taxi demand-supply forecasting with complex spatial and temporal patterns.
method Investigated impact of spatial partitioning techniques (Voronoi vs. Geohash) on LSTM network performance.
result GraphLSTM offers competitive performance against ConvLSTM, at lower complexity, across real-world data sets.
Bin Packing problems have been widely studied because of their broad applications in different domains. Known as a set of NP-hard problems, they have different vari- ations and many heuristics have been proposed for obtaining approximate solutions. Specifically, for the 1D variable sized bin packing problem, the two ke…
A new distance for mixed-variable, hierarchical datasets with meta variables.
problem Heterogeneous datasets limit generalizability and performance in machine learning and optimization.
method Developed a modeling framework for mixed-variable and hierarchical domains with meta variables, and a novel distance function.
result The novel distance function allows comparison of heterogeneous datasets, improving model performance.
Methods for learning feature representations for Offline Handwritten Signature Verification have been successfully proposed in recent literature, using Deep Convolutional Neural Networks to learn representations from signature pixels. Such methods reported large performance improvements compared to handcrafted feature …
Text segmentation plays an important role in various Natural Language Processing (NLP) tasks like summarization, context understanding, document indexing and document noise removal. Previous methods for this task require manual feature engineering, huge memory requirements and large execution times. To the best of our …
Study compares RL and SL for TSP, finds RL better for variable graph sizes.
problem Training deep neural networks for the Travelling Salesman Problem.
method Controlled experiments with supervised and reinforcement learning models on fixed and variable sized graphs.
result Reinforcement learning leads to better generalization to variable graph sizes.
Unordered feature sets are a nonstandard data structure that traditional neural networks are incapable of addressing in a principled manner. Providing a concatenation of features in an arbitrary order may lead to the learning of spurious patterns or biases that do not actually exist. Another complication is introduced …
New method makes machine learning approximations unbiased and efficient.
problem Efficient sampling of complex probability distributions.
method Uses autoregressive neural networks with cluster updates and physical symmetries.
result Shows unbiased and low-variance approximations for phase transitions.
Persistent homology (PH) is a rigorous mathematical theory that provides a robust descriptor of data in the form of persistence diagrams (PDs) which are 2D multisets of points. Their variable size makes them, however, difficult to combine with typical machine learning workflows. In this paper we introduce persistence c…
Learning representation for graph classification turns a variable-size graph into a fixed-size vector (or matrix). Such a representation works nicely with algebraic manipulations. Here we introduce a simple method to augment an attributed graph with a virtual node that is bidirectionally connected to all existing nodes…
Regularization for matrix factorization (MF) and approximation problems has been carried out in many different ways. Due to its popularity in deep learning, dropout has been applied also for this class of problems. Despite its solid empirical performance, the theoretical properties of dropout as a regularizer remain qu…
The relationship between the size and the variance of firm growth rates is known to follow an approximate power-law behavior σ(S)∼S−β(S) where S is the firm size and β(S)≈0.2 is an exponent weakly dependent on S. Here we show how a model of proportional growth which treats firms as classes compos…
Graph Laplacian spectrum serves as a robust feature representation.
problem Difficulties in analyzing and comparing graphs due to their structure.
method Proposes using the graph Laplacian spectrum (GLS) as a feature representation.
result Graph Laplacian spectrum (GLS) preserves structural information and is consistent under deformation and invariance under isomorphism.
Hyper-SAGNN learns patterns in hypergraphs for complex interactions.
problem Learning patterns in hypergraphs with variable-sized heterogeneous hyperedges.
method Self-attention based graph neural network for homogeneous and heterogeneous hypergraphs.
result Significantly outperforms state-of-the-art methods on various tasks.
We propose in this paper a differentiable learning loss between time series, building upon the celebrated dynamic time warping (DTW) discrepancy. Unlike the Euclidean distance, DTW can compare time series of variable size and is robust to shifts or dilatations across the time dimension. To compute DTW, one typically so…
A new differentiable divergence for time series comparison.
problem Computing discrepancies between time series of varying lengths.
method Proposed a new divergence, soft-DTW divergence, addressing issues of differentiability and positivity.
result Showed that the new divergence is a valid divergence: non-negative and minimized when time series are equal.
AdaptiveNet tackles disease progression prediction in rheumatoid arthritis using deep neural networks.
problem Predicting disease progression in rheumatoid arthritis using clinical data.
method AdaptiveNet, a novel recurrent neural network architecture, that handles multiple lists of different events and missing data.
result AdaptiveNet outperforms classical baselines in disease progression prediction.
We consider large scale empirical risk minimization (ERM) problems, where both the problem dimension and variable size is large. In these cases, most second order methods are infeasible due to the high cost in both computing the Hessian over all samples and computing its inverse in high dimensions. In this paper, we pr…
New method estimates graphons from multiple networks with high accuracy and low complexity.
problem Estimating graphon function from multiple networks with different node sets and sizes.
method Histogram-based estimator that aligns nodes across all networks.
result High accuracy and low computational complexity achieved.
FREDE efficiently embeds graphs using linear space and guarantees quality.
problem Efficiently embedding graphs with quality guarantees and linear space complexity.
method FREDE combines matrix sketching with a nonlinear transform of PageRank similarities to achieve linear space and quality guarantees.
result FREDE provides column-covariance approximation guarantees that are nearly as good as SVD, even with limited node similarities.
Deep RL learns to construct objects from 2D images by avoiding brick overlaps.
problem Learning to construct objects from partial 2D images with constraints.
method Combination of reinforcement learning and action validity prediction.
result Successfully constructs unseen objects from 2D images.
Reinforcement Patching optimizes dynamic sequence patching for efficient time series forecasting.
problem Efficiently learning data-adaptive representations for long-horizon sequence data, especially continuous sequences.
method Reinforcement Patching (ReinPatch) uses reinforcement learning to optimize dynamic patching policies and sequence backbones.
result ReinPatch achieves compelling performance in time-series forecasting compared to state-of-the-art methods.
Online distributional prediction with latent cluster geometry
problem Predicting the full data-generating distribution in non-stationary streams
method Representing candidate laws as latent cluster geometry and using Gibbs quasi-posterior
result Achieving sublinear cumulative Wasserstein regret under bounded support and stable latent geometry
The market events of 2007-2009 have reinvigorated the search for realistic return models that capture greater likelihoods of extreme movements. In this paper we model the medium-term log-return dynamics in a market with both fundamental and technical traders. This is based on a Poisson trade arrival model with variable…
ETC improves Transformer models for long and structured inputs.
problem Scaling input length and encoding structured inputs in Transformers.
method Introduces global-local attention, relative position encodings, and CPC pre-training.
result Achieves state-of-the-art results on four natural language datasets.
Unified method for input, data, and model uncertainty in neural networks.
problem Uncertainty in neural network inputs and outputs.
method Propagating uncertainty through inputs using a unified formulation.
result More stable decision boundaries with input noise, and propagation of input uncertainty to model outputs.
New model evaluates how well models handle input faults.
problem Fault tolerance of models to input variations.
method Evaluates fault tolerance using information-based characteristic for arbitrary valid inputs.
result Proposes a new way to measure model robustness.
Improves sample efficiency in reinforcement learning with input representation.
problem Poor sample efficiency in reinforcement learning.
method Attention-based method to project inputs into an invariant representation space.
result Representation space is m! smaller for inputs of m objects, improving sample efficiency. Partial-input models fail to detect dataset artifacts, even when they perform poorly.
problem The effectiveness of partial-input models in detecting dataset artifacts is questionable.
method Design artificial datasets and identify trivial patterns in the SNLI dataset.
result Partial-input models can solve examples previously considered hard, indicating potential dataset artifacts.
New methods handle uncertainty in identifying input regions for a black-box function.
problem Handling uncertainty in identifying input regions for a black-box function.
method Introduce a basic framework and efficient methods for LSE under input uncertainty.
result Proposed methods can be applied to various LSE challenges under input uncertainty.
Detects unusual inputs to neural networks to prevent flawed predictions.
problem Erratic predictions from neural networks on unexpected inputs.
method Evaluates input unusualness by comparing its content to learned parameters.
result Simple, effective method for comparing input metrics across different scales.
Enhances construction input modeling with Bayesian deep neural networks.
problem Deriving reliable simulation input models from construction data.
method Bayesian deep neural networks integrated with multi-source construction data.
result Derives detailed input models for construction operations.
TVS-FNNs can approximate any continuous function on expanded input spaces.
problem Processing a broader range of inputs like sequences and matrices.
method Proving a universal approximation theorem for TVS-FNNs.
result TVS-FNNs can approximate any continuous function on expanded input spaces.
We consider reinforcement learning in input-driven environments, where an exogenous, stochastic input process affects the dynamics of the system. Input processes arise in many applications, including queuing systems, robotics control with disturbances, and object tracking. Since the state dynamics and rewards depend on…