New framework learns robot tasks quickly from simplified simulations.
problem Long training times and variable-length inputs in RL.
method Combines deep sets encoding with modular RL.
result Effective policies learned in minutes from simplified simulations.
Proposes VILMAP for finding motifs and segmenting words in time series.
problem Finding motifs and segmenting words in time series data.
method Self-Organizing Maps with variable input length.
result VILMAP identifies motifs of varying lengths and avoids catastrophic forgetting.
Variable selection for Gaussian process models is often done using automatic relevance determination, which uses the inverse length-scale parameter of each input variable as a proxy for variable relevance. This implicitly determined relevance has several drawbacks that prevent the selection of optimal input variables i…
A deep learning approach for clustering time series of varying lengths.
problem Clustering time series with variable lengths and temporal relations.
method Recurrent Deep Divergence-based Clustering framework.
result Outperforms previous methods on benchmark datasets.
We introduce a new neural architecture to learn the conditional probability of an output sequence with elements that are discrete tokens corresponding to positions in an input sequence. Such problems cannot be trivially addressed by existent approaches such as sequence-to-sequence and Neural Turing Machines, because th…
Paper tackles model failures producing outputs of undesirable length.
problem Model failures producing outputs of undesirable length.
method Develops a differentiable proxy objective and a verification approach.
result Can produce outputs 50 times longer than input and prove output length below a certain size.
Tree-based LSTM improves sequential regression with missing data.
problem Regression for variable-length sequential data with missing samples.
method Tree architecture of LSTM networks, selecting LSTM networks based on presence-pattern of previous inputs.
result Significant performance improvements on financial and real-life datasets.
LMGPs extend GPs to handle mixed data, offering better accuracy and interpretability.
problem Handling mixed data types (quantitative and qualitative) in metamodeling.
method Introduce LMGPs that learn a latent manifold for qualitative inputs, using a low-rank linear map.
result LMGPs outperform existing methods in accuracy and versatility.
Neural machine translation is a relatively new approach to statistical machine translation based purely on neural networks. The neural machine translation models often consist of an encoder and a decoder. The encoder extracts a fixed-length representation from a variable-length input sentence, and the decoder generates…
Self-attention prefers sparse functions of input sequences, reducing sample complexity.
problem Understanding the inductive biases of self-attention in modeling long-range dependencies.
method Theoretical analysis and synthetic experiments to probe sample complexity of learning sparse functions with Transformers.
result Bounded-norm Transformer networks can represent sparse functions of the input sequence with logarithmic sample complexity.
We introduce a new regression framework, Gaussian process regression networks (GPRN), which combines the structural properties of Bayesian neural networks with the non-parametric flexibility of Gaussian processes. This model accommodates input dependent signal and noise correlations between multiple response variables,…
We present the Latent Sequence Decompositions (LSD) framework. LSD decomposes sequences with variable lengthed output units as a function of both the input sequence and the output sequence. We present a training algorithm which samples valid extensions and an approximate decoding algorithm. We experiment with the Wall …
PSI models and infers feature attributions efficiently and accurately.
problem Modeling and inferring feature attributions in flexible predictive models.
method Probabilistic Shapley inference (PSI) framework using latent random variables and a masking-based neural network architecture.
result PSI learns feature attribution distributions centered at Shapley values, revealing meaningful uncertainty.
BOSS optimizes string inputs using string kernels and genetic algorithms.
problem Optimizing string inputs with constraints.
method Bayesian optimization over string kernels and genetic algorithms.
result Significantly improved optimization across various string constraints.
Ensemble method detects time series anomalies without preselecting parameter values.
problem Anomaly detection requires known anomaly length, limiting practicality.
method Ensemble grammar induction for variable-length anomalies.
result Ensemble approach outperforms existing methods with different parameter selections.
Study on elastic curves with variable stiffness, derived from bending energy.
problem Modeling elastic wires with varying thickness.
method Derive Euler-Lagrange equations for curves with variable bending stiffness.
result Characterizations of elastic curves with variable stiffness.
The study examines how the choice of activation function affects the hidden node distribution in deep networks.
problem The effect of activation function on the distribution of hidden nodes in deep networks.
method Analysis of joint probability distribution of hidden variables in deep networks with random weights and biases.
result The convergence of hidden node lengths to a simple function of network parameters and activation function under minimal assumptions.
Generates long programs from inputs, optimizing multiple tasks.
problem Creating long programs from input-output pairs.
method Trains a neural network to map state and outputs to next program statement, optimizing multiple tasks concurrently.
result Creates programs twice as long as existing solutions, improving success rate and runtime.
We propose learning flexible but interpretable functions that aggregate a variable-length set of permutation-invariant feature vectors to predict a label. We use a deep lattice network model so we can architect the model structure to enhance interpretability, and add monotonicity constraints between inputs-and-outputs.…
New method for Transformer models to encode position information without sequential bias.
problem Lack of flexible and learnable position encoding for Transformer models.
method Continuous dynamical model to learn position encoding.
result Consistent improvements over baselines in various NLP tasks.
This study provides benchmarks for different implementations of LSTM units between the deep learning frameworks PyTorch, TensorFlow, Lasagne and Keras. The comparison includes cuDNN LSTMs, fused LSTM variants and less optimized, but more flexible LSTM implementations. The benchmarks reflect two typical scenarios for au…
RATQ is a new quantizer for optimizing noisy gradients in machine learning.
problem Optimizing noisy gradients in stochastic optimization.
method RATQ uses Hadamard transform and adaptive uniform quantization, and achieves near-optimal performance.
result RATQ nearly achieves information theoretic lower bounds for optimization accuracy.
Unsupervised method learns universal embeddings for variable-length multivariate time series.
problem Challenges in learning representations for time series data due to varying lengths and sparse labeling.
method Combines causal dilated convolutions with triplet loss for time-based negative sampling.
result Demonstrates quality, transferability, and practicability of learned representations.
SummerTime summarizes variable-length time series for machine learning applications.
problem Classical machine learning methods struggle with variable-length time series data.
method Summarizes time series into a fixed-length feature vector using Gaussian Mixture Models (GMM).
result Improves classification and regression performance in physical activity analysis.
Paper tackles variable-length, incomplete wearable sensor data to improve personalized insights.
problem Variable-length and incomplete time series data from wearable sensors.
method HeartSpace integrates a time series encoding module and pattern aggregation network, along with a Siamese-triplet network for representation learning.
result Empirical evaluation shows significant performance gains in personality prediction, demographics inference, and user identification.
We give an algorithm to compute stable commutator length in free products of cyclic groups which is polynomial time in the length of the input, the number of factors, and the orders of the finite factors. We also describe some experimental and theoretical applications of this algorithm.
SentenceMIM learns rich latent representations for variable-length language data.
problem Challenges in learning VAEs for variable-length language data, especially posterior collapse.
method Probabilistic auto-encoder trained with Mutual Information Machine (MIM) learning.
result SentenceMIM learns informative latent representations with high mutual information.
This paper improves RNN generalization without normalization conditions.
problem Generalization of over-parameterized RNNs without normalization constraints.
method Detailed analysis of neural tangent kernel matrix for improved generalization bounds.
result RNNs can learn functions without normalization conditions and with almost-polynomial scaling in input length.
I-BERT extends Transformer's self-attention to arbitrary input lengths.
problem Transformer models struggle with inductive generalization to unseen input lengths.
method Replaces positional encodings with a recurrent layer.
result I-BERT achieves state-of-the-art results on algorithmic tasks.
New bounds show transformers need longer training for length generalization.
problem Understanding when transformers can generalize to longer inputs.
method Analyzing different settings of transformers, providing quantitative bounds.
result Transformers need training data longer than previously thought for length generalization.
A neural network finds causal relationships among latent variables.
problem Learning causal structure among latent variables in high-dimensional data.
method Redundant Input Neural Network (RINN) with modified architecture and regularized objective function.
result The RINN method successfully recovers latent causal structure between input and output variables.
XL-Editor improves sentence post-editing using XLNet's variable-length insertion probability.
problem Post-editing sentences to refine generated text.
method XL-Editor trains XLNet to estimate variable-length insertion probabilities and apply post-editing operations.
result XL-Editor outperforms XLNet on text insertion and deletion tasks, and achieves significant style transfer improvements.
Robust algorithm detects season length without parameters.
problem Automatic detection of season length in time series.
method Interpolating, filtering, detrending followed by autocorrelation analysis.
result Outperforms existing algorithms in practical applications.
Embeddings in machine learning are low-dimensional representations of complex input patterns, with the property that simple geometric operations like Euclidean distances and dot products can be used for classification and comparison tasks. The proposed meta-embeddings are special embeddings that live in more general in…
Formula for integrating random variables on hyperbolic surfaces.
problem Integrating random variables on the moduli space of hyperbolic surfaces.
method Integration formula for lengths of closed geodesics.
result Integral of geometric random variables can be expressed as an integral over R.
Based on empirical financial time-series, we show that the "silence-breaking" probability follows a super-universal power law: the probability of observing a large movement is inversely proportional to the length of the on-going low-variability period. Such a scaling law has been previously predicted theoretically [R. …
A new kernel Stein test assesses fit for variable-length sequential data.
problem Evaluating goodness of fit for varying-dimensional data like text documents of different lengths.
method Extends kernel Stein discrepancy (KSD) to variable-dimension settings by identifying appropriate Stein operators and proposing a novel KSD goodness-of-fit test.
result The proposed test performs well on discrete sequential data benchmarks.
ReLU networks don't exponentially distort curve lengths as previously thought.
problem Understanding how neural networks distort curve lengths with depth.
method Analyzing expected length distortion of ReLU networks with random initialization.
result Expected length distortion does not grow with depth, and shrinks slightly.
Optimal kernel learning improves GP regression for high-dimensional inputs.
problem High computational costs and low prediction accuracy in GP models with many inputs.
method Approximates GP covariance with a convex combination of kernel functions, identifying active variables.
result Improves prediction accuracy and correctly identifies active input variables.
New method finds sparse groups of input variables for neural networks.
problem Finding optimal groups of input variables for neural networks.
method Developed a new loss function and optimization algorithm for multi-layer non-linear neural networks to achieve group sparsity.
result Achieved group sparsity in three real-world datasets, improving model performance and excluding a significant number of variables.
RNN models perform similarly with or without extraneous variables.
problem Impact of extraneous variables on RNN performance in clinical tasks.
method Investigated the effect of extraneous input variables on RNN predictive performance using EMR and randomly drawn variables.
result Degradations in RNN's predictive performance with extraneous variables were negligible.
Recurrent neural networks and sequence to sequence models require a predetermined length for prediction output length. Our model addresses this by allowing the network to predict a variable length output in inference. A new loss function with a tailored gradient computation is developed that trades off prediction accur…
New approach reduces malware detection memory requirements and speeds up training.
problem Efficiently classifying long sequences of malware detection data.
method Developed a new temporal max pooling method and global channel gating design.
result 116x more memory efficient and 25.8x faster training on original dataset.
Paper shows SGD can learn RNNs efficiently for certain functions.
problem Understanding what concept class RNNs can learn and how efficiently.
method Vanilla stochastic gradient descent (SGD) approach.
result RNNs can learn some functions efficiently with polynomial complexity in input length.
IANN visualizes all input variables effects simultaneously.
problem Inability to visualize all input variables effects simultaneously in black-box functions.
method Interpretable Architecture Neural Network (IANN) approach.
result Visualization of all input variables effects directly and simultaneously.
A heuristic method for determining input ranges for complex processes.
problem Determining input variable ranges for non-numeric, high-dimensional processes.
method Create synthetic training data and use a decision tree classifier.
result Validated on a real use case in a lamination factory.
Enhances sensitivity analysis for correlated inputs.
problem Estimating sensitivity indices in models with correlated inputs.
method Proposes an extension of Sobol' estimator using a linear correlation model.
result Improves accuracy in variance-based sensitivity analysis.
TOQ-Nets learn to recognize complex temporal events with varying objects and sequences.
problem Recognizing complex relational-temporal events with varying numbers of objects and sequence lengths.
method Neuro-symbolic networks with reasoning layers for finite-domain quantification over objects and time.
result TOQ-Nets can generalize to scenarios with more objects than training data and temporal warpings.