Decodes neural activity to detect context effects in natural settings.
problem Detecting context-dependent changes in neural encoding.
method Decoding-based approach controlling for confounding factors.
result Demonstrates context-dependent changes in neural encoding.
ContextFlow++ improves generative models by conditioning on mixed-variable contexts.
problem Lack of effective methods for context conditioning in flow-based generative models.
method Proposes ContextFlow++ with additive conditioning and mixed-variable architecture.
result ContextFlow++ achieves higher performance metrics and faster training.
Proposes a method to improve skull stripping accuracy in MRI images.
problem Skull stripping accuracy in MRI images is improved.
method Context-encoding method to empower 2D networks with 3D semantic information.
result Achieves superior accuracy (dice score 99.6% on NFBS, 99.09% on LPBA40, 99.17% on OASIS) compared to state-of-the-art methods.
The paper identifies saddlepoints in unsupervised auto-encoding neural nets.
problem The risk landscape of unsupervised least squares in auto-encoding neural nets.
method Established an equivalence between unsupervised least squares and principal manifolds, discussed regularization strategies for auto-encoders.
result All non-trivial critical points in auto-encoding are saddlepoints, which are degenerate in overcomplete auto-encoding.
Paper proposes FOFE for efficient WSD.
problem Word sense disambiguation (WSD) problem.
method Fixed-size ordinally forgetting encoding (FOFE) combined with FFNN.
result FOFE-based FFNN achieves comparable performance to state-of-the-art at lower cost.
Adaptive feature extraction boosts contextual bandit performance.
problem Improving performance in online decision making with contextual bandit.
method Adaptive feature extraction based on online clustering and online selection of encoders.
result Clear advantages over standard contextual bandit in various environments.
Word2vec improved but lacks multi-meaning words; ConEc creates new embeddings.
problem Lack of meaningful embeddings for words with multiple meanings and OOV words.
method Context encoders (ConEc) extend word2vec by multiplying embeddings with context vectors.
result ConEc creates embeddings for OOV words and words with multiple meanings based on local contexts.
Deep model learns graph structure with context diffusion.
problem Processing structured graph data efficiently.
method Constructive deep architecture with probabilistic models.
result Generative approach improves graph structure classification.
New analysis shows PE in Transformers increases generalization gap and vulnerability.
problem Understanding the impact of PE on Transformer generalization and robustness.
method Generalization analysis and adversarial Rademacher bounds for a single-layer Transformer with trainable PE.
result PE systematically enlarges the generalization gap and makes models more vulnerable to attacks.
MEDL_CVAE learns complex correlations among multiple entities using rich context.
problem Learning complex correlations among multiple entities with rich context.
method Conditional Variational Auto-Encoder (CVAE) for encoding conditional multivariate distributions.
result MEDL_CVAE captures rich dependency structures and improves joint likelihood.
ceVAE improves anomaly detection in medical images by combining reconstruction and density-based scoring.
problem Lack of formal assertions and comparability in anomaly scores based on reconstruction error.
method Proposes Context-encoding Variational Autoencoder (ceVAE) that combines reconstruction and density-based scoring.
result ceVAE achieves unsupervised ROC-AUCs of 0.95 and 0.89 on BraTS-2017 and ISLES-2015 benchmarks, outperforming state-of-the-art methods.
MQTransformer improves forecast accuracy with context-aware attention.
problem Improving probabilistic demand prediction accuracy.
method Incorporates Transformer architectures for context alignment and feedback-aware attention.
result Significant improvements in forecast accuracy, reducing excess variability.
Improves dialogue response model interpretability using attention and regularization.
problem Improving interpretability of dual encoder models for dialogue response suggestions.
method Integrates attention mechanism and novel regularization loss to emphasize important words.
result Improves model accuracy and interpretability compared to existing methods.
CCM improves context for Meta-RL by contrastive learning.
problem Improving context for Meta-RL to enable task generalization.
method CCM framework using contrastive learning for context encoding and information-gain-based trajectory collection.
result CCM outperforms state-of-the-art algorithms in benchmarks and sparse-reward environments.
This paper assesses biases in contextualized word representations.
problem Analyzing biases in contextualized word representations.
method Proposes assessing bias at the contextual word level, capturing contextual effects of bias.
result Demonstrates evidence of bias in contextual word models, including racial bias and exacerbated effects for intersectional minorities.
End-to-end TTS learns context features from text input.
problem Lack of understanding of context features learned by end-to-end TTS.
method Evaluated encoder outputs against context criteria derived from parametric TTS.
result Encoder outputs reflect linguistic and phonetic context features.
Learning the Markov network structure from data is a problem that has received considerable attention in machine learning, and in many other application fields. This work focuses on a particular approach for this purpose called independence-based learning. Such approach guarantees the learning of the correct structure …
Efficiently combines autoregressive and set-based models for joint distributions.
problem Joint distributions over multiple predictions from set-based models.
method Causal autoregressive buffer that caches context and captures dependencies.
result Up to 20x faster joint sampling and density evaluation, up to 7x lower memory usage.
TIMeSynC combines financial service interactions for intent prediction.
problem Aligning and learning from multi-domain, multi-resolution sequences for accurate intent prediction.
method An encoder-decoder transformer model addressing sequence alignment, temporal dynamics, and dynamic/static sequence combination.
result Significant improvement in intent prediction over existing methods.
New techniques save bits in image compression with upsampling.
problem Lack of context dependence in current image compression methods with upsampling.
method Simple, inexpensive techniques exploiting context to predict Laplace distribution parameters.
result Average savings of 0.645 bits per difference, up to 1.489 bits.
VectorNet predicts car behavior using vectorized HD maps and agent dynamics.
problem Predicting behavior in multi-agent systems with self-driving cars.
method VectorNet uses hierarchical graph neural networks on vectorized representations of HD maps and agent trajectories.
result VectorNet achieves comparable or better performance than state-of-the-art methods while using fewer parameters and less computational power.
New RNN model fuses sensor data from multiple stations.
problem Modeling distributed sensor networks for future behavior prediction.
method Multi-Encoder-Decoder RNN architecture with attention mechanism.
result Model improves prediction accuracy on real-world sensor datasets.
The paper introduces a tensor-based approach to improve neural models' aggregation of structural context.
problem Sub-optimal use of simple aggregation functions in neural models for structured data.
method Tensor-based formulation and Tucker tensor decomposition to control parameter space size.
result Effective regulation of trade-off between expressivity, computational complexity, and generalisation.
Boosting trees predict Twitch subscriptions from user activity.
problem Predicting Twitch user subscriptions from activity data.
method Used boosting trees and target-encodings for high cardinality categoricals.
result User activity can be better predicted than content alone.
Randomized positional encodings boost transformer performance on longer sequences.
problem Transformers struggle with generalizing to sequences of arbitrary length.
method Introduced randomized positional encodings that simulate longer sequences and randomly select positions.
result Randomized positional encodings increase test accuracy by 12.0% on average for sequences of unseen length.
Paper explores combining auto-encoder representations to fool adversarial discriminators.
problem Combining auto-encoder representations to fool adversarial discriminators.
method Mixing function to produce interpolations of hidden states or masked combinations of latent representations.
result Shows that mixing function can produce interpolations consistent with a conditioned class label.
Transformers learn causal structure through gradient descent on self-attention mechanisms.
problem Understanding how transformers learn causal structure during training.
method In-context learning task and simplified two-layer transformer model.
result Gradient descent on a simplified transformer learns to encode latent causal graphs.
Paper proposes a faster RAE with sequence-aware encoding.
problem Training recurrent autoencoders is challenging and time-consuming.
method Introduces a recurrent autoencoder with sequence-aware encoding using 1D convolutional layers.
result The proposed autoencoder trains faster than standard RAE.
Contextualized ML learns context-dependent effects using deep learning.
problem Learning heterogeneous and context-dependent effects in data.
method Applying deep learning to the meta-relationship between contextual information and context-specific parametric models.
result Unified framework for cluster analysis and cohort modeling.
CRsAE auto-encoder recovers convolutional dictionary from noisy signals.
problem Recovering a convolutional dictionary from noisy signals.
method Constrained recurrent sparse auto-encoder (CRsAE) architecture.
result CRsAE successfully recovers the underlying dictionary in the presence of noise.
Transformer-XL extends language models' context length without disrupting temporal coherence.
problem Fixed-length context limitation in Transformers.
method Segment-level recurrence and novel positional encoding.
result 80% longer dependency, 450% longer than vanilla Transformers.
A new histogram layer improves texture analysis performance.
problem Extracting features for texture analysis from local spatial regions.
method Directly computes local spatial distribution of features during backpropagation.
result Improves performance on three material/texture datasets.
Recent progress on automatic generation of image captions has shown that it is possible to describe the most salient information conveyed by images with accurate and meaningful sentences. In this paper, we propose an image caption system that exploits the parallel structures between images and sentences. In our model, …
A new model for context-aware recommendations using LSTM and latent context.
problem Challenges in incorporating context into recommendation models, especially sparsity and dimensionality issues.
method Sequential latent context modeling using LSTM, reducing context dimensions to a compressed latent space.
result The proposed SLCM outperforms state-of-the-art CARS models in empirical analysis.
This paper tackles text infilling, a task of filling missing text portions, and presents a self-attention model that outperforms other methods.
problem The task of filling missing text portions, especially when the number and length of missing portions are unknown.
method A self-attention model with segment-aware position encoding and bidirectional context modeling, trained on extensive supervised data.
result The self-attention model significantly outperforms other approaches, setting a strong baseline for future research.
Develops a new dataset and model for summarizing scientific papers.
problem Lack of large datasets for summarizing scientific papers.
method Exploits author-provided summaries, uses neural sentence encoding and summarisation features.
result Models that encode sentences and their context perform best, significantly outperforming baselines.
A method for learning a context latent vector to improve generalization in model-based RL.
problem Learning a global dynamics model that can generalize across different dynamics.
method Decomposes learning a global dynamics model into two stages: learning a context latent vector and predicting next states.
result Achieves superior generalization across various simulated robotics and control tasks.
Paper proposes a deep learning architecture for generating long stories from images.
problem Maintaining context in long event sequences for visual storytelling.
method Hierarchical deep learning architecture with encoder-decoder networks and natural language descriptions.
result Our method outperforms state-of-the-art techniques on automatic evaluation metrics.
Enhances financial data signal-to-noise ratio using auto-encoders and mutual regularization.
problem Improving signal-to-noise ratio in financial data.
method Combining target and context variables, using auto-encoders with mutual regularization to learn common ground.
result Discover new regularities in financial time-series data.
A new method uses persistent homology to assess auto-encoders' latent manifold quality.
problem Chaos in auto-encoders' latent manifold and failure of current distance measures.
method Persistent Homology for Wasserstein Auto-Encoders (PHom-WAE).
result PHom-WAE improves auto-encoders' performance in credit card transaction data.
We present an approach to learning multi-sense word embeddings relying both on monolingual and bilingual information. Our model consists of an encoder, which uses monolingual and bilingual context (i.e. a parallel sentence) to choose a sense for a given word, and a decoder which predicts context words based on the chos…
NOTMAD estimates context-specific Bayesian networks without breaking datasets.
problem Non-convexity of acyclic graphs limits sharing information between context-specific estimators.
method NOTMAD models context-specific Bayesian networks as mixtures of archetypal DAGs, estimating structures and parameters jointly.
result NOTMAD shares information between context-specific acyclic graphs, enabling single-sample resolution.
This paper introduces a new task to better understand Transformers in quantitative contexts.
problem Understanding Transformers in high-stakes quantitative and scientific applications.
method Introduces a novel contextual counting task and analyzes it with causal and non-causal Transformer architectures.
result Causal attention is better suited for the contextual counting task, and no positional embeddings lead to the best accuracy.
The dissertation establishes a contexture theory to mathematically characterize representation learning.
problem The lack of a scientific understanding of representation learning in foundation models.
method Introduces the contexture theory as a unified framework for analyzing representation learning methods.
result Representation learning is optimal when the association between input and context is neither too strong nor too weak.
This paper explains how transformers learn from unstructured data in ICL.
problem Understanding how transformers learn from unstructured data in in-context learning.
method A simple transformer model with one or two attention layers and positional encoding is used to study the role of each component in ICL.
result A transformer with two attention layers and a look-ahead attention mask can learn from unstructured data.
CADE learns dual node representations for better generalization.
problem Transductive graph embeddings cannot generalize to unseen nodes or across different graphs.
method CADE combines real-time neighborhoods with neighbor-attentioned representation, preserving known node memory.
result CADE outperforms state-of-the-art methods in generalization and context-awareness.
This paper improves search efficiency by augmenting autoencoder encodings.
problem Improving search speed and relevance in information retrieval.
method Gradient Augmented Information Retrieval with Autoencoders and Semantic Hashing.
result Gradient Augmented Search (GSA) enhances TF-IDF-based systems.
Enhances neural processes for better context handling.
problem Real-world context sets are complex, requiring richer prior distributions.
method Introduces a graphical model for a richer prior on latent variables, enabling end-to-end optimization.
result Improves function modeling and test-time robustness with mixture and Student-t assumptions.