Visual analytics system for comparing medical records using sequence embeddings.
problem Challenges in analyzing medical records due to high dimensionality, irregularity, and sparsity.
method Event and sequence embeddings using autoencoder and self-attention mechanism, with sequence alignment for comparison.
result Demonstrated effectiveness with real-world neonatal ICU dataset.
We introduce the notion of the visual core of a hyperbolic 3-manifold N and explore its basic properties. The visual core can be thought of as a harmonic analysis analogue of the convex core. We investigate circumstances in which the visual core of a cover N' of N embeds under the covering map from N' to N. We apply th…
Recent breakthroughs in computer vision and natural language processing have spurred interest in challenging multi-modal tasks such as visual question-answering and visual dialogue. For such tasks, one successful approach is to condition image-based convolutional network computation on language via Feature-wise Linear …
MIK improves t-SNE's local structure preservation in biological sequence data.
problem Efficiently preserving local structure in high-dimensional biological sequence data.
method Modified Isolation Kernel (MIK) using adaptive density estimation.
result MIK preserves local and global structure better than Gaussian and isolation kernels.
Tangled String visualizes stock market changes at various timescales.
problem Explaining multi-scale contextual shifts in stock market prices.
method Sequence visualization tool based on metaphor of tangled strings.
result Change points in stock prices coincide with real changes.
New method embeds RNN Seq2Seq models to visualize spatiotemporal data.
problem Visualizing and interpreting spatiotemporal data in sequence prediction tasks.
method Embedding approach to visualize and interpret RNN Seq2Seq model representations.
result Embedding space projections of RNN Seq2Seq models capture spatiotemporal dynamics.
Paper proposes a deep learning architecture for generating long stories from images.
problem Maintaining context in long event sequences for visual storytelling.
method Hierarchical deep learning architecture with encoder-decoder networks and natural language descriptions.
result Our method outperforms state-of-the-art techniques on automatic evaluation metrics.
Tree-SNE combines t-SNE and hierarchical clustering for data visualization.
problem Data visualization and clustering in complex datasets.
method Stacked one-dimensional t-SNE embeddings and alpha-clustering.
result Effective hierarchical clustering and visualization of various datasets.
Framework uses machine learning to distinguish major COVID-19 variants.
problem Discriminate and visualize associations between major COVID-19 variants based on genome sequences.
method Unsupervised machine learning methods, including k-mer analysis, PCA, t-SNE, UMAP, and agglomerative hierarchical clustering.
result Framework effectively distinguishes between major variants and identifies emerging variants.
Paper improves video feature learning for better downstream tasks.
problem Improving video feature learning for better performance on downstream tasks.
method Self-supervised learning approach using contrastive bidirectional transformer, extending BERT for real-valued feature vectors.
result Significantly improved performance on video classification, captioning, and segmentation tasks.
Intelligent agents can learn to represent the action spaces of other agents simply by observing them act. Such representations help agents quickly learn to predict the effects of their own actions on the environment and to plan complex action sequences. In this work, we address the problem of learning an agent's action…
Planar neural networks learn image transformations from sequences.
problem Learning image transformations for mental simulation.
method Using planar neural networks, the study investigates various factors affecting the learning of image transformations.
result The approach can effectively learn and transfer image transformations, including translation, rotation, and scaling.
The paper extends the functional geometry of the visual cortex to more complex architectures using contactization and symplectization.
problem Understanding the functional architecture of the visual cortex.
method Contactization and symplectization processes to extend the dimension of the space.
result Extension of the functional geometry of the visual cortex to more complex architectures.
Visualizes movement control optimization landscapes to understand why it's hard and how to make it easier.
problem Understanding and optimizing movement control problems in animation research.
method Novel visualizations of high-dimensional control optimization landscapes.
result Trajectory optimization becomes increasingly ill-conditioned with longer trajectories, while parameterizing control as partial target states can act as an efficient preconditioner.
A new memory model enhances deep learning's visual understanding.
problem Lack of short-term memory in deep learning models.
method Introduces a biologically inspired visual working memory architecture.
result Model achieves competitive classification performance and reconstructs images.
DSM improves visual quality of fetoscopic videos by mosaicking.
problem Challenges in identifying all vascular connections during fetoscopy due to low visual quality and drift.
method Proposes a Deep Sequential Mosaicking (DSM) framework using deep learning techniques to handle visual variations and reduce drift.
result DSM framework reduces accumulated drift and improves visual quality of fetoscopic videos.
Automatic speech recognition can potentially benefit from the lip motion patterns, complementing acoustic speech to improve the overall recognition performance, particularly in noise. In this paper we propose an audio-visual fusion strategy that goes beyond simple feature concatenation and learns to automatically align…
In recent years, deep generative models have been shown to 'imagine' convincing high-dimensional observations such as images, audio, and even video, learning directly from raw data. In this work, we ask how to imagine goal-directed visual plans -- a plausible sequence of observations that transition a dynamical system …
KeyIn discovers keyframes to simplify video analysis.
problem Extracting essential dynamics from videos interleaved with predictable details.
method A hierarchical Keyframe-Inpainter (KeyIn) model learns keyframes and fills in the rest.
result KeyIn finds informative keyframes across various video dynamics.
In the last decade, supervised deep learning approaches have been extensively employed in visual odometry (VO) applications, which is not feasible in environments where labelled data is not abundant. On the other hand, unsupervised deep learning approaches for localization and mapping in unknown environments from unlab…
Learning visual feature representations for video analysis is a daunting task that requires a large amount of training samples and a proper generalization framework. Many of the current state of the art methods for video captioning and movie description rely on simple encoding mechanisms through recurrent neural networ…
Multimodal machine learning is a core research area spanning the language, visual and acoustic modalities. The central challenge in multimodal learning involves learning representations that can process and relate information from multiple modalities. In this paper, we propose two methods for unsupervised learning of j…
New method learns both module structure and sequencing in neural networks.
problem Learning only the parameters and order of execution of neural modules.
method Expands the approach to learn the internal structure of modules, including the ordering and combination of arithmetic operators.
result Performance comparable to hand-designed modules achieved without extra supervisory signals.
New insights into how encoder-decoder networks generate attention matrices.
problem Understanding how encoder-decoder networks use attention matrices.
method Decomposing hidden states into temporal and input-driven components.
result Attention matrices are formed based on task requirements, not architecture type.
This work tackles long-term visual planning by goal-conditioned hierarchical predictors.
problem Current learning approaches fail on long-horizon tasks due to lack of goal information and coarse-to-fine planning.
method Formulate goal-conditioned predictors (GCPs) and hierarchical models to predict trajectories between observations.
result GCPs enable effective long-term planning with much longer horizons than before.
New methods improve analysis of single cell RNA sequencing data.
problem High dimensionality and complexity of scRNA-seq data.
method Topological Nonnegative Matrix Factorization (TNMF) and Robust Topological NMF (rTNMF).
result TNMF and rTNMF significantly outperform other NMF-based methods.
A GAN variant synthesizes missing MRI sequences from available ones.
problem Missing MRI sequences due to various constraints.
method Multi-modal Generative Adversarial Network (GAN) that combines multiple available sequences to synthesize missing ones.
result The proposed GAN method outperforms competing approaches in synthesizing missing MRI sequences.
We use Bayesian optimal experimental design to generate near-optimal attention sequences for faster hard attention training.
problem Training hard attention mechanisms is computationally expensive and high-variance.
method We frame hard attention as a BOED problem and use approximation methods from BOED to generate near-optimal attention sequences.
result Near-optimal attention sequences can speed up hard attention training and be reused by other networks.
SeqAttnGAN generates interactive images based on multi-turn text descriptions.
problem Interactive image editing with multi-turn textual commands.
method SeqAttnGAN uses a neural state tracker and GAN framework for sequential image generation and refinement.
result SeqAttnGAN outperforms state-of-the-art models on interactive image editing tasks.
We suggest a novel method of clustering and exploratory analysis of temporal event sequences data (also known as categorical time series) based on three-dimensional data grid models. A data set of temporal event sequences can be represented as a data set of three-dimensional points, each point is defined by three varia…
RNNs and their variants have been widely adopted for image captioning. In RNNs, the production of a caption is driven by a sequence of latent states. Existing captioning models usually represent latent states as vectors, taking this practice for granted. We rethink this choice and study an alternative formulation, name…
Each human genome is a 3 billion base pair set of encoding instructions. Decoding the genome using deep learning fundamentally differs from most tasks, as we do not know the full structure of the data and therefore cannot design architectures to suit it. As such, architectures that fit the structure of genomics should …
Deep network predicts action sequences for complex tasks from a scene image.
problem Scalable task and motion planning from initial scene images.
method Deep convolutional recurrent neural network that predicts action sequences.
result Predicts promising action sequences, reducing motion planning problems.
Neural nets learn and forget tasks sequentially, showing promising scalability.
problem Learning and forgetting of multiple visual tasks in a sequential setting.
method Simulated sequential learning of ten related visual tasks.
result Neural nets show forward facilitation and backward interference, which are key phenomena.
Paper proposes DTW-SOM for visual exploration of time-series motifs.
problem Exploring motifs extracted by time-series motif discovery algorithms.
method Adapted Self-Organizing Map (DTW-SOM) using Dynamic Time Warping distance.
result DTW-SOM effectively visualizes motifs from time-series data.
We argue that the existing regret matchings for Nash equilibrium approximation conduct "jumpy" strategy updating when the probabilities of future plays are set to be proportional to positive regret measures. We propose a geometrical regret matching which features "smooth" strategy updating. Our approach is simple, intu…
Applying convolutional neural networks to large images is computationally expensive because the amount of computation scales linearly with the number of image pixels. We present a novel recurrent neural network model that is capable of extracting information from an image or video by adaptively selecting a sequence of …
System solves author name ambiguity in e-commerce catalogs.
problem Finding correct author names in e-commerce catalogs with abbreviations and spelling variants.
method Composite system using open data sources and machine learning techniques for natural language processing.
result Top proposal of the system is the normalized author name with 72% accuracy.
Alternative sampling method for autoregressive models using Langevin dynamics.
problem Efficiently sampling from autoregressive models.
method Initialize sequences with white noise and follow Langevin dynamics on global log-likelihood.
result Parallelizes and generalizes sampling process for autoregressive models.
Algorithm improves imitation learning from visual data in partially observable environments.
problem Imitation learning from visual observations with missing expert actions and partial observability.
method Theoretical analysis and Latent Adversarial Imitation from Observations algorithm combining adversarial and latent representations.
result Latent Adversarial Imitation from Observations achieves state-of-the-art performance in high-dimensional robotic tasks.
Improved animated faces using audiovisual and modality dropout.
problem Creating realistic animated faces using speech and visual cues.
method Training a deep learning model with modality dropout to balance audio and visual inputs.
result Modality dropout improves viewer preference for audiovisual-driven animation.
A new VAD method uses respiration patterns from video to detect speech.
problem Improving VAD performance in noisy audio recordings.
method Extract respiration patterns from video, use neural models to detect speech.
result Efficacy demonstrated through experiments on real acoustic environments.
Reinforcement Patching optimizes dynamic sequence patching for efficient time series forecasting.
problem Efficiently learning data-adaptive representations for long-horizon sequence data, especially continuous sequences.
method Reinforcement Patching (ReinPatch) uses reinforcement learning to optimize dynamic patching policies and sequence backbones.
result ReinPatch achieves compelling performance in time-series forecasting compared to state-of-the-art methods.
Feature engineering remains a major bottleneck when creating predictive systems from electronic medical records. At present, an important missing element is detecting predictive regular clinical motifs from irregular episodic records. We present Deepr (short for Deep record), a new end-to-end deep learning system that …
Compact Recurrent Transformer (CRT) improves Transformer efficiency for long sequences.
problem Efficiently scaling Transformer architecture to long sequences with limited compute resources.
method Combines shallow Transformer models with recurrent neural networks and persistent memory.
result CRT achieves comparable or superior performance to full-length Transformers with shorter segments and reduced FLOPs.
Generative Map learns interpretable neural network maps for camera localization.
problem Creating interpretable maps for neural network-based camera localization.
method Combining generative models with Kalman filters and incorporating additional sensor information.
result Generative Map predicts images closely resembling the true scene and achieves comparable localization performance.
In this work we propose a method to compute continuous embeddings for kmers from raw RNA-seq data, without the need for alignment to a reference genome. The approach uses an RNN to transform kmers of the RNA-seq reads into a 2 dimensional representation that is used to predict abundance of each kmer. We report that our…
This paper applies deep learning to ordinal regression, modeling it as a binary search.
problem Ordinal regression with deep learning models.
method Formulated ordinal regression as a binary search problem, using recurrent neural networks.
result Deep learning model shows comparable or better predictive power compared to traditional methods.