Model tags biomedical texts with tree-structured tags.
problem Tagging unstructured texts with an arbitrary number of terms from an ontology.
method Neural tree decoding approach starting from the root node of an ontology.
result Outperforms state-of-the-art approaches on MeSH term assignment.
Graph-to-Tree Neural Networks improve structured input-output translation in tasks like semantic parsing and math word problems.
problem Improving performance on tasks like semantic parsing and math word problem solving.
method Graph-to-Tree Neural Networks, consisting of a graph encoder and a hierarchical tree decoder.
result Graph2Tree model outperforms or matches state-of-the-art models on neural semantic parsing and math word problem tasks.
The task of translating between programming languages differs from the challenge of translating natural languages in that programming languages are designed with a far more rigid set of structural and grammatical rules. Previous work has used a tree-to-tree encoder/decoder model to take advantage of the inherent tree s…
The paper develops a theory for speculative decoding acceptance criteria.
problem Speculative decoding's acceptance criteria and their rejection regions.
method Characterization of rejection regions as lower level sets of the target distribution, derivation of exact and margin-based certificates.
result Relaxed and tree-based acceptance criteria substantially enlarge the region of certified acceptance.
We discuss an autoencoder model in which the encoding and decoding functions are implemented by decision trees. We use the soft decision tree where internal nodes realize soft multivariate splits given by a gating function and the overall output is the average of all leaves weighted by the gating values on their path. …
Resource-efficient oblique trees reduce neural signal classification costs.
problem Implementing efficient neural signal classifiers on resource-constrained devices.
method Integrating model compression, probabilistic routing, and cost-aware learning.
result Significant reduction in model size and feature extraction cost compared to state-of-the-art models.
Novel low-rank neural decoder improves μ-ECoG neural decoding.
problem Challenging neural decoding from high-dimensional μ-ECoG data. method Low-rank structure in neural network decoder.
result Low-rank decoder outperforms standard PCA.
Neural decoder improves topological code performance.
problem Improving error correction for topological codes.
method Two-step neural network using pseudo-inverse of parity check matrix.
result Outperforms state-of-the-art non-neural decoders for 2D hexagonal color codes.
A new tree-Wasserstein distance for high-dimensional data with latent feature hierarchy.
problem Finding meaningful distances between high-dimensional data samples with latent feature hierarchy.
method Proposes a new tree-Wasserstein distance (TWD) for high-dimensional data with a latent feature hierarchy, using diffusion geometry and tree decoding.
result The proposed TWD effectively recovers the latent feature hierarchy and is efficient and scalable.
A new CNN-based code generator outperforms RNNs by 5%.
problem Capturing long sequences in code generation using RNNs.
method Grammar-based structural CNN with tree-based and pre-order convolution modules.
result Significantly outperforms previous state-of-the-art methods by 5 percentage points.
Finding optimal correction of errors in generic stabilizer codes is a computationally hard problem, even for simple noise models. While this task can be simplified for codes with some structure, such as topological stabilizer codes, developing good and efficient decoders still remains a challenge. In our work, we syste…
Deep neural nets predict aircraft flight paths from weather data.
problem Accurate prediction of aircraft trajectories for aviation efficiency.
method Deep generative convolutional recurrent neural network with tree-based matching.
result Model accurately predicts aircraft flight paths from weather data.
Despite rapid advances in machine learning tools, the majority of neural decoding approaches still use traditional methods. Modern machine learning tools, which are versatile and easy to use, have the potential to significantly improve decoding performance. This tutorial describes how to effectively apply these algorit…
Finding efficient decoders for quantum error correcting codes adapted to realistic experimental noise in fault-tolerant devices represents a significant challenge. In this paper we introduce several decoding algorithms complemented by deep neural decoders and apply them to analyze several fault-tolerant error correctio…
Deep neural networks decode natural visual scenes from neural spikes.
problem Decoding visual scenes from neural spikes for brain-machine interfaces.
method Developed a novel spike-image decoder (SID) using deep neural networks.
result SID reconstructs natural visual scenes from neural spikes with high accuracy.
Decodes neural activity to detect context effects in natural settings.
problem Detecting context-dependent changes in neural encoding.
method Decoding-based approach controlling for confounding factors.
result Demonstrates context-dependent changes in neural encoding.
A new method for hierarchical clustering using continuous embeddings and optimization.
problem Hierarchical clustering with provable quality guarantees.
method Continuous relaxation of discrete optimization problem using hyperbolic embeddings and decoding.
result Continuous relaxation yields a discrete tree with (1 + epsilon)-factor approximation for optimal tree.
New measures link neural representation geometry to decoding ability.
problem Understanding how neural representations relate to decoding ability.
method Showed that popular similarity measures can be interpreted from a decoding perspective.
result Proved that measures like CKA and CCA quantify alignment between optimal linear readouts.
Proposes a secure communication method independent of eavesdropper's decoder.
problem Lack of practical security constraints in existing methods.
method Dual MINE-based neural secure communications model.
result Security performance is not affected by eavesdropper's decoding means.
A major hurdle to clinical translation of brain-machine interfaces (BMIs) is that current decoders, which are trained from a small quantity of recent data, become ineffective when neural recording conditions subsequently change. We tested whether a decoder could be made more robust to future neural variability by train…
Deep generative models have been wildly successful at learning coherent latent representations for continuous data such as video and audio. However, generative modeling of discrete data such as arithmetic expressions and molecular structures still poses significant challenges. Crucially, state-of-the-art methods often …
New method improves neural decoding accuracy and reveals latent memory organization.
problem Improving neural decoding of temporal memory organization.
method Bayesian neural decoding using a diversity-encouraging latent representation learning method.
result Substantially higher accuracy in neural decoding and clear latent representation.
Improved neural keyphrase generation by beam search with reward functions.
problem Sequence length bias and beam diversity issues in neural keyphrase generation.
method Beam search decoding strategy with word-level and ngram-level reward functions.
result Significant improvement in generating diverse and accurate keyphrases.
Neural machine translation is a relatively new approach to statistical machine translation based purely on neural networks. The neural machine translation models often consist of an encoder and a decoder. The encoder extracts a fixed-length representation from a variable-length input sentence, and the decoder generates…
Neural decoders improve performance on large algebraic block codes.
problem Improving decoding performance for large algebraic block codes.
method Graph neural networks for message passing with hypernetworks and arctanh activation.
result Neural decoding outperforms traditional methods for various code families.
DyEnsemble improves BCI accuracy by adapting to nonstationary neural signals.
problem Nonstationary neural signals in BCI cause decoding errors.
method Dynamic ensemble modeling that learns and combines diverse models online.
result DyEnsemble outperforms Kalman filters, especially with noisy signals.
A decoder helps interpret neural network layers.
problem Understanding the output of neural network layers.
method Developed a Classifier-Decoder (ClaDec) architecture.
result Reconstructed images from ClaDec are more relevant for classification.
In this study we present a kernel based convolution model to characterize neural responses to natural sounds by decoding their time-varying acoustic features. The model allows to decode natural sounds from high-dimensional neural recordings, such as magnetoencephalography (MEG), that track timing and location of human …
Deep decoder generates images with few parameters, comparable to wavelets.
problem Representing and generating natural images efficiently.
method Untrained deep neural network with simple architecture (no convolutions).
result Deep decoder compresses images into concise weights, comparable to wavelet thresholding.
Memory-augmented neural networks improve machine translation performance.
problem Improving machine translation accuracy and flexibility.
method Evaluation of Neural Turing Machines and Differentiable Neural Computers for machine translation tasks.
result Memory-augmented neural networks perform similarly to attentional encoders on Vietnamese to English tasks but have lower BLEU scores on Romanian to English tasks.
Hybrid model combines RNNs, encoders-decoders, and Transformers for sequence tasks.
problem Sequence labelling tasks
method Combination of bidirectional RNNs, encoder-decoder, and Transformer models
result Results are close to state-of-the-art and better for some tasks
Develops a new neural spike train decoding framework using topological data.
problem Decoding neural spike trains from head direction and grid cells.
method Combines simplicial complex discovery with deep learning to capture higher-order connectivity.
result Demonstrates effectiveness on head direction and trajectory prediction datasets.
The paper uses neural networks to forecast time series data.
problem Forecasting high-dimensional stationary processes.
method Encoder-decoder neural network structure to model past observations.
result Upper bounds for forecast error under specific assumptions.
A model learns stock trading rules from raw prices using encoder-decoder neural network.
problem Extracting features from long price sequences for profitable trading rules.
method Neural encoder-decoder framework combined with DRL.
result The model outperforms state-of-the-art models in dynamic environments.
Low-density parity-check codes, a class of capacity-approaching linear codes, are particularly recognized for their efficient decoding scheme. The decoding scheme, known as the sum-product, is an iterative algorithm consisting of passing messages between variable and check nodes of the factor graph. The sum-product alg…
In this paper, we propose a novel neural network model called RNN Encoder-Decoder that consists of two recurrent neural networks (RNN). One RNN encodes a sequence of symbols into a fixed-length vector representation, and the other decodes the representation into another sequence of symbols. The encoder and decoder of t…
SIVAE integrates sentences and their syntactic trees for improved text generation.
problem Improving the grammar of generated text.
method SIVAE uses two separate latent spaces for sentences and syntactic trees, optimizing a joint distribution with two encoders and two decoders.
result SIVAE generates sentences with better grammar compared to existing models.
A simple encoder and complex decoder for secure image encryption and decryption.
problem Secure and efficient image encryption and decryption.
method Uses a shallow encoder neural network for encryption and a deep decoder for decryption, trained independently.
result Decrypted images are nearly identical to the original, demonstrating the effectiveness of the framework.
Although neural machine translation with the encoder-decoder framework has achieved great success recently, it still suffers drawbacks of forgetting distant information, which is an inherent disadvantage of recurrent neural network structure, and disregarding relationship between source words during encoding step. Wher…
NP2MT uses phrase attention to translate phrases efficiently.
problem Phrase-based machine translation inefficiency and performance degradation across different distributions.
method Phrase attention mechanism and dynamic programming for efficient decoding.
result NP2MT achieves comparable performance and outperforms state-of-the-art methods in certain conditions. As a technology to read brain states from measurable brain activities, brain decoding are widely applied in industries and medical sciences. In spite of high demands in these applications for a universal decoder that can be applied to all individuals simultaneously, large variation in brain activities across individual…
Unified framework explains geometric properties of CNNs.
problem Understanding why encoder-decoder CNNs perform well.
method Unified mathematical framework based on recent neural network theories.
result Encoder-decoder CNNs are related to nonlinear basis representation using combinatorial convolution frames.
This work proposes a method to control gradient descent in neural networks using a decoder with shared weights.
problem Efficient few-shot learning and model adaptation in meta-learning.
method A decoder network is used to control the gradient descent process of model parameters in a low-dimensional latent space, sharing weights to reduce parameter requirements.
result The proposed approach outperforms existing methods in tasks like Omniglot and miniImageNet classification.
CGNP embeds functional processes into latent vectors using graph neural networks.
problem Embedding and sampling functional processes over arbitrary domains.
method CGNP employs graph neural networks to embed and decode functional processes.
result CGNP effectively samples encoded functions over any domain.
Deep learning aids ADMM-based decoding for binary linear codes.
problem Improving decoding efficiency for binary linear codes.
method Designing a decoding network based on ADMM and deep learning.
result Numerical results show improved performance compared to original ADMM.
C3 compresses images and videos with low complexity and high performance.
problem High complexity and low performance in neural compression models.
method Overfits a small model to each image or video separately, improving RD performance with low complexity.
result Matches the RD performance of state-of-the-art neural and video codecs with significantly lower decoding complexity.
Recent advances in conditional recurrent language modelling have mainly focused on network architectures (e.g., attention mechanism), learning algorithms (e.g., scheduled sampling and sequence-level training) and novel applications (e.g., image/video description generation, speech recognition, etc.) On the other hand, …
Proposes Decodable Information Bottleneck for optimal representation learning.
problem Finding optimal representations for supervised learning.
method Integrates information retention and compression with the desired predictive family.
result Optimal representations lead to better expected test performance and can be estimated with guarantees.