CARDS improves decoding efficiency and alignment quality for LLMs.
problem Efficiency bottlenecks in decoding-time alignment for LLMs.
method Cascade Reward Sampling (CARDS) with segment-level rejection sampling and uncertainty-based segmentation.
result Significant improvement in decoding efficiency and alignment quality.
Efficient decoder design improves semantic segmentation for real-time automated driving.
problem Efficient design of decoders for real-time semantic segmentation in automated driving.
method Design of a novel non-bottleneck layer and family of decoders using VGG10 as an efficient encoder.
result Improvement of 10% in performance compared to a baseline.
Entropy-based decoding improves DLM sampling efficiency.
problem Decoding strategy challenges in flexible DLMs.
method Entropy sum-based confidence-based decoding.
result Entropy sum-based decoding achieves ε \varepsilon ε -accuracy with O ~ ( H ( X 0 ) / ε ) \widetilde O(H(X_0)/\varepsilon) O ( H ( X 0 ) / ε ) iterations. Finding optimal correction of errors in generic stabilizer codes is a computationally hard problem, even for simple noise models. While this task can be simplified for codes with some structure, such as topological stabilizer codes, developing good and efficient decoders still remains a challenge. In our work, we syste…
This work proposes an efficient autoregressive model for text generation.
problem The challenge of generating high-quality text with autoregressive models.
method Introduces a cascaded decoding approach using Markov transformers to achieve sub-linear parallel time generation.
result Shows competitive accuracy/speed tradeoff compared to existing methods on five machine translation datasets.
RL-VAE uses RL to decode molecular graphs from latent embeddings.
problem Efficiently decoding molecular graphs from latent embeddings.
method Repurposed simple graph generator for efficient decoding.
result Decoding molecular graphs from latent embeddings is possible with a simple graph generator.
Parallel decoding improves machine translation efficiency and accuracy.
problem Efficiently generating translations from left to right.
method Conditional masked language modeling for parallel decoding.
result Improves translation performance by over 4 BLEU points.
RED-SC improves deep subspace clustering efficiency.
problem Slow convergence in training deep subspace clustering models.
method Residual Encoder-Decoder network with skip-layer connections and self-expressive layer.
result Training converges much faster with RED-SC.
Paper presents an efficient approach for integrating LSTM language models in LVCSR systems.
problem Difficulty in incorporating LSTM language models efficiently into LVCSR systems.
method First-pass decoding with LSTM-LM followed by lattice rescoring and hypothesis recombination.
result Competitive results on Hub5'00 and Librispeech corpora with runtime better than real-time.
NP 2 ^2 2 MT uses phrase attention to translate phrases efficiently.
problem Phrase-based machine translation inefficiency and performance degradation across different distributions.
method Phrase attention mechanism and dynamic programming for efficient decoding.
result NP 2 ^2 2 MT achieves comparable performance and outperforms state-of-the-art methods in certain conditions. Improved GP decoder training with SAS approximations.
problem Training expensive Gaussian process decoders is challenging and computationally expensive.
method Developed a new stochastic estimate of log-marginal likelihood based on cross-validation.
result SAS-GP improves robustness and reduces computational cost compared to variational autoencoders.
New method aligns brain data across individuals for better brain decoding.
problem Inter-individual variability in brain response patterns limits decoder generalization.
method SpectralOT method that embeds cortical geometry into Laplace-Beltrami eigenmodes.
result SpectralOT strikes balance between aligning functional features and preserving anatomical structure.
This paper analyzes speculative decoding, a method to speed up large language model inferences.
problem Theoretical understanding of speculative decoding is lacking.
method Conceptualizes speculative decoding as a markov chain problem and studies its key properties.
result Reveals fundamental connections between LLM components and their impact on decoding efficiency.
Accelerates TPP sampling with speculative decoding for faster sequence generation.
problem Efficiently sampling from complex temporal point processes.
method Adapting speculative decoding techniques from language models to TPPs.
result Achieves significant speedup (2-6x) while maintaining distributional accuracy.
Constrained sequence codes have been widely used in modern communication and data storage systems. Sequences encoded with constrained sequence codes satisfy constraints imposed by the physical channel, hence enabling efficient and reliable transmission of coded symbols. Traditional encoding and decoding of constrained …
Generative AI decodes quantum codes without labeled data.
problem Efficient decoding of quantum error-correcting codes.
method Generative Transformers learn logical operators from unsupervised syndromes.
result Significantly better decoding accuracy than traditional methods.
Paper develops a decoder for sparse codes without encoder matrix, achieving optimal recovery.
problem Designing a decoder for sparse codes from linear measurements alone.
method Matrix factorization to recover encoder and sparse coding matrices from measurements.
result Decoder-Expander Based Factorisation recovers encoder and sparse coding matrix at optimal measurement rate with high probability.
In extreme classification problems, learning algorithms are required to map instances to labels from an extremely large label set. We build on a recent extreme classification framework with logarithmic time and space, and on a general approach for error correcting output coding (ECOC) with loss-based decoding, and intr…
Finding efficient decoders for quantum error correcting codes adapted to realistic experimental noise in fault-tolerant devices represents a significant challenge. In this paper we introduce several decoding algorithms complemented by deep neural decoders and apply them to analyze several fault-tolerant error correctio…
Improved unlearning in LLMs using contrastive decoding.
problem Removing specific information from LLMs while preserving overall performance.
method Inference-time unlearning algorithm using contrastive decoding with two auxiliary models.
result Notable gains in forget quality and retained performance compared to prior approaches.
High-performance quantum codes decoded with minimal data.
problem Efficient decoding of linear-rate LDPC quantum codes.
method Tessellations of hyperbolic manifolds, Coxeter groups, and Galois fields.
result Achieved encoding rate of 13/72 with high performance.
New algorithms decode Markov chains with near-optimal performance, even with small latency.
problem Online decoding of n t h n^{th} n t h order ergodic Markov chains with latency constraints. method Deterministic and randomized algorithms using dynamic programs, with lower bounds established.
result Near-optimal performance of algorithms with minimal latency, outperforming existing methods.
VAD uses efficient inference to learn from incomplete data.
problem Learning generative models from partial data.
method Variational Auto-Decoder framework using Variational Bayes and MCMC methods.
result VAD reduces inference computational cost and allows for more complex optimization.
Algorithm improves continuous control with context, achieving provable efficiency.
problem Building an agent that generalizes to unseen environments in continuous control.
method Uses upper confidence bound (UCB) to refine decoder estimates and balance exploration-exploitation.
result Proves $\widetilde{O}\left(\sqrt{T}
ight)$ regret bound and ε ε ε -suboptimal policy transfer. Study efficient neural operator learning using variation spaces.
problem Operator learning using encoder-decoder neural networks.
method Introduce variation space for nonlinear operators, establish approximation bounds.
result Algebraic approximation and learning rates for polynomially decaying input and output encoding errors.
Efficient autoregressive entity linking with correction for faster, more accurate results.
problem High computational cost and non-parallelizable decoding in autoregressive entity linking.
method Parallelizes autoregressive linking across all mentions, uses a shallow decoder, and adds a discriminative correction term.
result 70 times faster and more accurate than previous methods, outperforming state-of-the-art approaches.
GSI improves efficiency of large language model inference.
problem Efficiently guiding test-time alignment in large language models.
method Combines soft best-of- n n n scaling with a reward model and speculative samples. result Achieves higher accuracy and reduced latency compared to standard methods.
QATS efficiently decodes HMMs with polylogarithmic complexity.
problem Efficiently decoding hidden Markov models from noisy observations.
method Divide-and-conquer procedure with polylogarithmic sequence complexity and cubic state space complexity.
result QATS outperforms Viterbi and PMAP in speed and accuracy.
Gradient flow autoencoder improves data efficiency over traditional autoencoders.
problem Sub-optimal latent space representations in autoencoders.
method Gradient flow through ODE with adaptive step size for optimization.
result Gradient flow autoencoder achieves higher data efficiency.
A high-parallelism SNN improves feature learning efficiency and robustness.
problem Slow learning speed and limited learning capability in existing SNNs.
method Inspired by Inception modules, high-parallelism architecture, Vote-for-All decoding, adaptive repolarization mechanism.
result Superior performance and competitive accuracy compared to state-of-the-art unsupervised SNNs.
Polynomial-time algorithm for list-decodable linear regression with batches.
problem Efficiently decoding linear regression with a fraction of adversarial data.
method Polynomial time algorithm using batches of i.i.d. samples.
result Returns a list of size O(1/α^2) with one item close to true parameter.
DOS improves language model generation by considering inter-token dependencies.
problem Lack of sequence-level information and inter-token dependencies in existing decoding strategies.
method Dependency-Oriented Sampler (DOS) that uses attention matrices to approximate inter-token dependencies.
result DOS consistently achieves superior performance on code generation and mathematical reasoning tasks.
A new method directly encodes data into latent space using gradient flow.
problem Suboptimal representations in physical sciences due to encoder inversion.
method Decoder-only approach using gradient flow and ODEs, avoiding integrals.
result Superior data efficiency and explicit encoding compared to traditional autoencoders.
New method reduces neural image compression run-time by 50%.
problem Computational efficiency of neural image compression models.
method Automatic network optimization to reduce decoder complexity.
result Decreased decoder run-time by over 50%.
Many machine learning tasks can be formulated in terms of predicting structured outputs. In frameworks such as the structured support vector machine (SVM-Struct) and the structured perceptron, discriminative functions are learned by iteratively applying efficient maximum a posteriori (MAP) decoding. However, maximum li…
Low-density parity-check codes, a class of capacity-approaching linear codes, are particularly recognized for their efficient decoding scheme. The decoding scheme, known as the sum-product, is an iterative algorithm consisting of passing messages between variable and check nodes of the factor graph. The sum-product alg…
Algorithm estimates covariance from noisy data efficiently.
problem Estimating covariance from a noisy set of points.
method Spectral techniques for list-decodable covariance estimation.
result Efficient algorithm with poly(1/α) sample and time complexity.
Unified framework for learning function representations using INRs and Transformers.
problem Scalability and efficiency limitations in existing generative models.
method Integrates INRs and Transformer-based hypernetworks into latent variable models.
result Improved scalability, expressiveness, and generalization over existing models.
Paper tackles NAT translation issues with auxiliary regularization.
problem Improves NAT translation quality by addressing repeated and incomplete translations.
method Improves decoder hidden representations via two auxiliary regularization terms.
result Significant improvement in NAT model accuracy with better inference efficiency.
CRRN detects printer defects in SPI data of SMT boards.
problem Detecting defects in solder paste inspection data.
method Convolutional Recurrent Reconstructive Network (CRRN) with CSTMs and ST-Attention.
result CRRN outperforms conventional models in anomaly detection.
A machine learning method predicts rock permeability from 3D images.
problem Efficiently predict permeability of heterogeneous rocks for planetary and robotic applications.
method Machine learning guided 3D properties recognition of rock morphology from 3D micro CT and MRI images.
result The morphology decoder method accurately predicts permeability from 3D images.
RichID learns optimal control policies from nonlinear observations.
problem Continuous control with unknown nonlinearity in system model.
method RichID algorithm using a least-squares regression oracle.
result First provable sample complexity guarantee for continuous control.
CSLVAE generates large chemical libraries efficiently.
problem Navigating ultra-large combinatorial synthesis libraries.
method Hierarchically-organized database with molecular encoder and decoder.
result Generates valid molecular graphs without autoregression.
We present the Video Ladder Network (VLN) for efficiently generating future video frames. VLN is a neural encoder-decoder model augmented at all layers by both recurrent and feedforward lateral connections. At each layer, these connections form a lateral recurrent residual block, where the feedforward connection repres…
Neural network VQ-VAE with WaveNet decodes speech at 1.6 kbps with high quality.
problem Efficiently transmitting and storing speech signals at low bit-rates.
method VQ-VAE and WaveNet architecture for speech coding.
result Speech coding at 1.6 kbps with perceptual quality between MELP and AMR-WB.
ConformalHDC improves HDC's uncertainty quantification for neuromorphic learning.
problem Lack of rigorous uncertainty quantification in Hyperdimensional Computing.
method Combines conformal prediction with HDC's efficiency, proposing set-valued and point-valued formulations.
result Demonstrates improved robustness and accuracy in decoding neural stimulus information.
New method for estimating sparse means in noisy data.
problem Estimating the mean of a sparse distribution in the presence of outliers.
method Difference-of-Pairs Filtering technique for list-decodable sparse mean estimation.
result First sample and computationally efficient algorithm for list-decodable sparse mean estimation.
Method learns latent states from rich observations to improve RL exploration.
problem Improving RL performance with rich observations and latent states.
method Estimates latent states from observations through regression and clustering, providing finite-sample guarantees.
result Exponential improvement over Q Q Q -learning with naïve exploration.