Beam search is widely used for approximate decoding in structured prediction problems. Models often use a beam at test time but ignore its existence at train time, and therefore do not explicitly learn how to use the beam. We develop an unifying meta-algorithm for learning beam search policies using imitation learning.…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “beam search”
Beam search improves feature selection for better model performance.
problem Improving feature selection for better model performance.
method Proposed beam search as a generalization of forward selection for feature selection.
result Beam search can outperform forward selection, especially with correlated features.
Optimizes tree models for better beam search performance.
problem Beam search causes retrieval performance deterioration in tree models.
method Develops Bayes optimality and calibration under beam search, proposes a novel algorithm for optimal tree model learning.
result Eliminates the training-testing discrepancy in tree models.
Don't Throw Away Your Beams: Improving Consistency-based Uncertainties in LLMs via Beam Searchstat.ML
Beam search improves UQ in LLMs by reducing duplicates and variance.
problem Peaked distributions in multinomial sampling lead to duplicates and high variance in uncertainty estimates.
method Employ beam search to generate candidates for consistency-based UQ, providing a theoretical lower bound and empirical evaluation.
result Beam search achieves smaller error than multinomial sampling, leading to state-of-the-art UQ performance.
Attention-based encoder decoder network uses a left-to-right beam search algorithm in the inference step. The current beam search expands hypotheses and traverses the expanded hypotheses at the next time step. This traversal is implemented using a for-loop program in general, and it leads to speed down of the recogniti…
Federated LIDAR aided beam selection reduces mmWave beam search overhead.
problem Efficient link configuration in mmWave communication systems with reduced beam search overhead.
method Federated learning of LIDAR data to train a shared neural network for beam selection.
result Proposed method significantly outperforms previous works in performance and complexity.
This study mainly investigates two common decoding problems in neural keyphrase generation: sequence length bias and beam diversity. To tackle the problems, we introduce a beam search decoding strategy based on word-level and ngram-level reward function to constrain and refine Seq2Seq inference at test time. Results sh…
We introduce a new sampling method for large language models that balances diversity and parallelism.
problem Balancing diversity and parallelism in decoding for large language models.
method Arithmetic sampling framework compatible with various sampling variations.
result Improves estimation of expected BLEU score reward and reduces the gap with beam search.
The well-known Gumbel-Max trick for sampling from a categorical distribution can be extended to sample elements without replacement. We show how to implicitly apply this 'Gumbel-Top-' trick on a factorized distribution over sequences, allowing to draw exact samples without replacement using a Stochastic Beam Sea…
Bardo Composer generates tabletop RPG music based on player speech.
problem Creating immersive background music for tabletop RPGs.
method Speech recognition, emotion classification, and music generation using a novel beam search algorithm.
result Generated music pieces can be accurately identified by human subjects as conveying the intended emotion.
Adaptive contrastive search improves text generation quality and diversity.
problem Decoding high-quality text from language model outputs.
method Adaptive contrastive search incorporating uncertainty-guided degeneration penalty.
result Enhanced text generation quality and diversity across different models and datasets.
We study the calibration of several state of the art neural machine translation(NMT) systems built on attention-based encoder-decoder models. For structured outputs like in NMT, calibration is important not just for reliable confidence with predictions, but also for proper functioning of beam-search inference. We show …
Neural text generation is a key tool in natural language applications, but it is well known there are major problems at its core. In particular, standard likelihood training and decoding leads to dull and repetitive outputs. While some post-hoc fixes have been proposed, in particular top- and nucleus sampling, they …
A new framework predicts hidden Markov model regimes online.
problem Efficiently identify hidden Markov model regimes in streaming data.
method Develops a predictive-first optimisation framework for streaming HMMs, approximating the full posterior predictive distribution.
result The method provides competitive prequential performance compared to Online EM and Sequential Monte Carlo.
A novel beam training scheme optimizes multi-hop THz communications with up to 75% performance gain.
problem Optimizing beam training for multi-hop THz communications with high data rates and low time overhead.
method Developed a reinforcement learning-based hierarchical beam training scheme with dynamic training levels.
result The proposed scheme achieves up to 75% performance gain in spectral efficiency compared to conventional methods.
Sequence-to-Sequence (seq2seq) modeling has rapidly become an important general-purpose NLP tool that has proven effective for many text-generation and sequence-labeling tasks. Seq2seq builds on deep neural language modeling and inherits its remarkable accuracy in estimating local, next-word distributions. In this work…
Globally normalized neural sequence models are considered superior to their locally normalized equivalents because they may ameliorate the effects of label bias. However, when considering high-capacity neural parametrizations that condition on the whole input sequence, both model classes are theoretically equivalent in…
OKRidge solves sparse ridge regression problems for nonlinear systems.
problem Identifying sparse governing equations for nonlinear dynamical systems.
method OKRidge algorithm using saddle point formulation and ADMM-based approach with efficient proximal operators.
result OKRidge achieves provable optimality with significantly faster run times than Gurobi.
Proposes a method to generate diverse translations by conditioning on target domain.
problem NMT models lack diversity in translations, even with search algorithms.
method Condition the decoder on a latent variable representing target domain, generated by a target encoder.
result Generated diverse translations without affecting performance or training time.
New seq2seq model can copy entire spans, outperforming simpler models in editing tasks.
problem Editing documents or source code using seq2seq models with explicit token copying.
method Extended seq2seq model capable of copying entire input spans to output in one step, new training and inference methods.
result New model consistently outperforms simpler baselines in editing tasks of natural language and source code.
A neural program synthesis method with iterative fix operations.
problem Creating correct programs from input-output examples.
method Combines encoder-decoder synthesis with a differentiable fixer.
result Improves synthesis accuracy by reducing discrepancies between outputs and desired outputs.
Bayesian method adapts to unknown distribution shifts in online learning.
problem Online learning with unknown and irregular distribution shifts.
method Bayesian inference with change-point detection and beam search.
result Improves adaptation to new data distributions over state-of-the-art methods.
New model uses pretrained biochemical language models to generate drug compounds.
problem Developing novel compounds targeting specific proteins.
method Exploits pretrained language models to initialize and fine-tune targeted molecule generation models.
result Warm-started models outperform baseline models, with one-stage strategy showing better generalization.
Sequence-to-sequence text-to-speech (TTS) is dominated by soft-attention-based methods. Recently, hard-attention-based methods have been proposed to prevent fatal alignment errors, but their sampling method of discrete alignment is poorly investigated. This research investigates various combinations of sampling methods…
Adversarial attacks reduce deep learning beam selection performance in mmWave 5G.
problem Adversarial attacks degrade deep learning-based beam selection in mmWave 5G.
method Generates adversarial perturbations to RSS inputs to manipulate DNN predictions.
result Significant reduction in IA performance due to adversarial perturbations.
Growing Together: Modeling Human Language Learning With n-Best Multi-Checkpoint Machine Translationcs.CL
Modeling human language learning with multi-checkpoint machine translation.
problem Improving machine translation quality for language education.
method Ensemble of multi-checkpoints from a single model, sampling n-best sequences.
result Achieved 37.57 macro F1 score, outperforming baseline.
Deep Retrieval learns a retrievable structure for efficient large-scale recommendations.
problem Efficiently retrieving top relevant candidates in large-scale recommendation systems.
method Deep Retrieval learns a retrievable structure directly from user-item interaction data, encoding candidates into a discrete latent space and optimizing a model to maximize accuracy.
result Deep Retrieval achieves almost the same accuracy as brute-force baseline and significantly outperforms ANN baselines in a live production system.
A new score SiNNE improves OAM efficiency and accuracy.
problem Comparing outlier scores across subspaces of different dimensions.
method Introducing SiNNE, a new score independent of subspace dimensionality.
result SiNNE produces better or at least as good results as existing scores and significantly improves runtime.
We present a supervised neural network model for polyphonic piano music transcription. The architecture of the proposed model is analogous to speech recognition systems and comprises an acoustic model and a music language model. The acoustic model is a neural network used for estimating the probabilities of pitches in …
Active learning improves neutron spectroscopy experiments by automating measurement selection.
problem Efficiently searching for signals in neutron scattering experiments with limited beam time.
method Probabilistic active learning using log-Gaussian processes.
result Automated selection of informative measurements improves experimental efficiency.
Having a sequence-to-sequence model which can operate in an online fashion is important for streaming applications such as Voice Search. Neural transducer is a streaming sequence-to-sequence model, but has shown a significant degradation in performance compared to non-streaming models such as Listen, Attend and Spell (…
Physics-informed model predicts beam stiffness and monitors structural health.
problem Predicting and monitoring the stiffness of Euler-Bernoulli beams.
method Physics-informed Gaussian process model using the Euler-Bernoulli beam equation.
result Model accurately predicts bending stiffness and detects structural damage.
Finding the diameter of a dataset in multidimensional Euclidean space is a well-established problem, with well-known algorithms. However, most of the algorithms found in the literature do not scale well with large values of data dimension, so the time complexity grows exponentially in most cases, which makes these algo…
Generalizes beam models to include curvature and torsion.
problem Modeling curvature and torsion in unidimensional structures.
method Generalized Euler-Bernoulli and Timoshenko beam models using non-holonomic kinematics.
result Generalized beams exhibit curvature and torsion, with torsion in Timoshenko model.
In this paper we introduce various techniques to improve the performance of electroencephalography (EEG) features based continuous speech recognition (CSR) systems. A connectionist temporal classification (CTC) based automatic speech recognition (ASR) system was implemented for performing recognition. We introduce tech…
Gaussian beams describe the amplitude and phase of rays and are widely used to model acoustic propagation. This paper describes four new results in the theory of Gaussian beams. (1) A new version of the Červený equations for the amplitude and phase of Gaussian beams is developed by applying the equivalence of Hamilton-…
We propose a method to build quantum memristors in quantum photonic platforms. We firstly design an effective beam splitter, which is tunable in real-time, by means of a Mach-Zehnder-type array with two equal 50:50 beam splitters and a tunable retarder, which allows us to control its reflectivity. Then, we show that th…
ASKAP observes a region of the Galactic plane, identifying 3963 radio sources.
problem Characterizing radio sources in the Galactic plane using ASKAP observations.
method 912 MHz observations with 15 ASKAP antennas, source characterization using CAESAR.
result Differential source counts in agreement with previous surveys, spectral index estimation for sources.
The article recovers tensor fields from partial data using weighted divergent ray transforms.
problem Recovering tensor fields from partial data.
method Weighted divergent ray transforms, unique continuation property of fractional Laplacian, explicit reconstruction formulas.
result Recovery of symmetric -tensor fields and unique continuation for vector fields and symmetric 2-tensor fields.
New algorithm optimizes beam and rate allocation in mmWave systems for multiple users.
problem Optimizing beam and rate allocation in mmWave systems for multiple users with limited feedback.
method Introducing SAT-CTS, a combinatorial semi-bandit policy with satisficing objective.
result SAT-CTS achieves finite-time regret bounds and reduces satisficing regret in mmWave systems.
Despite strong performance on a variety of tasks, neural sequence models trained with maximum likelihood have been shown to exhibit issues such as length bias and degenerate repetition. We study the related issue of receiving infinite-length sequences from a recurrent language model when using common decoding algorithm…
We address the problem of combining sequence models of symbolic music with user defined constraints. For typical models this is non-trivial as only the conditional distribution of each symbol given the earlier symbols is available, while the constraints correspond to arbitrary times. Previously this has been addressed …
Restricted Boltzmann Machines (RBMs) are a class of generative neural network that are typically trained to maximize a log-likelihood objective function. We argue that likelihood-based training strategies may fail because the objective does not sufficiently penalize models that place a high probability in regions where…
This paper generalizes Michell Truss to higher dimensions using geometric measure theory.
problem Finding optimal designs of k-beam structures under equilibrium forces.
method Geometric measure theory and flat chain complex.
result Existence of optimal k-beam structures has been solved completely.
Two novel search strategies reduce complexity for target localization with size-dependent noise.
problem Target localization with varying measurement noise based on query region size.
method Proposes and strategies with low complexity and connected query geometry.
result Unified analysis shows asymptotically optimal in search time, near-optimal in rate.
We stabilize the activations of Recurrent Neural Networks (RNNs) by penalizing the squared distance between successive hidden states' norms. This penalty term is an effective regularizer for RNNs including LSTMs and IRNNs, improving performance on character-level language modeling and phoneme recognition, and outperfor…
This paper introduces a new learning-based approach for approximately solving the Travelling Salesman Problem on 2D Euclidean graphs. We use deep Graph Convolutional Networks to build efficient TSP graph representations and output tours in a non-autoregressive manner via highly parallelized beam search. Our approach ou…
Physics-constrained neural nets solve EM fields of charged particle beams.
problem Solving Maxwell's equations for intense charged particle beams.
method 3D Convolutional Neural Networks (CNNs) constrained by physics.
result 3D CNNs generate electromagnetic fields from current and charge densities.