Beam search improves UQ in LLMs by reducing duplicates and variance.
problem Peaked distributions in multinomial sampling lead to duplicates and high variance in uncertainty estimates.
method Employ beam search to generate candidates for consistency-based UQ, providing a theoretical lower bound and empirical evaluation.
result Beam search achieves smaller error than multinomial sampling, leading to state-of-the-art UQ performance.
A new method samples sequences without replacement using Gumbel-Top-k trick.
problem Sampling sequences without replacement from large domains efficiently.
method Gumbel-Top-k trick applied to Stochastic Beam Search.
result Efficient sampling with linear model evaluations in k. Beam search is widely used for approximate decoding in structured prediction problems. Models often use a beam at test time but ignore its existence at train time, and therefore do not explicitly learn how to use the beam. We develop an unifying meta-algorithm for learning beam search policies using imitation learning.…
Beam search improves feature selection for better model performance.
problem Improving feature selection for better model performance.
method Proposed beam search as a generalization of forward selection for feature selection.
result Beam search can outperform forward selection, especially with correlated features.
Improved neural keyphrase generation by beam search with reward functions.
problem Sequence length bias and beam diversity issues in neural keyphrase generation.
method Beam search decoding strategy with word-level and ngram-level reward functions.
result Significant improvement in generating diverse and accurate keyphrases.
Bardo Composer generates tabletop RPG music based on player speech.
problem Creating immersive background music for tabletop RPGs.
method Speech recognition, emotion classification, and music generation using a novel beam search algorithm.
result Generated music pieces can be accurately identified by human subjects as conveying the intended emotion.
Optimizes tree models for better beam search performance.
problem Beam search causes retrieval performance deterioration in tree models.
method Develops Bayes optimality and calibration under beam search, proposes a novel algorithm for optimal tree model learning.
result Eliminates the training-testing discrepancy in tree models.
This research explores various sampling methods and probability distributions for hard alignment in sequence-to-sequence TTS synthesis.
problem Improving alignment accuracy in sequence-to-sequence text-to-speech synthesis.
method Investigated various sampling methods (greedy, beam, random) and probability distributions (Bernoulli, Concrete) for hard alignment.
result Deterministic search is more preferable than stochastic search for natural alignment transition.
Attention-based encoder decoder network uses a left-to-right beam search algorithm in the inference step. The current beam search expands hypotheses and traverses the expanded hypotheses at the next time step. This traversal is implemented using a for-loop program in general, and it leads to speed down of the recogniti…
Federated LIDAR aided beam selection reduces mmWave beam search overhead.
problem Efficient link configuration in mmWave communication systems with reduced beam search overhead.
method Federated learning of LIDAR data to train a shared neural network for beam selection.
result Proposed method significantly outperforms previous works in performance and complexity.
Paper improves EEG-based speech recognition using CTC and beam search.
problem Improving continuous speech recognition from EEG signals.
method Implemented CTC-based ASR system, initialized weights, used external language model, studied articulatory feature prediction.
result Enhanced performance of EEG-based speech recognition systems.
We introduce a new sampling method for large language models that balances diversity and parallelism.
problem Balancing diversity and parallelism in decoding for large language models.
method Arithmetic sampling framework compatible with various sampling variations.
result Improves estimation of expected BLEU score reward and reduces the gap with beam search.
Study improves calibration of NMT models, especially EOS and attention uncertainty.
problem Improper functioning of beam-search inference due to miscalibrated NMT models.
method Design and apply recalibration methods based on EOS and attention uncertainty signals.
result Improved accuracy and better sequence-level calibration of NMT models.
New method improves text generation by assigning lower probability to repetitive sequences.
problem Dull and repetitive text generated by standard likelihood training.
method Unlikelihood training, which forces unlikely generations to be assigned lower probability.
result Unlikelihood training produces less repetitive, less dull text compared to standard techniques.
This study investigates global normalization in neural models, showing its effectiveness in search-aware training.
problem Theoretical equivalence of global and local normalization in high-capacity models, practical advantage unclear.
method Continuous relaxation of beam search for training globally normalized recurrent sequence models.
result Globally normalized models are more effective than locally normalized ones in inexact search.
Adaptive contrastive search improves text generation quality and diversity.
problem Decoding high-quality text from language model outputs.
method Adaptive contrastive search incorporating uncertainty-guided degeneration penalty.
result Enhanced text generation quality and diversity across different models and datasets.
A new framework predicts hidden Markov model regimes online.
problem Efficiently identify hidden Markov model regimes in streaming data.
method Develops a predictive-first optimisation framework for streaming HMMs, approximating the full posterior predictive distribution.
result The method provides competitive prequential performance compared to Online EM and Sequential Monte Carlo.
A novel beam training scheme optimizes multi-hop THz communications with up to 75% performance gain.
problem Optimizing beam training for multi-hop THz communications with high data rates and low time overhead.
method Developed a reinforcement learning-based hierarchical beam training scheme with dynamic training levels.
result The proposed scheme achieves up to 75% performance gain in spectral efficiency compared to conventional methods.
Sequence-to-Sequence (seq2seq) modeling has rapidly become an important general-purpose NLP tool that has proven effective for many text-generation and sequence-labeling tasks. Seq2seq builds on deep neural language modeling and inherits its remarkable accuracy in estimating local, next-word distributions. In this work…
OKRidge solves sparse ridge regression problems for nonlinear systems.
problem Identifying sparse governing equations for nonlinear dynamical systems.
method OKRidge algorithm using saddle point formulation and ADMM-based approach with efficient proximal operators.
result OKRidge achieves provable optimality with significantly faster run times than Gurobi.
Proposes a method to generate diverse translations by conditioning on target domain.
problem NMT models lack diversity in translations, even with search algorithms.
method Condition the decoder on a latent variable representing target domain, generated by a target encoder.
result Generated diverse translations without affecting performance or training time.
New seq2seq model can copy entire spans, outperforming simpler models in editing tasks.
problem Editing documents or source code using seq2seq models with explicit token copying.
method Extended seq2seq model capable of copying entire input spans to output in one step, new training and inference methods.
result New model consistently outperforms simpler baselines in editing tasks of natural language and source code.
A neural program synthesis method with iterative fix operations.
problem Creating correct programs from input-output examples.
method Combines encoder-decoder synthesis with a differentiable fixer.
result Improves synthesis accuracy by reducing discrepancies between outputs and desired outputs.
Bayesian method adapts to unknown distribution shifts in online learning.
problem Online learning with unknown and irregular distribution shifts.
method Bayesian inference with change-point detection and beam search.
result Improves adaptation to new data distributions over state-of-the-art methods.
AutoCross automatically generates feature interactions for tabular data.
problem Capturing interactions among categorical features in tabular data.
method Beam search in a tree-structured space, successive mini-batch gradient descent, multi-granularity discretization.
result Significantly enhances performance of linear and deep models.
New model uses pretrained biochemical language models to generate drug compounds.
problem Developing novel compounds targeting specific proteins.
method Exploits pretrained language models to initialize and fine-tune targeted molecule generation models.
result Warm-started models outperform baseline models, with one-stage strategy showing better generalization.
Adversarial attacks reduce deep learning beam selection performance in mmWave 5G.
problem Adversarial attacks degrade deep learning-based beam selection in mmWave 5G.
method Generates adversarial perturbations to RSS inputs to manipulate DNN predictions.
result Significant reduction in IA performance due to adversarial perturbations.
Modeling human language learning with multi-checkpoint machine translation.
problem Improving machine translation quality for language education.
method Ensemble of multi-checkpoints from a single model, sampling n-best sequences.
result Achieved 37.57 macro F1 score, outperforming baseline.
Deep Retrieval learns a retrievable structure for efficient large-scale recommendations.
problem Efficiently retrieving top relevant candidates in large-scale recommendation systems.
method Deep Retrieval learns a retrievable structure directly from user-item interaction data, encoding candidates into a discrete latent space and optimizing a model to maximize accuracy.
result Deep Retrieval achieves almost the same accuracy as brute-force baseline and significantly outperforms ANN baselines in a live production system.
Deep RL learns 2-opt heuristics to improve TSP solutions.
problem Improving TSP solutions beyond initial heuristics.
method Deep reinforcement learning to learn 2-opt operations.
result Learned policies improve solutions faster than previous methods.
A new score SiNNE improves OAM efficiency and accuracy.
problem Comparing outlier scores across subspaces of different dimensions.
method Introducing SiNNE, a new score independent of subspace dimensionality.
result SiNNE produces better or at least as good results as existing scores and significantly improves runtime.
We present a supervised neural network model for polyphonic piano music transcription. The architecture of the proposed model is analogous to speech recognition systems and comprises an acoustic model and a music language model. The acoustic model is a neural network used for estimating the probabilities of pitches in …
Active learning improves neutron spectroscopy experiments by automating measurement selection.
problem Efficiently searching for signals in neutron scattering experiments with limited beam time.
method Probabilistic active learning using log-Gaussian processes.
result Automated selection of informative measurements improves experimental efficiency.
Having a sequence-to-sequence model which can operate in an online fashion is important for streaming applications such as Voice Search. Neural transducer is a streaming sequence-to-sequence model, but has shown a significant degradation in performance compared to non-streaming models such as Listen, Attend and Spell (…
Physics-informed model predicts beam stiffness and monitors structural health.
problem Predicting and monitoring the stiffness of Euler-Bernoulli beams.
method Physics-informed Gaussian process model using the Euler-Bernoulli beam equation.
result Model accurately predicts bending stiffness and detects structural damage.
Finding the diameter of a dataset in multidimensional Euclidean space is a well-established problem, with well-known algorithms. However, most of the algorithms found in the literature do not scale well with large values of data dimension, so the time complexity grows exponentially in most cases, which makes these algo…
Generalizes beam models to include curvature and torsion.
problem Modeling curvature and torsion in unidimensional structures.
method Generalized Euler-Bernoulli and Timoshenko beam models using non-holonomic kinematics.
result Generalized beams exhibit curvature and torsion, with torsion in Timoshenko model.
We developed a new quantum annealing (QA) algorithm for Dirichlet process mixture (DPM) models based on the Chinese restaurant process (CRP). QA is a parallelized extension of simulated annealing (SA), i.e., it is a parallel stochastic optimization technique. Existing approaches [Kurihara et al. UAI2009, Sato et al. UA…
Gaussian beams describe the amplitude and phase of rays and are widely used to model acoustic propagation. This paper describes four new results in the theory of Gaussian beams. (1) A new version of the Červený equations for the amplitude and phase of Gaussian beams is developed by applying the equivalence of Hamilton-…
We propose a method to build quantum memristors in quantum photonic platforms. We firstly design an effective beam splitter, which is tunable in real-time, by means of a Mach-Zehnder-type array with two equal 50:50 beam splitters and a tunable retarder, which allows us to control its reflectivity. Then, we show that th…
New deep learning method solves TSP faster and more efficiently.
problem Approximately solving the Travelling Salesman Problem on 2D Euclidean graphs.
method Uses Graph Convolutional Networks for efficient TSP graph representations and non-autoregressive beam search.
result Significantly reduces optimality gap for large problem instances.
LineBO tackles high-dimensional Bayesian optimization by solving 1D subproblems.
problem Bayesian optimization struggles in high dimensions due to complex acquisition steps.
method LineBO restricts high-dimensional problems to 1D subproblems iteratively solved efficiently.
result LineBO converges globally and achieves a fast local rate for strongly convex functions.
ASKAP observes a region of the Galactic plane, identifying 3963 radio sources.
problem Characterizing radio sources in the Galactic plane using ASKAP observations.
method 912 MHz observations with 15 ASKAP antennas, source characterization using CAESAR.
result Differential source counts in agreement with previous surveys, spectral index estimation for sources.
Paper investigates a new type of elastica formed during beam rupture.
problem Understanding the shape of a beam during sudden rupture.
method Developed a mathematical theory using elliptic ζ-function.
result Explicit shape of the new elastica (Λ-elastica) is described.
The article recovers tensor fields from partial data using weighted divergent ray transforms.
problem Recovering tensor fields from partial data.
method Weighted divergent ray transforms, unique continuation property of fractional Laplacian, explicit reconstruction formulas.
result Recovery of symmetric m-tensor fields and unique continuation for vector fields and symmetric 2-tensor fields. New algorithm optimizes beam and rate allocation in mmWave systems for multiple users.
problem Optimizing beam and rate allocation in mmWave systems for multiple users with limited feedback.
method Introducing SAT-CTS, a combinatorial semi-bandit policy with satisficing objective.
result SAT-CTS achieves finite-time regret bounds and reduces satisficing regret in mmWave systems.
Restricted Boltzmann Machines (RBMs) are a class of generative neural network that are typically trained to maximize a log-likelihood objective function. We argue that likelihood-based training strategies may fail because the objective does not sufficiently penalize models that place a high probability in regions where…
Two novel search strategies reduce complexity for target localization with size-dependent noise.
problem Target localization with varying measurement noise based on query region size.
method Proposes dyaPM and hiePM strategies with low complexity and connected query geometry. result Unified analysis shows dyaPM asymptotically optimal in search time, hiePM near-optimal in rate.