Classifies intrinsically linked tournaments by their score sequences.
problem Classifying intrinsically linked tournaments using their score sequences.
method Examining the score sequences of tournaments and identifying linkless sequences.
result The vast majority of score sequences for 8-vertex tournaments are linkless.
Algorithm optimizes biological sequences using bootstrapped training with a score-conditioned generator.
problem Optimizing biological sequences for a black-box score function.
method Bootstrapped training of score-conditioned generator (BootGen) algorithm.
result Our method outperforms competitive baselines on biological sequential design tasks.
In this paper we show that reporting a single performance score is insufficient to compare non-deterministic approaches. We demonstrate for common sequence tagging tasks that the seed value for the random number generator can result in statistically significant (p < 10^-4) differences for state-of-the-art systems. For …
OptiGAN uses GAN and RL to optimize sequence generation for specific goals.
problem Challenging in sequence generation tasks to generate sequences with specific desired goals.
method Integrates GAN and RL to optimize desired goal scores using policy gradients.
result Achieves higher desired scores in text and real-valued sequence generation.
Transformer models improve malware classification, especially for imbalanced datasets.
problem Imbalanced multiclass malware classification.
method Bagging-based random transformer forest (RTF) ensemble of pre-trained transformer models (BERT or CANINE).
result Ensemble of pre-trained transformer models achieves state-of-the-art F1-score of 0.6149 on a benchmark dataset.
SMURF-THP improves Transformer Hawkes process models by providing uncertainty quantification.
problem Uncertainty quantification for Transformer Hawkes process predictions.
method Score matching for learning the score function of event arrival times.
result SMURF-THP outperforms likelihood-based methods in confidence calibration.
The paper tackles video prediction by estimating conditional densities implicitly.
problem Temporal prediction uncertainty and high-dimensional probabilistic inference in natural scenes.
method Score-based conditional density estimation using sequence-to-image networks trained on a resilience-to-noise objective.
result The method handles occlusion boundaries and weights predictive evidence by reliability.
Paper compares two forecasters using novel online inference methods.
problem How to compare forecasters without distributional assumptions.
method Confidence sequences and game-theoretic statistical framework for sequential testing.
result Valid methods for comparing forecasters without distributional assumptions.
DLM-One speeds up language generation by 500x with continuous models.
problem Efficiently generating text sequences in natural language processing.
method Score-distillation of continuous diffusion language models.
result Achieves up to 500x speedup in inference time with competitive performance.
TF-MoDISco (Transcription Factor Motif Discovery from Importance Scores) is an algorithm for identifying motifs from basepair-level importance scores computed on genomic sequence data. This technical note focuses on version v0.5.6.5. The implementation is available at https://github.com/kundajelab/tfmodisco/tree/v0.5.6…
Proposes MVG-CRPS for robust multivariate forecasting.
problem Outliers in multivariate forecasting lead to significant errors.
method Integrates CRPS for MVG distributions, optimizing with MVG-CRPS.
result Improves robustness, accuracy, and uncertainty quantification.
The α-Alternator adapts to varying noise levels in sequences, improving robustness and performance.
problem Current models assume uniform noise levels, limiting performance on noisy temporal data.
method Introduces α-Alternator using Vendi Score to dynamically adjust noise sensitivity. result Outperforms Alternators and state-of-the-art models in trajectory prediction, imputation, and forecasting.
A new method optimizes neural sequence models for better task performance.
problem Training neural sequence models with maximum likelihood estimation ignores task losses.
method Maximum likelihood guided parameter search (MGS) in the parameter space.
result MGS optimizes sequence-level losses, reducing repetition and non-termination.
PRAGMA models financial event sequences for various banking tasks.
problem Handling diverse financial data for multiple applications.
method Pre-training a Transformer model on a large banking event corpus with a self-supervised objective.
result PRAGMA achieves superior performance across multiple financial domains from raw event sequences.
Phylogenetic tree reconstruction is traditionally based on multiple sequence alignments (MSAs) and heavily depends on the validity of this information bottleneck. With increasing sequence divergence, the quality of MSAs decays quickly. Alignment-free methods, on the other hand, are based on abstract string comparisons …
SDE Matching eliminates simulation for training Latent SDEs, achieving similar performance.
problem Training Latent SDEs with adjoint sensitivity methods is computationally expensive and limited.
method SDE Matching, inspired by Score- and Flow Matching, eliminates simulation for training Latent SDEs.
result SDE Matching achieves performance comparable to adjoint sensitivity methods while reducing computational complexity.
New approach speeds up DNA sequence alignment.
problem Efficiently estimating alignment scores for large sets of reads.
method Rank-one crowdsourcing models and multi-armed bandit algorithm.
result Adaptive algorithm identifies pairs with large alignment scores.
Deep learning model forecasts PV power production with high accuracy.
problem Accurate forecasting of photovoltaic power production is needed for electricity infrastructure.
method Sequence to sequence model with attention mechanism using weather predictions and historical data.
result The model outperforms existing methods in forecast skill score.
Differentiable language model attacks improve adversarial examples for categorical sequence classifiers.
problem Challenges in adversarial attacks for categorical sequence models due to non-differentiability.
method Fine-tuning a language model as a generator of adversarial examples, using a differentiable loss function that combines a surrogate classifier score and approximate edit distance.
result Semantically better and more resistant adversarial examples.
Sequence-to-Sequence (seq2seq) modeling has rapidly become an important general-purpose NLP tool that has proven effective for many text-generation and sequence-labeling tasks. Seq2seq builds on deep neural language modeling and inherits its remarkable accuracy in estimating local, next-word distributions. In this work…
This paper introduces a novel recurrent model for music composition that is tailored to the structure of polyphonic music. We propose an efficient new conditional probabilistic factorization of musical scores, viewing a score as a collection of concurrent, coupled sequences: i.e. voices. To model the conditional distri…
A new perspective on self-attention models using MLPs.
problem Improving sequence modeling with self-attention mechanisms.
method Introducing HyperMLP and HyperGLU, which use dynamic two-layer MLPs with reverse-offset layout.
result HyperMLP/HyperGLU consistently outperform softmax-attention baselines.
E-valuator converts verifier scores into reliable decision rules.
problem Ensuring the correctness of agent trajectories based on heuristic scores.
method Sequential hypothesis testing framework for online monitoring of agent trajectories.
result E-valuator provides better false alarm rate control and statistical power than other strategies.
This paper compares log-likelihood and BLEU scores for sequence generation tasks.
problem The discrepancy between density estimation and sequence generation performance.
method Comparing several density estimators on five machine translation tasks.
result The correlation between log-likelihood and BLEU varies depending on model families.
AntBO optimizes antibody design using Bayesian optimization for efficient and effective CDRH3 sequence generation.
problem Designing optimal antigen-specific CDRH3 regions in antibody design due to combinatorial sequence space.
method Combinatorial Bayesian optimization framework with trust region for developability.
result AntBO designs CDRH3 regions with diverse biophysical properties and outperforms existing methods.
We present a new method for computing ASR word confidences that effectively mitigates the effect of ASR errors for diverse downstream applications, improves the word error rate of the 1-best result, and allows better comparison of scores across different models. We propose 1) a new method for modeling word confidence u…
In this paper, a new deep reinforcement learning based augmented general sequence tagging system is proposed. The new system contains two parts: a deep neural network (DNN) based sequence tagging model and a deep reinforcement learning (DRL) based augmented tagger. The augmented tagger helps improve system performance …
SBMs learn manifold-like structures by mixing samples with a non-conservative field.
problem How SBMs learn data distributions on low-dimensional manifolds.
method Investigating linear approximations and subspaces of local feature vectors during diffusion.
result SBMs mix samples by a non-conservative field within the manifold, maintaining manifold-like structure.
Generative models on spheres improve discrete sequence sampling.
problem Learning generative models for discrete sequences in continuous space.
method Work on sphere Sd−1, using von Mises-Fisher distribution and radial symmetry. result Improved results on Sudoku and language modeling with vMF path.
A new method simplifies sampling from complex distributions without using diffusions.
problem Sampling from complex, high-dimensional distributions efficiently.
method Reduces sampling to solving a sequence of 'nice' sampling problems using SLC distributions.
result Shows how to traverse backwards paths using high-accuracy routines for SLC distributions.
New method detects RNA modifications without prior training, revealing novel sites.
problem Detecting RNA modifications with high accuracy and sensitivity.
method Anomaly detection using nanopore raw ionic current signals and nearest neighbor comparison.
result Detects diverse RNA modifications without prior training, including a novel 2'-O-methylated site in DENV.
Nonparametric IPSS selects features with false discovery control.
problem Feature selection in high-dimensional data with theoretical false discovery control.
method Integrated Path Stability Selection (IPSS) applied to nonparametric feature importance scores.
result IPSS accurately controls false discovery rate and detects more true positives than existing methods.
HKT improves sequence processing with multi-scale attention and kernel analysis.
problem Processing sequences at multiple scales with efficient attention mechanisms.
method Trainable causal downsampling and convex weights for level-specific score matrices.
result HKT achieves consistent gains over standard attention across various tasks.
Emergenet predicts animal influenza strain emergence, outperforming current methods.
problem Limited ability to quantitatively assess animal influenza strain emergence.
method Infer digital twin of sequence evolution using 220,151 HA sequences.
result Emergenet predictions outperform WHO seasonal vaccine recommendations and CDC IRAT scores.
This thesis improves protein contact prediction using unsupervised and supervised methods.
problem Improving accuracy of protein contact prediction.
method Unsupervised and supervised deep learning methods.
result A scoring system called diversity score for measuring contact novelty.
Sequence generative adversarial networks (SeqGAN) have been used to improve conditional sequence generation tasks, for example, chit-chat dialogue generation. To stabilize the training of SeqGAN, Monte Carlo tree search (MCTS) or reward at every generation step (REGS) is used to evaluate the goodness of a generated sub…
Improved diffusion models using energy distillation and sequential Monte Carlo.
problem Training instability and inferior performance in energy parameterized diffusion models.
method Introduced a novel training regime for energy functions through distillation of pre-trained diffusion models, and cast the sampling procedure as a Feynman Kac model.
result Demonstrated improved performance and new sampling techniques.
One problem in the application of reinforcement learning to real-world problems is the curse of dimensionality on the action space. Macro actions, a sequence of primitive actions, have been studied to diminish the dimensionality of the action space with regard to the time axis. However, previous studies relied on human…
A method for finding most influential sets reduces a complex problem to a sequence of simpler top-k problems.
problem Identifying most influential subsets in complex models.
method Reduces the problem to a sequence of top-k problems using Dinkelbach's method. result The method returns a globally optimal set for the univariate ratio objective, including partial linear models.
Multimodal machine learning is a core research area spanning the language, visual and acoustic modalities. The central challenge in multimodal learning involves learning representations that can process and relate information from multiple modalities. In this paper, we propose two methods for unsupervised learning of j…
Modern navigation services often provide multiple paths connecting the same source and destination for users to select. Hence, ranking such paths becomes increasingly important, which directly affects the service quality. We present PathRank, a data-driven framework for ranking paths based on historical trajectories us…
The well-known Gumbel-Max trick for sampling from a categorical distribution can be extended to sample k elements without replacement. We show how to implicitly apply this 'Gumbel-Top-k' trick on a factorized distribution over sequences, allowing to draw exact samples without replacement using a Stochastic Beam Sea…
Transformer improves sequence generation with insertion and deletion phases.
problem Sequence generation challenges in machine translation.
method Insertion-Deletion Transformer with iterative insertion and deletion phases.
result Significant BLEU score improvement over insertion-only models.
Annealed Langevin dynamics improves sampling from composite scores in SBI.
problem Irreducible bias in sampling from composite scores of SBI methods.
method Derive Wasserstein bounds and decision rules for hyperparameters.
result Explicit decision rules for hyperparameters guarantee prescribed sampling accuracy.
Generative networks minimize predictive scoring rules for probabilistic forecasting.
problem Evaluating and improving probabilistic forecasts using generative models.
method Training generative networks to minimize predictive-sequential scoring rules on temporal sequences.
result Our method outperforms adversarial approaches in probabilistic calibration.
Predictions are issued on the basis of certain information. If the forecasting mechanisms are correctly specified, a larger amount of available information should lead to better forecasts. For point forecasts, we show how the effect of increasing the information set can be quantified by using strictly consistent scorin…
Grammatical error correction, like other machine learning tasks, greatly benefits from large quantities of high quality training data, which is typically expensive to produce. While writing a program to automatically generate realistic grammatical errors would be difficult, one could learn the distribution of naturally…
BERT embeddings improve sequence quality metrics.
problem Measuring the quality of generated sequences against references.
method Employ contextual BERT embeddings for sequence-level reward.
result Contextual embeddings provide a more effective learning signal.