We study the relationship between catastrophic forgetting and properties of task sequences. In particular, given a sequence of tasks, we would like to understand which properties of this sequence influence the error rates of continual learning algorithms trained on the sequence. To this end, we propose a new procedure …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “Sequence Error Rate”
Sequence-to-sequence models, such as attention-based models in automatic speech recognition (ASR), are typically trained to optimize the cross-entropy criterion which corresponds to improving the log-likelihood of the data. However, system performance is usually measured in terms of word error rate (WER), not log-likel…
The paper offers error bounds for quantized dynamical models.
problem Accuracy of dynamical models from dependent data sequences.
method Developed uniform error bounds for quantized models and imperfect optimization algorithms.
result Unified bounds for slow and fast rates, scaling with model encoding bits.
This paper concerns error bounds for recursive equations subject to Markovian disturbances. Motivating examples abound within the fields of Markov chain Monte Carlo (MCMC) and Reinforcement Learning (RL), and many of these algorithms can be interpreted as special cases of stochastic approximation (SA). It is argued tha…
Study of supervised learning from multiple non-independent sequences.
problem Efficient learning from many non-independent sequences.
method Generalizes conditions for efficient learning from independent examples and single auto-correlated sequences.
result Error rate changes from to as the number of trajectories increases.
This paper presents a new optimisation approach to train Deep Neural Networks (DNNs) with discriminative sequence criteria. At each iteration, the method combines information from the Natural Gradient (NG) direction with local curvature information of the error surface that enables better paths on the parameter manifol…
Proposes selective inference for testing differences in means between clusters.
problem Inflated type I error rate when testing differences in means between clusters.
method Selective inference approach to control selective type I error rate.
result Controls selective type I error rate by accounting for data-driven cluster definition.
Biological research often involves testing a growing number of null hypotheses as new data is accumulated over time. We study the problem of online control of the familywise error rate (FWER), that is testing an apriori unbounded sequence of hypotheses (p-values) one by one over time without knowing the future, such th…
Constrained sequence codes have been widely used in modern communication and data storage systems. Sequences encoded with constrained sequence codes satisfy constraints imposed by the physical channel, hence enabling efficient and reliable transmission of coded symbols. Traditional encoding and decoding of constrained …
The dominant automatic lexical stress detection method is to split the utterance into syllable segments using phoneme sequence and their time-aligned boundaries. Then we extract features from syllable to use classification method to classify the lexical stress. However, we can't get very accurate time boundaries of eac…
Theoretical analysis shows MDMs can be efficient but not for all metrics.
problem Understanding the efficiency-accuracy trade-off of diffusion language models.
method Theoretical analysis of Masked Diffusion Model (MDM) using perplexity and sequence error rate as metrics.
result MDM achieves near-optimal perplexity but requires linear scaling for sequence error rate, highlighting efficiency-accuracy trade-offs.
RQMC improves QMC by providing practical error bounds for financial applications.
problem Lack of practical error estimates in QMC methods.
method Combines Sobol LDS with randomized scrambling methods.
result RQMC outperforms standard QMC in convergence rates and provides error bounds.
We propose sequenced-replacement sampling (SRS) for training deep neural networks. The basic idea is to assign a fixed sequence index to each sample in the dataset. Once a mini-batch is randomly drawn in each training iteration, we refill the original dataset by successively adding samples according to their sequence i…
This work bounds classification error in machine learning for low Bayes error conditions.
problem Understanding the error mismatch between Bayes error and model-based classification error.
method Applying classification error bounds to study the relationship with Kullback-Leibler divergence and proposing a linear approximation for low Bayes error conditions.
result A linear approximation of the classification error bound for low Bayes error conditions is proposed.
The recently proposed Sequence-to-Sequence (seq2seq) framework advocates replacing complex data processing pipelines, such as an entire automatic speech recognition system, with a single neural network trained in an end-to-end fashion. In this contribution, we analyse an attention-based seq2seq speech recognition syste…
A hybrid ASR system using conformer architecture improves word-error-rate and training speed.
problem Improving word-error-rate and training efficiency for hybrid ASR systems.
method Used conformer architecture, applied time downsampling, and transposed convolutions.
result Conformer-based hybrid model achieves competitive results and significantly outperforms BLSTM-based hybrid model.
We present and analyze several strategies for improving the performance of stochastic variance-reduced gradient (SVRG) methods. We first show that the convergence rate of these methods can be preserved under a decreasing sequence of errors in the control variate, and use this to derive variants of SVRG that use growing…
This study improves knowledge distillation for RNN-T models with noisy labels.
problem Challenges in distilling knowledge from RNN-T models with variable quality teachers.
method Full-sum distillation and sequence-level knowledge distillation.
result Full-sum distillation outperforms other methods for RNN-T models, especially for bad teachers.
Near-optimal tests and confidence sequences for non-parametric data.
problem Flexible statistical inference and decision-making with non-parametric data.
method Classic delayed-start normal-mixture sequential probability ratio tests with asymptotic guarantees.
result Asymptotically optimal type-I error and expected rejection time guarantees.
We propose a novel neural sequence prediction method based on \textit{error-correcting output codes} that avoids exact softmax normalization and allows for a tradeoff between speed and performance. Instead of minimizing measures between the predicted probability distribution and true distribution, we use error-correcti…
This paper investigates to what extent one can improve reinforcement learning algorithms. Our study is split in three parts. First, our analysis shows that the classical asymptotic convergence rate is pessimistic and can be replaced by with and the number…
Deep Neural Network (DNN) acoustic models often use discriminative sequence training that optimises an objective function that better approximates the word error rate (WER) than frame-based training. Sequence training is normally implemented using Stochastic Gradient Descent (SGD) or Hessian Free (HF) training. This pa…
Selective inference controls Type I error in k-means clustering tests.
problem Inflated Type I error in classical hypothesis tests for k-means clusters.
method Selective inference approach to control Type I error.
result Proposes a computable finite-sample p-value for selective inference.
A new algorithm for differential privacy in kernelized contextual bandits reduces error rate.
problem Joint differential privacy in kernelized contextual bandits.
method Proposes a novel algorithm with a specific error rate and privacy parameter dependence.
result Achieves an error rate of $\mathcal{O}\left(\sqrt{\frac{γ_T}{T}} + \frac{γ_T}{T \varepsilon}
ight)$ after queries.
EM algorithm converges linearly and achieves sharp rate in estimating mixtures of pairwise differences.
problem Estimating mixtures of pairwise differences from noisy data.
method Sharp analysis of the EM algorithm locally around the ground truth.
result The EM sequence converges linearly with an -norm guarantee on the estimation error and achieves the sharp rate of estimation in the -norm.
Paper studies CLT rates for dependent data in Wasserstein-p distance.
problem CLT rates for multivariate dependent data in Wasserstein-p distance.
method Analyzes locally dependent sequences and geometrically ergodic Markov chains.
result Establishes optimal CLT rates and () rates for dependent data.
Comparative Study of Monte Carlo and Quasi-Monte Carlo Techniques for Enhanced Derivative Pricingq-fin.PR
This study compares MC and QMC methods for derivative pricing, showing QMC's superior convergence rates.
problem Improving derivative pricing accuracy and efficiency in high-dimensional settings.
method Compared Monte Carlo and quasi-Monte Carlo techniques, focusing on convergence rates and low-discrepancy sequences.
result Quasi-Monte Carlo methods achieve superior convergence rates and reduce root mean square error in derivative pricing.
Paper analyzes convergence of two time-scale stochastic approximation using martingale approach.
problem Analyzing convergence of two time-scale stochastic approximation algorithms.
method Uses martingale approach to establish convergence conditions and rates.
result Establishes different rates of convergence for fast and slow subsystems.
Paper presents a probabilistic model to improve LLM cascade performance.
problem Complexity of LLM cascades and their interaction error rates.
method Probabilistic model for joint performance distribution of LLMs.
result Improves area under the error-cost curve by 4.3% on average for cascades with k≥3 models.
New ASR system handles multiple languages without needing language-specific encoding.
problem Joint training of data-rich and data-scarce languages in a single model.
method Transforms all languages to a single writing system through transliteration, separating modeling and rendering.
result Language-agnostic multilingual ASR system reduces WER up to 10% over language-dependent models.
We formulate statistical watermarking as hypothesis testing and establish near-optimal bounds.
problem Statistical watermarking in the context of hypothesis testing.
method Formulated as a hypothesis testing problem, using coupling of output tokens and rejection regions.
result Established nearly matching upper and lower bounds on the number of i.i.d. tokens required for small Type I and Type II errors.
Sharp convergence analysis for nonconvex regression models.
problem Nonconvex optimization in regression models with normally distributed covariates.
method Gaussian comparison theorems for analyzing iterative algorithms.
result Sharp global convergence rates for various statistical models.
Establishes a microstructural foundation for a rough log-normal volatility model.
problem Developing a robust model for financial volatility under microstructural effects.
method Introduced a sequence of order-driven financial market models with Poisson process arrivals and analyzed their convergence to a log-normal rough volatility model.
result Weak convergence of price-volatility process to a log-normal rough volatility model with established weak error rates.
Transformers can efficiently approximate nonparametric regression with minimal parameters and sequences.
problem Efficiently approximating nonparametric regression functions with transformers.
method Kernel-weighted polynomial basis and gradient descent.
result Achieves minimax optimal rate of convergence with fewer parameters and sequences.
Adaptive importance samplers are adaptive Monte Carlo algorithms to estimate expectations with respect to some target distribution which \textit{adapt} themselves to obtain better estimators over a sequence of iterations. Although it is straightforward to show that they have the same convergen…
Improved bounds for non-linear SA with fast convergence.
problem Stochastic approximation with non-linear mappings and multiple time scales.
method Mean squared error bounds with rate for contractive mappings.
result First rate for non-linear two-time-scale SA without additional smoothness assumptions.
Attention-based models have recently shown great performance on a range of tasks, such as speech recognition, machine translation, and image captioning due to their ability to summarize relevant information that expands through the entire length of an input sequence. In this paper, we analyze the usage of attention mec…
We present a factorized hierarchical variational autoencoder, which learns disentangled and interpretable representations from sequential data without supervision. Specifically, we exploit the multi-scale nature of information in sequential data by formulating it explicitly within a factorized hierarchical graphical mo…
For real-world speech recognition applications, noise robustness is still a challenge. In this work, we adopt the teacher-student (T/S) learning technique using a parallel clean and noisy corpus for improving automatic speech recognition (ASR) performance under multimedia noise. On top of that, we apply a logits select…
Improved speech recognition with language model integration in sequence-to-sequence models.
problem Improving word error rate in speech recognition models.
method Log-linear combination of acoustic and language models with per-token renormalization.
result The proposed method shows good improvements over standard model combination on Librispeech system.
By lifting the ReLU function into a higher dimensional space, we develop a smooth multi-convex formulation for training feed-forward deep neural networks (DNNs). This allows us to develop a block coordinate descent (BCD) training algorithm consisting of a sequence of numerically well-behaved convex optimizations. Using…
Post-training optimizes model performance beyond base model limits.
problem Optimizing sequence prediction models beyond the base model's support.
method Policy gradient (PG) and adaptive learning rate (LR) techniques.
result Post-training with PG can achieve near-optimal performance beyond the base model's support.
Universal algorithm learns unknown distribution for various decision-making problems.
problem Various statistical measures in contextual sequential decision-making.
method Infinite-dimensional functional regression oracle for cumulative distribution functions.
result Utility regret rate bounded by polynomial decay of eigenvalue sequence.
Bayesian sequence prediction is a simple technique for predicting future symbols sampled from an unknown measure on infinite sequences over a countable alphabet. While strong bounds on the expected cumulative error are known, there are only limited results on the distribution of this error. We prove tight high-probabil…
GAAVI offers anytime-valid tests for CMF global null and contrasts.
problem Inference on the conditional mean function for high confidence decisions.
method Asymptotic anytime-valid tests for CMF global null and contrasts.
result Achieves asymptotic type-I error guarantees, power one, and optimal sample complexity.
Model proposes neural network for continuous time dynamics with inductive biases.
problem Training neural networks for small datasets with nonlinear dynamics.
method Inductive biases on decay rates and frequencies using Koopman operator theory.
result Higher forecasting performance with single short training sequence.
Study finds polynomial convergence rate for Farey sequences linked to Riemann hypothesis.
problem Understanding convergence rates of maximum mean discrepancies for Farey sequences.
method Identifying positive-semidefinite kernels and their polynomial convergence rates.
result Polynomial convergence rate of maximum mean discrepancies of Farey sequences is equivalent to the Riemann hypothesis.
We consider the problem of learning high-dimensional Gaussian graphical models. The graphical lasso is one of the most popular methods for estimating Gaussian graphical models. However, it does not achieve the oracle rate of convergence. In this paper, we propose the graphical nonconvex optimization for optimal estimat…