Improved speech recognition model with better performance.
problem Speech recognition accuracy on Librispeech.
method Integrates an external language model with an internal LM correction.
result Over 14% relative improvement in performance.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “probability transducer”
A new method calculates optimal decisions from classifier outputs, improving predictions in drug discovery.
problem Finding optimal decisions from classifier outputs in fields like medicine.
method Develops a transducer that calculates probabilities from classifier outputs, enabling expected-utility maximization.
result Improves prediction accuracy in drug discovery problems, sometimes close to theoretical maximum.
Improved neural transducer model outperforms attention model on longer sequences.
problem Improving performance of neural transducer models.
method Comparison of training criteria (marginalization vs. maximum approximation), model generalization, and output label topology.
result Final transducer model outperforms attention model by over 6% relative WER on Switchboard 300h.
Conditional random fields (CRFs) have been shown to be one of the most successful approaches to sequence labeling. Various linear-chain neural CRFs (NCRFs) are developed to implement the non-linear node potentials in CRFs, but still keeping the linear-chain hidden structure. In this paper, we propose NCRF transducers, …
Conformal Prediction Regions match Imprecise Highest Density Regions under consonance.
problem Matching conformal prediction regions with highest density regions.
method Using consonance and the Imprecise Probability theory of clouds.
result Imprecise Highest Density Regions are equivalent to Conformal Prediction Regions under consonance.
Framework for differentiating WFSTs for structured loss functions.
problem Training dynamic structured loss functions in neural networks.
method Automatic differentiation with WFSTs, combining pruning and back-off.
result Demonstrated learning over WFST latent phrase decomposition.
Having a sequence-to-sequence model which can operate in an online fashion is important for streaming applications such as Voice Search. Neural transducer is a streaming sequence-to-sequence model, but has shown a significant degradation in performance compared to non-streaming models such as Listen, Attend and Spell (…
Develops STC for sequential data with missing labels.
problem Learning from partially labeled and unsegmented sequential data.
method Introduces Star Temporal Classification (STC) using a star token and GTN framework.
result Recover most of supervised baseline performance with up to 70% missing labels.
Streamable model improves speech recognition performance.
problem Improving speech recognition in long-form audio.
method Chunk-based attention model for streaming speech recognition.
result Streamable model maintains competitive performance and generalizes well to long-form speech.
New methods improve integration of external LMs with AED models.
problem Improving performance of AED models by integrating external LMs.
method Comparing and proposing novel methods to estimate implicit LM from AED models.
result Proposed methods outperform previous approaches.
A new framework improves ASR alignment accuracy via optimal transport.
problem Peaky behavior and alignment inaccuracies in E2E ASR models.
method Differentiable alignment framework based on 1D optimal transport.
result Significant improvement in ASR alignment performance compared to CTC.
A fixed point theorem is proved for inverse transducers, leading to an automata-theoretic proof of the fixed point subgroup of an endomorphism of a finitely generated virtually free group being finitely generated. If the endomorphism is uniformly continuous for the hyperbolic metric, it is proved that the set of regula…
We stabilize the activations of Recurrent Neural Networks (RNNs) by penalizing the squared distance between successive hidden states' norms. This penalty term is an effective regularizer for RNNs including LSTMs and IRNNs, improving performance on character-level language modeling and phoneme recognition, and outperfor…
Acoustic Neighbor Embeddingseess.AS
Acoustic Neighbor Embeddings map speech and text to fixed dimensions for phonetic confusability.
problem Mapping speech and text to fixed dimensions for phonetic confusability.
method Adapting SNE to sequential inputs, training two encoder neural networks.
result More accurate results with low-dimensional embeddings in word recognition tasks.
This paper formalizes Uniswap v3 using PTA and FST for rigorous analysis.
problem Formal modeling of Uniswap v3's concentrated liquidity for rigorous analysis.
method Formal state machine models using PTA and FST, proving rounding bounds.
result Formal justification of Uniswap v3's -slack and rounding safety.
Most of the parameters in large vocabulary models are used in embedding layer to map categorical features to vectors and in softmax layer for classification weights. This is a bottle-neck in memory constraint on-device training applications like federated learning and on-device inference applications like automatic spe…
A keyword spotting (KWS) system determines the existence of, usually predefined, keyword in a continuous speech stream. This paper presents a query-by-example on-device KWS system which is user-specific. The proposed system consists of two main steps: query enrollment and testing. In query enrollment step, phonetic pos…
Requirements elicitation can be very challenging in projects that require deep domain knowledge about the system at hand. As analysts have the full control over the elicitation process, their lack of knowledge about the system under study inhibits them from asking related questions and reduces the accuracy of requireme…
We propose a method for modeling and learning turn-taking behaviors for accessing a shared resource. We model the individual behavior for each agent in an interaction and then use a multi-agent fusion model to generate a summary over the expected actions of the group to render the model independent of the number of age…
New ASR system handles multiple languages without needing language-specific encoding.
problem Joint training of data-rich and data-scarce languages in a single model.
method Transforms all languages to a single writing system through transliteration, separating modeling and rendering.
result Language-agnostic multilingual ASR system reduces WER up to 10% over language-dependent models.
This study improves knowledge distillation for RNN-T models with noisy labels.
problem Challenges in distilling knowledge from RNN-T models with variable quality teachers.
method Full-sum distillation and sequence-level knowledge distillation.
result Full-sum distillation outperforms other methods for RNN-T models, especially for bad teachers.
Phylogenetic tree reconstruction is traditionally based on multiple sequence alignments (MSAs) and heavily depends on the validity of this information bottleneck. With increasing sequence divergence, the quality of MSAs decays quickly. Alignment-free methods, on the other hand, are based on abstract string comparisons …
Requirements elicitation requires extensive knowledge and deep understanding of the problem domain where the final system will be situated. However, in many software development projects, analysts are required to elicit the requirements from an unfamiliar domain, which often causes communication barriers between analys…
New concept of attitude towards probability introduced in risk sharing problems.
problem Risk sharing problems and attitudes towards probability.
method Generalized definition of probability premium, local approximation, rank-dependent utility model, dual theory.
result Attitude towards probability can be first-order or second-order, depending on the model.
Non-trivialization probability of arc system in 3D space
problem Defining and generalizing the knotting probability of an arc diagram in 3D space
method Transforming polygonal arcs in 3D space into unique arc diagrams
result Introducing and generalizing the Non-Trivialization probability (NT probability) for arc systems in 3D space
There are many advantages to use probability method for nonlinear system identification, such as the noises and outliers in the data set do not affect the probability models significantly; the input features can be extracted in probability forms. The biggest obstacle of the probability model is the probability distribu…
Identifies conditions for multiple invariant probabilities in Markov kernels.
problem Global irreducibility and recurrence do not guarantee uniqueness of invariant probabilities.
method Uses Jordan decomposition of the difference of two invariant probabilities.
result A Markov kernel has more than one invariant probability if and only if it admits a visible absorbing decomposition.
NT probability measures knotting in 3D arc systems.
problem Measuring knotting in 3D arc systems.
method Transforming polygonal arcs into unique diagrams, generalizing NT probability.
result Properties of NT probability for 3D arc systems are shown.
We give an overview of two approaches to probability theory where lower and upper probabilities, rather than probabilities, are used: Walley's behavioural theory of imprecise probabilities, and Shafer and Vovk's game-theoretic account of probability. We show that the two theories are more closely related than would be …
This work improves deep neural network probability estimation methods.
problem Estimating probabilities from high-dimensional data with inherent uncertainty.
method Investigates and compares methods for probability estimation using deep neural networks, proposing a new method that promotes consistent probabilities.
result The new method outperforms existing approaches on most metrics on simulated and real-world data.
Dually flat structure with escort probability and its application to alpha-Voronoi diagramscond-mat.stat-mech
This paper studies geometrical structure of the manifold of escort probability distributions and shows its new applicability to information science. In order to realize escort probabilities we use a conformal transformation that flattens so-called alpha-geometry of the space of discrete probability distributions, which…
A new method for adapting to label shifts using class probability matching.
problem Adapting to label shifts where class probabilities differ between source and target domains.
method Class Probability Matching using Kernel Methods (CPMKM) framework.
result CPMKM outperforms existing methods on real datasets.
Study classifies submanifolds in probability simplex.
problem Classifying submanifolds in the probability simplex.
method Complete classification through geometric analysis.
result Doubly totally-umbilical submanifolds identified and classified.
Study generalizes property elicitation to imprecise probabilities.
problem Minimizing risk over imprecise probability distributions.
method Maximin risk minimization over a set of imprecise probabilities.
result Conditions for elicitability of IP-properties.
Study on the probability of immunity and its bounds.
problem Estimating the probability of immunity and its bounds.
method Derive necessary and sufficient conditions for non-immunity and ε-bounded immunity; introduce indirect immunity; propose sensitivity analysis.
result Estimate the probability of benefit and produce tighter bounds of the probability of benefit.
This work presents a new classifier that is specifically designed to be fully interpretable. This technique determines the probability of a class outcome, based directly on probability assignments measured from the training data. The accuracy of the predicted probability can be improved by measuring more probability es…
The paper addresses probability calibration for incomplete sequences.
problem Improving probability estimates from incomplete sequences.
method Adapting traditional calibration techniques to sequences of varying lengths.
result Proposed methods improve probability calibration for modern sequential models.
Investigates statistical properties of perturb-softmax and perturb-argmax distributions.
problem Underexplored statistical properties of Gumbel-Softmax and Gumbel-Argmax distributions.
method Investigates convexity and differentiability to determine completeness and minimality of these distributions.
result Identifies parameters that admit complete and minimal representation of probability distributions.
Categorical d-separation criterion simplifies probability graph analysis.
problem Detecting causal relationships in probability distributions.
method Introducing categorical definitions for causal models and d-separation.
result Abstract version of d-separation criterion applies to various probability theories.
Paper constructs unfaithful probability distributions in binary causal graphs.
problem Unfaithful probability distributions in binary causal graphs.
method Constructs unfaithful probability distributions in binary causal graphs.
result Examples of unfaithful probability distributions in binary causal graphs.
Study improves estimation of rare language model outputs.
problem Estimating probabilities of rare outputs in language models.
method Importance sampling vs. activation extrapolation for low probability estimation.
result Importance sampling outperforms activation extrapolation.
The study examines Fisher-Riemann geodesics for nonparametric probability densities.
problem Understanding nonparametric probability densities using Fisher-Riemann geometry.
method Obtaining Fisher-Riemann geodesics as a limit of parametric cases with increasing parameters.
result The weak limit approach for nonparametric probability densities.
AI can learn true probabilities if data and assumptions align.
problem Understanding when AI models can accurately represent true objective probabilities.
method Proved conditions under which AI can learn true probabilities.
result Conditions for learning true probabilities are identified.
Quantum-inspired probability metrics define a complete, universal space for statistical learningstat.ML
Quantum probability metrics improve distribution comparison in high dimensions.
problem Challenges in comparing probability distributions, especially in high-dimensional and non-compact domains.
method Quantum probability metrics (QPMs) derived from quantum state spaces, overcoming limitations of MMD.
result QPMs offer enhanced sensitivity to subtle distributional differences in high dimensions and improve performance in generative modeling.
A method for diffusion on probability simplex for generative models.
problem Tension between continuous and discrete data in diffusion models.
method Proposes using softmax function applied to Ornstein-Uhlenbeck Process on probability simplex.
result Method extends to bounded image generation.
Bayesian approach approximates probability functions of Gaussian mixtures.
problem Approximating probability functions of non-spherical Gaussian mixtures.
method Bayesian decomposition, spherical radial decomposition, random sampling.
result Established differentiability and integral representation of gradient for probability functions.
We describe a Groebner basis of relations among conditional probabilities in a discrete probability space, with any set of conditioned-upon events. They may be specialized to the partially-observed random variable case, the purely conditional case, and other special cases. We also investigate the connection to generali…
We study the minimax optimal rates for estimating a range of Integral Probability Metrics (IPMs) between two unknown probability measures, based on independent samples from them. Curiously, we show that estimating the IPM itself between probability measures, is not significantly easier than estimating the probabili…