arXiv research

Improved neural transducer model outperforms attention model on longer sequences.

Conditional random fields (CRFs) have been shown to be one of the most successful approaches to sequence labeling. Various linear-chain neural CRFs (NCRFs) are developed to implement the non-linear node potentials in CRFs, but still keeping the linear-chain hidden structure. In this paper, we propose NCRF transducers, …

Framework for differentiating WFSTs for structured loss functions.

Improved speech recognition model with better performance.

A new method calculates optimal decisions from classifier outputs, improving predictions in drug discovery.

Having a sequence-to-sequence model which can operate in an online fashion is important for streaming applications such as Voice Search. Neural transducer is a streaming sequence-to-sequence model, but has shown a significant degradation in performance compared to non-streaming models such as Listen, Attend and Spell (…

Develops STC for sequential data with missing labels.

Streamable model improves speech recognition performance.

We stabilize the activations of Recurrent Neural Networks (RNNs) by penalizing the squared distance between successive hidden states' norms. This penalty term is an effective regularizer for RNNs including LSTMs and IRNNs, improving performance on character-level language modeling and phoneme recognition, and outperfor…

New methods improve integration of external LMs with AED models.

A new framework improves ASR alignment accuracy via optimal transport.

A fixed point theorem is proved for inverse transducers, leading to an automata-theoretic proof of the fixed point subgroup of an endomorphism of a finitely generated virtually free group being finitely generated. If the endomorphism is uniformly continuous for the hyperbolic metric, it is proved that the set of regula…

Acoustic Neighbor Embeddings map speech and text to fixed dimensions for phonetic confusability.

Conformal Prediction Regions match Imprecise Highest Density Regions under consonance.

This study improves knowledge distillation for RNN-T models with noisy labels.

This paper formalizes Uniswap v3 using PTA and FST for rigorous analysis.

Most of the parameters in large vocabulary models are used in embedding layer to map categorical features to vectors and in softmax layer for classification weights. This is a bottle-neck in memory constraint on-device training applications like federated learning and on-device inference applications like automatic spe…

A keyword spotting (KWS) system determines the existence of, usually predefined, keyword in a continuous speech stream. This paper presents a query-by-example on-device KWS system which is user-specific. The proposed system consists of two main steps: query enrollment and testing. In query enrollment step, phonetic pos…

Requirements elicitation can be very challenging in projects that require deep domain knowledge about the system at hand. As analysts have the full control over the elicitation process, their lack of knowledge about the system under study inhibits them from asking related questions and reduces the accuracy of requireme…

We propose a method for modeling and learning turn-taking behaviors for accessing a shared resource. We model the individual behavior for each agent in an interaction and then use a multi-agent fusion model to generate a summary over the expected actions of the group to render the model independent of the number of age…

New ASR system handles multiple languages without needing language-specific encoding.

Phylogenetic tree reconstruction is traditionally based on multiple sequence alignments (MSAs) and heavily depends on the validity of this information bottleneck. With increasing sequence divergence, the quality of MSAs decays quickly. Alignment-free methods, on the other hand, are based on abstract string comparisons …

Requirements elicitation requires extensive knowledge and deep understanding of the problem domain where the final system will be situated. However, in many software development projects, analysts are required to elicit the requirements from an unfamiliar domain, which often causes communication barriers between analys…

Graphs of neural networks are represented to preserve symmetry, improving performance across various tasks.

Convolutional Neural Processes improve data efficiency in neural processes.

Neural Ordinary Differential Equation (Neural ODE) has been proposed as a continuous approximation to the ResNet architecture. Some commonly used regularization mechanisms in discrete neural networks (e.g. dropout, Gaussian noise) are missing in current Neural ODE networks. In this paper, we propose a new continuous ne…

Investigates how neural network graph structure impacts predictive performance.

Neural Tangents simplifies infinite-width neural networks for research.

Novel framework explains generalization in deep neural networks.

Neural networks can approximate functions uniformly across various measures.

Investigates neural codes and their embeddings, proving conjectures and introducing new code types.

Neural dynamical systems are dynamical systems that are described at least in part by neural networks. The class of continuous-time neural dynamical systems must, however, be numerically integrated for simulation and learning. Here, we present a compact neural circuit for two common numerical integrators: the explicit …

Quadratic models explain neural network behavior during training.

Graph Metanetworks process diverse neural architectures efficiently.

Optimal rates for shallow ReLU networks in nonparametric regression.

New metric compares noisy neural trajectories using optimal transport.

Paper analyzes Q-learning with neural networks, proving a fast convergence rate.

The neural tangent kernel equivalence theorem fails in practice.

Equivariant neural networks use symmetry to interpret complex data.

Two new criteria help understand the advantage of deep neural networks.

Use simplified layerwise linear models to understand neural dynamics.

Analysis of over-parameterized neural networks has drawn significant attention in recentyears. It was shown that such systems behave like convex systems under various restrictedsettings, such as for two-level neural networks, and when learning is only restricted locally inthe so-called neural tangent kernel space aroun…

The paper proves consistency of neural networks with regularization.

This study shows why training Neural ODEs is hard and proposes a new method.

We introduce Graph Neural Processes (GNP), inspired by the recent work in conditional and latent neural processes. A Graph Neural Process is defined as a Conditional Neural Process that operates on arbitrary graph data. It takes features of sparsely observed context points as input, and outputs a distribution over targ…

Paper benchmarks quantum neural networks against classical ones for binary classification tasks.

Affine spiking neural networks learn efficiently and generalize well.

Explains equivariant neural networks for machine learning.