Improved speech recognition with end-to-end attention models.
problem End-to-end speech recognition with open-vocabulary.
method Sequence-to-sequence attention-based models on subword units, new pretraining scheme, CTC loss function, LSTM language models.
result State-of-the-art word error rates (3.54% and 3.82%) on LibriSpeech test-clean.
New combinatorial framework for geometric realizations of subword complexes.
problem Proving or disproving geometric realizations of subword complexes of Coxeter groups.
method Algebraic combinatorics and discrete geometry framework, parameter matrices.
result Existence of parameter matrices equivalent to realizability of subword complexes as chirotopes.
New techniques reuse subword embeddings in neural models, reducing size and improving performance.
problem Improving performance and reducing model size in subword-aware neural language models.
method Reusing subword embeddings and other weights in multi-layer input embedding models, tying layers consecutively bottom-up.
result Best morpheme-aware model with reused weights outperforms competitive word-level model by a large margin.
Estimator Vectors learns OOV word embeddings using subword and context clues.
problem Lack of OOV word representations in neural network models.
method Jointly learns word, subword, and context clue representations.
result Strong estimates for OOV words via combined subword and context clue embeddings.
Improved Polish language model using subword tokenization.
problem Adapting ULMFiT for high inflection languages like Polish.
method Subword tokenization to adapt ULMFiT for Polish.
result First place in PolEval'18, 35% improvement over second best.
Byte-level machine translation outperforms embedding-based methods.
problem Improving machine translation without embedding layers.
method Replacing embedding layers with one-hot representations of bytes, and using decoder-input dropout.
result Byte-to-byte machine translation achieves BLEU scores comparable to character-level and subword-level models.
The study proves a theorem about subword complexity for free group automorphisms.
problem Analyzing subword complexity for attracting fixed points of automorphisms of free groups.
method Combinatorial arguments and train tracks.
result Subword complexity of attracting fixed points is equivalent to n, n log log n, n log n, or n^2.
The paper studies fibers of maps in totally nonnegative spaces.
problem Understanding the structure of fibers of Chevalley exponentiation maps.
method Cell stratifications, face posets, and homeomorphic CW complexes.
result Fibers are homeomorphic to interior dual block complexes of subword complexes.
A model corrects Lithuanian grammatical errors.
problem Lack of language skills and typing errors in Lithuanian.
method Transformer architectures for subword and byte-level approaches.
result F0.5=0.92 for Lithuanian grammatical error correction. Study on representations of four-punctured sphere group in hyperbolic spaces.
problem Understanding representations of the four-punctured sphere group.
method Investigation into simple-stable and Bowditch representations in Gromov-hyperbolic spaces.
result Simple-stable representations and Bowditch representations are equivalent.
AV-ASR system improves speech recognition with visual context.
problem Improving speech recognition accuracy with visual information.
method Transformer-based architecture with multiresolution and multimodal training.
result Multiresolution training speeds up convergence and improves WER by 18%.
Neural model improves text normalization for non-English languages.
problem Improving text normalization in non-English languages with limited data.
method Sequence-to-sequence model with character and word embeddings, using pre-trained word embeddings with subword information.
result Achieved state-of-the-art F1 score on Arabic language correction dataset.
Proposes MorphMine for unsupervised morpheme segmentation to improve word embeddings.
problem Lack of semantic information in word-level analysis for infrequent and out-of-vocabulary words.
method MorphMine applies a parsimony criterion to hierarchically segment words into the fewest number of morphemes.
result MorphMine segments words into human-verified morphemes and improves word embedding quality.
Study geodesics in hyperbolic surfaces and trees, proving edge length properties.
problem Investigate intersections and geodesic lines in hyperbolic surfaces and trees.
method Construct and analyze triangles formed by geodesic lines and conjugates of words.
result Edges in certain triangles can be longer than geodesics, depending on word structure.
Acoustic Neighbor Embeddings map speech and text to fixed dimensions for phonetic confusability.
problem Mapping speech and text to fixed dimensions for phonetic confusability.
method Adapting SNE to sequential inputs, training two encoder neural networks.
result More accurate results with low-dimensional embeddings in word recognition tasks.
Synthetic noise training improves machine translation robustness to spelling mistakes.
problem Making machine translation robust to spelling mistakes and natural noise.
method Training on synthetic noise to improve robustness to natural noise.
result Training on synthetic noise improves robustness to natural noise without diminishing performance on clean text.
Paper reduces vocabulary losslessly for language model cooperation.
problem Language models struggle to cooperate with different tokenizations.
method Established a theoretical framework for lossless vocabulary reduction.
result Efficiently converts models with different tokenizations to cooperate with maximal common vocabulary.
Probabilistic FastText captures multiple word senses and sub-word structures.
problem Capturing multiple word senses and sub-word structures in word embeddings.
method Probabilistic FastText uses Gaussian mixture densities to represent words, sharing statistical strength across sub-word structures and capturing different word senses.
result Probabilistic FastText outperforms existing models on word-similarity benchmarks and discerning different meanings.
SAFER method certifies robustness to word substitutions without model structure.
problem Certified robustness against synonymous word substitutions in NLP models.
method Randomized smoothing with stochastic ensemble of randomized inputs.
result Significantly outperforms state-of-the-art methods for certified robustness.
The study proves biharmonic unit sections on 2-tori are always harmonic and exists in each homotopy class.
problem Characterizing biharmonic unit vector fields and sections on 2-tori.
method Analyzing variational problems for unit vector fields under conformal metrics, proving properties through homotopy classes.
result Biharmonic unit sections on 2-tori are always harmonic and exist in each homotopy class.
Characterizes magnetic unit vector fields on Lie groups.
problem Classifying magnetic unit vector fields on Lie groups.
method Characterization through critical points of Landau Hall and Dirichlet energy functionals.
result Classification of all magnetic left invariant unit vector fields on 3-dimensional Lie groups.
In this paper we propose and investigate a novel nonlinear unit, called Lp unit, for deep neural networks. The proposed Lp unit receives signals from several projections of a subset of units in the layer below and computes a normalized Lp norm. We notice two interesting interpretations of the Lp unit. First…
Paper proves extension of unit normal vector field from a hypersurface.
problem Need to extend unit normal vector field from a hypersurface.
method Provides an elementary proof of existence and uniqueness of such an extension.
result Elementary proof of existence and uniqueness of unit gradient field extension.
Randomly chosen primary hidden units and derived secondary units reduce neural network complexity.
problem Large number of hidden units in neural networks.
method Introducing primary and secondary hidden units with random weights for primary units and derived weights for secondary units.
result Significant reduction in the number of hidden units without compromising accuracy.
Study examines how business units can benefit from group cohesion under regulatory constraints.
problem Regulatory constraints limit business units' ability to form a single cohesive group.
method Defined and analyzed cohesive risk measures to minimize capital costs.
result Cohesive risk measures allow groups to achieve minimal capital costs without altering individual liabilities.
Study examines dependence properties of Bayesian neural network units in finite-width networks.
problem Understanding dependence properties of hidden units in practical finite-width Bayesian neural networks.
method Theoretical analysis and empirical evaluation of depth and width impacts.
result Hidden units in finite-width Bayesian neural networks are dependent, contrary to the infinite-width limit assumption.
Smooth groupoid algebras are H-unital, with implications for algebraic and homological properties.
problem Understanding the structure of convolution algebras on Lie groupoids.
method Analyzing smooth functions and invariant subsets to prove H-unitality.
result H-unitality of groupoid algebras and their quotients, leading to excision properties.
We study deep Bayesian neural networks with Gaussian priors, revealing heavy-tailed unit activations.
problem Characterizing regularization effects in deep Bayesian neural networks.
method Investigation of deep Bayesian neural networks with Gaussian weight priors and ReLU-like nonlinearities.
result The prior distribution on units becomes increasingly heavy-tailed with depth, influencing activation patterns.
WEST compresses word embeddings and softmax layers for memory efficiency.
problem Memory constraints in large vocabulary models.
method WEST encodes words with sequences of sub-units, improving compression without performance loss.
result WEST achieves significant compression without sacrificing performance.
Wasserstein t-SNE embeds hierarchical datasets considering within-unit distributions.
problem Exploring hierarchical datasets where units are compared based on means of sample distributions.
method Uses Wasserstein distance metric for 2D embeddings of units, approximating Gaussian distributions for efficiency.
result Demonstrates effective embedding of hierarchical datasets, uncovering meaningful structure.
Study on hidden units in finite Bayesian neural networks and their tail properties.
problem Understanding the behavior of hidden units in finite Bayesian neural networks.
method Introduced a generalized Weibull-tail property to describe hidden units tails.
result Unit priors become heavier-tailed going deeper, providing insights into finite Bayesian neural networks.
New characterization of Calabi torus in unit sphere found.
problem Rigidity of closed minimally immersed Legendrian submanifolds in unit sphere.
method Maximum principle and Simons' type integral inequality.
result New characterization of Calabi torus in unit sphere.
Study on fractional curvature flow on unit sphere, extending previous work.
problem Fractional Nirenberg problem on unit sphere.
method Fractional conformal curvature flow on unit sphere.
result Perturbation result for fractional Nirenberg problem with σ∈(1/2,1). We present a new equation with respect to a unit vector field on Riemannian manifold Mn such that its solution defines a totally geodesic submanifold in the unit tangent bundle with Sasaki metric and apply it to some classes of unit vector fields. We introduce a class of covariantly normal unit vector fields and pro…
Bayesian units improve speech recognition with minimal parameters.
problem Improving speech recognition models with fewer parameters.
method Derived Bayesian recurrent units integrated into deep learning frameworks.
result Adding Bayesian units improves speech recognition performance.
Let Σbe a k-dimensional minimal surface in the unit ball B^n which meets the unit sphere orthogonally. We show that the area of Σis bounded from below by the volume of the unit ball in R^k. This answers a question posed by R. Schoen.
New neural network units resist adversarial attacks effectively.
problem Adversarial attacks on neural networks that misclassify inputs.
method Introduced RBFI units with non-linear structure.
result RBFI units maintain high accuracy in adversarial attacks.
Minimal vector fields on oscillator groups studied, with specific conditions for minimality.
problem Characterizing minimal left-invariant unit vector fields on oscillator groups.
method Analyzing structure constants and harmonic maps into the unit tangent bundle.
result Minimal vector fields defined by specific conditions on oscillator groups.
Neural Power Unit (NPU) learns arbitrary power functions on real numbers.
problem Neural Networks struggle with generalizing beyond seen data and arithmetic operations.
method Introduces Neural Power Unit (NPU) that operates on real numbers and learns arbitrary power functions.
result NPU outperforms competitors in accuracy and sparsity on arithmetic datasets and discovers governing equations from data.
Study calculates first p-widths of unit disk.
problem Computing first p-widths of the unit disk. method Regularity result for integral 1-varifolds on compact 2-manifolds with convex boundary, applied to unit disk.
result Computed first p-widths for p=1,...,4. ReLU units can 'die' in neural networks, causing slower convergence.
problem ReLU units sometimes produce near-zero outputs during training.
method Simulation and statistical analysis of a simplified ReLU unit model.
result Activation probability decreases as training progresses, leading to slower convergence.
In a seminal paper Abadie, Diamond, and Hainmueller [2010] (ADH), see also Abadie and Gardeazabal [2003], Abadie et al. [2014], develop the synthetic control procedure for estimating the effect of a treatment, in the presence of a single treated unit and a number of control units, with pre-treatment outcomes observed f…
Bayesian SHMM discovers acoustic units from unlabeled speech.
problem Discovering language-specific acoustic units from unlabeled speech.
method Bayesian Subspace Hidden Markov Model (SHMM) trained on labeled data to find new acoustic units on target language.
result Significantly outperforms previous HMM-based systems and compares favorably with Variational Auto Encoder-HMM.
We construct homotopically non-trivial maps from the unit m-sphere to the unit (m-1)-sphere with arbitrarily small k-dilation for each k greater than (m + 1)/2. We prove that homotopically non-trivial maps from the unit m-sphere to the unit (m-1)-sphere cannot have arbitrarily small k-dilation for k less than or equal …
Can certain shapes be drawn with a pencil and eraser?
problem Characterizing which planar sets can be drawn with a pencil and eraser.
method Analyzes the properties of sets drawable with a pencil and eraser, using open and closed unit disks.
result Drawability cannot be characterized by local obstructions.
We present a probabilistic variant of the recently introduced maxout unit. The success of deep neural networks utilizing maxout can partly be attributed to favorable performance under dropout, when compared to rectified linear units. It however also depends on the fact that each maxout unit performs a pooling operation…
The paper classifies actions on unit spheres and symmetric spaces.
problem Classifying isometric cohomogeneity one actions on spheres and symmetric spaces.
method Complete classification using cohomogeneity one theory.
result All isometric cohomogeneity one actions on unit spheres and symmetric spaces are classified.
We study unit horizontal bundles associated with Riemannian submersions. First we investigate metric properties of an arbitrary unit horizontal bundle equipped with a Riemannian metric of the Cheeger-Gromoll type. Next we examine it from the Gromov-Hausdorff convergence theory point of view, and we state a collapse the…