A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
New method flattens decision boundary by targeting shortcut-aligned axes in disentangled latent space.
Study on shortcuts in deep networks, revealing their layer-wise distribution and impact.
Default-ERM shortcut learning persists even without additional information.
In established network architectures, shortcut connections are often used to take the outputs of earlier layers as additional inputs to later layers. Despite the extraordinary effectiveness of shortcuts, there remain open questions on the mechanism and characteristics. For example, why are shortcuts powerful? Why do sh…
The Residual Network (ResNet), proposed in He et al. (2015), utilized shortcut connections to significantly reduce the difficulty of training, which resulted in great performance boosts in terms of both training and generalization error. It was empirically observed in He et al. (2015) that stacking more layers of resid…
Regularization methods can overregulate, suppressing causal features.
This work identifies and mitigates reasoning shortcuts in Neuro-Symbolic models.
Due to the success of residual networks (resnets) and related architectures, shortcut connections have quickly become standard tools for building convolutional neural networks. The explanations in the literature for the apparent effectiveness of shortcuts are varied and often contradictory. We hypothesize that shortcut…
Matrix Product States (MPS), also known as Tensor Train (TT) decomposition in mathematics, has been proposed originally for describing an (especially one-dimensional) quantum system, and recently has found applications in various applications such as compressing high-dimensional data, supervised kernel linear classifie…
Transformers learn to use induction heads or shortcuts based on data diversity.
Transformers simulate finite-state automata with fewer layers.
Attention weights may not accurately highlight important parts due to combinatorial shortcuts.
Study reveals DNNs prefer easy-to-learn cues over essential ones in image recognition.
New method trains deep vanilla networks as fast as ResNets without shortcut connections.
We propose a neural architecture search (NAS) algorithm, Petridish, to iteratively add shortcut connections to existing network layers. The added shortcut connections effectively perform gradient boosting on the augmented layers. The proposed algorithm is motivated by the feature selection algorithm forward stage-wise …
Neurosymbolic predictors fail to model uncertainty under independence assumption.
Sleep apnea is a common respiratory disorder characterized by breathing pauses during the night. Consequences of untreated sleep apnea can be severe. Still, many people remain undiagnosed due to shortages of hospital beds and trained sleep technicians. To assist in the diagnosis process, automated detection methods are…
One pixel modification can make deep models unlearnable.
Residual Network (ResNet) is undoubtedly a milestone in deep learning. ResNet is equipped with shortcut connections between layers, and exhibits efficient training using simple first order algorithms. Despite of the great empirical success, the reason behind is far from being well understood. In this paper, we study a …
Very deep CNNs achieve state-of-the-art results in both computer vision and speech recognition, but are difficult to train. The most popular way to train very deep CNNs is to use shortcut connections (SC) together with batch normalization (BN). Inspired by Self- Normalizing Neural Networks, we propose the self-normaliz…
A network supporting deep unsupervised learning is presented. The network is an autoencoder with lateral shortcut connections from the encoder to decoder at each level of the hierarchy. The lateral shortcut connections allow the higher levels of the hierarchy to focus on abstract invariant features. While standard auto…
To have a superior generalization, a deep learning neural network often involves a large size of training sample. With increase of hidden layers in order to increase learning ability, neural network has potential degradation in accuracy. Both could seriously limit applicability of deep learning in some domains particul…
We prove that when n >= 5, the Dehn function of SL(n;Z) is quadratic. The proof involves decomposing a disc in SL(n;R)/SO(n) into triangles of varying sizes. By mapping these triangles into SL(n;Z) and replacing large elementary matrices by "shortcuts," we obtain words of a particular form, and we use combinatorial tec…
Approximate Incremental Value-at-Risk formulae provide an easy-to-use preliminary guideline for risk allocation. Both the cases of risk adding and risk pooling are examined and beta-based formulae achieved. Results highlight how much the conditions for adding new risky positions are stronger than those required for ris…
In this paper, we provide conceptional explanations for the geodesic and Jacobi field correspondences for homothetic navigation, and then let them guide us to the shortcuts to some well known flag curvature and S-curvature formulas. They also help us directly see the local correspondence between isoparametric functions…
Machine learning reveals hidden features in knot classification.
Gradient-based framework for optimizing text prompts in diffusion models.
A result of M. Ledoux is that a complete Riemannian manifold with non negative Ricci curvature satisfying the Euclidean Sobolev inequality is the Euclidean space. We present a shortcut of the proof. We also give a refinement of a result of B-L. Chen et X-P. Zhu about locally conformally flat manifolds with non negative…
Deep vanilla transformers trained without shortcuts achieve similar performance to standard models.
Many machine learning algorithms are trained and evaluated by splitting data from a single source into training and test sets. While such focus on in-distribution learning scenarios has led to interesting advancement, it has not been able to tell if models are relying on dataset biases as shortcuts for successful predi…
We give a self-contained introduction to the theory of Turaev's shadows as a tool to study 3 and 4-manifolds. The goal of the present paper twofold: on one side it is intended to be a shortcut to a basic use of the theory of shadows, on the other side it gives a sketchy overview of some of the recent results on shadows…
Ozsvath and Szabo have defined a knot concordance invariant tau that bounds the 4-ball genus of a knot. Here we discuss shortcuts to its computation. We include examples of Alexander polynomial one knots for which the invariant is nontrivial, including all iterated untwisted positive doubles of knots with nonnegative T…
In this paper, we study the Hausdorff dimension of the Floyd and Bowditch boundaries of a relatively hyperbolic group, and show that for the Floyd metric and shortcut metrics respectively, they are are both equal to a constant times the growth rate of the group. In the proof, we study a special class of conical points …
New diagonal knots found with non-torus structure.
This work explores non-negative low-rank matrix factorization based on regularized Poisson models (PF or "Poisson factorization" for short) for recommender systems with implicit-feedback data. The properties of Poisson likelihood allow a shortcut for very fast computations over zero-valued inputs, and oftentimes result…
Study highlights robustness issues in healthcare diagnostic models due to distribution shifts.
Materials discovery is crucial for making scientific advances in many domains. Collections of data from experiments and first-principle computations have spurred interest in applying machine learning methods to create predictive models capable of mapping from composition and crystal structures to materials properties. …
SGD quickly learns a spurious XOR feature before the signal feature, revealing learning dynamics.
Martingale Doppelgänger-Eval benchmarks VLMs on candlestick evidence vs. trend extrapolation
Learning long-term dependencies is a key long-standing challenge of recurrent neural networks (RNNs). Hierarchical recurrent neural networks (HRNNs) have been considered a promising approach as long-term dependencies are resolved through shortcuts up and down the hierarchy. Yet, the memory requirements of Truncated Bac…
The paper analyzes how generated data improves adversarial training in high-dimensional regression.
Sarkar and Wang proved that the hat version of Heegaard Floer homology group of a closed oriented 3-manifold is combinatorial starting from an arbitrary nice Heegaard diagram and in fact every closed oriented 3-manifold admits such a Heegaard diagram. Plamenevskaya showed that the contact Ozsvath-Szabo invariant is com…
Graph neural controlled differential equations learn graph dynamics from vertex observations.
Recently, we proposed to transform the outputs of each hidden neuron in a multi-layer perceptron network to have zero output and zero slope on average, and use separate shortcut connections to model the linear dependencies instead. We continue the work by firstly introducing a third transformation to normalize the scal…
Generative classifiers show surprising human-like performance.
This paper is a new step in the project of systematic description of colored knot polynomials started in arXiv:1506.00339. In this paper, we managed to explicitly find the inclusive Racah matrix, i.e. the whole set of mixing matrices in channels R^3->Q with all possible Q, for R=[3,1]. The calculation is made possible …
Paper explores anchoring for vision models, improving generalization and safety.
We introduce a new parameterization method for deep learning layers using spectral tensor train decomposition.