A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
ResiliNet improves distributed neural network inference resilience.
This article introduces proximal cell complexes in a hyperconnected space. Hyperconnectedness encodes how collections of path-connected sub-complexes in a Alexandroff-Hopf-Whitehead CW space are near to or far from each other. Several main results are given, namely, a hyper-connectedness form of CW (Closure Finite Weak…
Paper analyzes the free energy of CNNs with skip connections in Bayesian learning.
Skip connections make adversarial examples more transferable in ResNets.
Self-supervised skip-tree training improves mathematical reasoning in language models.
TabSODA improves imputation of surveys with skips and ordinal data.
The paper proves skip connections help neural networks avoid shallow local minima.
DFS dynamically decides bitwidths for layers to balance accuracy and efficiency.
This paper examines how skip connections prevent rank collapse in sequence models.
Boost-and-Skip generates minority samples without guidance, faster and more effectively.
New insights into GNN optimization reveal skip connections and depth accelerate training.
Corrected CBOW performs similarly to Skip-gram.
This work analyzes how bottleneck layers and skip connections affect linear denoising autoencoders' generalization.
PAM models generate dependent random distributions across groups with overlapping clusters.
DenseNets introduce concatenation-type skip connections that achieve state-of-the-art accuracy in several computer vision tasks. In this paper, we reveal that the topology of the concatenation-type skip connections is closely related to the gradient propagation which, in turn, enables a predictable behavior of DNNs' te…
DARTS- improves robustness by factoring out skip connections' advantage.
Variational autoencoders learn distributions of high-dimensional data. They model data with a deep latent-variable model and then fit the model by maximizing a lower bound of the log marginal likelihood. VAEs can capture complex distributions, but they can also suffer from an issue known as "latent variable collapse," …
We show that the skip-gram formulation of word2vec trained with negative sampling is equivalent to a weighted logistic PCA. This connection allows us to better understand the objective, compare it to other word embedding methods, and extend it to higher dimensional models.
Paper explores variable skipping to speed up range density estimation.
Theory explains why DARTS favors deep architectures over shallow ones.
Generates low-dimensional node vectors for graphs with privacy while preserving structural preferences.
Simplified Bayesian neural networks reduce model complexity and improve interpretability.
Study on autoencoder denoising in high dimensions.
This paper explains GCNs using NTKs and improves their performance.
Backpropagation is the workhorse of deep learning, however, several other biologically-motivated learning rules have been introduced, such as random feedback alignment and difference target propagation. None of these methods have produced a competitive performance against backpropagation. In this paper, we show that bi…
In this work, we extend the SchNet architecture by using weighted skip connections to assemble the final representation. This enables us to study the relative importance of each interaction block for property prediction. We demonstrate on both the QM9 and MD17 dataset that their relative weighting depends strongly on t…
A new method learns dynamic graph representations from time-varying data.
Proposes SNML for selecting word2vec Skip-gram dimensionality.
A long-standing obstacle to progress in deep learning is the problem of vanishing and exploding gradients. Although, the problem has largely been overcome via carefully constructed initializations and batch normalization, architectures incorporating skip-connections such as highway and resnets perform much better than …
This work shows dimension regularization can replace skip-gram negative sampling for graph embeddings, improving efficiency and performance.
We introduce a method which enables a recurrent dynamics model to be temporally abstract. Our approach, which we call Adaptive Skip Intervals (ASI), is based on the observation that in many sequential prediction tasks, the exact time at which events occur is irrelevant to the underlying objective. Moreover, in many sit…
Eliciting semantic similarity between concepts in the biomedical domain remains a challenging task. Recent approaches founded on embedding vectors have gained in popularity as they risen to efficiently capture semantic relationships The underlying idea is that two words that have close meaning gather similar contexts. …
DVNet efficiently segments large neurovascular datasets using skip connections.
New RL method learns to skip states in linearly -realizable MDPs, simplifying to linear MDPs.
CAggNet improves medical image segmentation by fusing coarse and fine features.
Recent work has demonstrated that embeddings of tree-like graphs in hyperbolic space surpass their Euclidean counterparts in performance by a large margin. Inspired by these results and scale-free structure in the word co-occurrence graph, we present an algorithm for learning word embeddings in hyperbolic space from fr…
Since the invention of word2vec, the skip-gram model has significantly advanced the research of network embedding, such as the recent emergence of the DeepWalk, LINE, PTE, and node2vec approaches. In this work, we show that all of the aforementioned models with negative sampling can be unified into the matrix factoriza…
The present paper develops a novel aggregated gradient approach for distributed machine learning that adaptively compresses the gradient communication. The key idea is to first quantize the computed gradients, and then skip less informative quantized gradient communications by reusing outdated gradients. Quantizing and…
Proposes PSCs for UQ in deep nets without retraining.
We revisit skip-gram negative sampling (SGNS), one of the most popular neural-network based approaches to learning distributed word representation. We first point out the ambiguity issue undermining the SGNS model, in the sense that the word vectors can be entirely distorted without changing the objective value. To res…
Theoretical limits of deep residual networks show consistent covariance structures.
We introduce a novel approach, requiring only mild assumptions, for the characterization of deep neural networks at initialization. Our approach applies both to fully-connected and convolutional networks and easily incorporates batch normalization and skip-connections. Our key insight is to consider the evolution with …
Study proves minimality of certain hyperplane intersections in wide cones.
We present a probabilistic language model for time-stamped text data which tracks the semantic evolution of individual words over time. The model represents words and contexts by latent trajectories in an embedding space. At each moment in time, the embedding vectors are inferred from a probabilistic version of word2ve…
InfiniteWalk connects deep network embeddings to spectral graph theory with a nonlinear transformation.
Proposes unbiased estimators for training mixture of experts models.
This paper introduces Selective-Backprop, a technique that accelerates the training of deep neural networks (DNNs) by prioritizing examples with high loss at each iteration. Selective-Backprop uses the output of a training example's forward pass to decide whether to use that example to compute gradients and update para…
Network embedding techniques inspired by word2vec represent an effective unsupervised relational learning model. Commonly, by means of a Skip-Gram procedure, these techniques learn low dimensional vector representations of the nodes in a graph by sampling node-context examples. Although many ways of sampling the contex…