A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Boost-and-Skip generates minority samples without guidance, faster and more effectively.
RNN operators solve Newton's equations with large timesteps for molecular dynamics.
Accurate forward modeling is important for solving inverse problems. An inaccurate wave-equation simulation, as a forward operator, will offset the results obtained via inversion. In this work, we consider the case where we deal with incomplete physics. One proxy of incomplete physics is an inaccurate discretization of…
Paper analyzes the free energy of CNNs with skip connections in Bayesian learning.
The paper improves model-based reinforcement learning by using multi-timestep objectives.
Self-supervised skip-tree training improves mathematical reasoning in language models.
TabSODA improves imputation of surveys with skips and ordinal data.
The paper proves skip connections help neural networks avoid shallow local minima.
Skip connections are an essential component of current state-of-the-art deep neural networks (DNNs) such as ResNet, WideResNet, DenseNet, and ResNeXt. Despite their huge success in building deeper and more powerful DNNs, we identify a surprising security weakness of skip connections in this paper. Use of skip connectio…
This paper examines how skip connections prevent rank collapse in sequence models.
While increasingly deep networks are still in general desired for achieving state-of-the-art performance, for many specific inputs a simpler network might already suffice. Existing works exploited this observation by learning to skip convolutional layers in an input-dependent manner. However, we argue their binary deci…
New insights into GNN optimization reveal skip connections and depth accelerate training.
High order splitting schemes with complex timesteps are applied to Kolmogorov backward equations stemming from stochastic differential equations in Stratonovich form. In the setting of weighted spaces, the necessary analyticity of the split semigroups can be easily proved. A numerical example from interest rate theory,…
Corrected CBOW performs similarly to Skip-gram.
Diffusion-GAN uses diffusion to improve GAN training stability and realism.
This work analyzes how bottleneck layers and skip connections affect linear denoising autoencoders' generalization.
Novel model improves clinical risk prediction by transferring knowledge between tasks over time.
PAM models generate dependent random distributions across groups with overlapping clusters.
DenseNets introduce concatenation-type skip connections that achieve state-of-the-art accuracy in several computer vision tasks. In this paper, we reveal that the topology of the concatenation-type skip connections is closely related to the gradient propagation which, in turn, enables a predictable behavior of DNNs' te…
DARTS- improves robustness by factoring out skip connections' advantage.
Variational autoencoders learn distributions of high-dimensional data. They model data with a deep latent-variable model and then fit the model by maximizing a lower bound of the log marginal likelihood. VAEs can capture complex distributions, but they can also suffer from an issue known as "latent variable collapse," …
We show that the skip-gram formulation of word2vec trained with negative sampling is equivalent to a weighted logistic PCA. This connection allows us to better understand the objective, compare it to other word embedding methods, and extend it to higher dimensional models.
Paper explores variable skipping to speed up range density estimation.
Theory explains why DARTS favors deep architectures over shallow ones.
Generates low-dimensional node vectors for graphs with privacy while preserving structural preferences.
Simplified Bayesian neural networks reduce model complexity and improve interpretability.
Study on autoencoder denoising in high dimensions.
This paper explains GCNs using NTKs and improves their performance.
Regularized SB process speeds up generative modeling.
Backpropagation is the workhorse of deep learning, however, several other biologically-motivated learning rules have been introduced, such as random feedback alignment and difference target propagation. None of these methods have produced a competitive performance against backpropagation. In this paper, we show that bi…
In this work, we extend the SchNet architecture by using weighted skip connections to assemble the final representation. This enables us to study the relative importance of each interaction block for property prediction. We demonstrate on both the QM9 and MD17 dataset that their relative weighting depends strongly on t…
A new method learns dynamic graph representations from time-varying data.
Proposes SNML for selecting word2vec Skip-gram dimensionality.
A long-standing obstacle to progress in deep learning is the problem of vanishing and exploding gradients. Although, the problem has largely been overcome via carefully constructed initializations and batch normalization, architectures incorporating skip-connections such as highway and resnets perform much better than …
This work shows dimension regularization can replace skip-gram negative sampling for graph embeddings, improving efficiency and performance.
XAI identifies key time steps for early crop classification.
We introduce a method which enables a recurrent dynamics model to be temporally abstract. Our approach, which we call Adaptive Skip Intervals (ASI), is based on the observation that in many sequential prediction tasks, the exact time at which events occur is irrelevant to the underlying objective. Moreover, in many sit…
Eliciting semantic similarity between concepts in the biomedical domain remains a challenging task. Recent approaches founded on embedding vectors have gained in popularity as they risen to efficiently capture semantic relationships The underlying idea is that two words that have close meaning gather similar contexts. …
Dealing with uncertainty in Bayesian Network structures using maximum a posteriori (MAP) estimation or Bayesian Model Averaging (BMA) is often intractable due to the superexponential number of possible directed, acyclic graphs. When the prior is decomposable, two classes of graphs where efficient learning can take plac…
New RL method learns to skip states in linearly -realizable MDPs, simplifying to linear MDPs.
We propose an algorithm which predicts each subsequent time step relative to the previous timestep of intractable short rate model (when adjusted for drift and overall distribution of previous percentile result) and show that the method achieves superior outcomes to the unbiased estimate both on the trained dataset and…
A new adaptive splitting method improves accuracy for Cox-Ingersoll-Ross model.
Memory-augmented neural networks consisting of a neural controller and an external memory have shown potentials in long-term sequential learning. Current RAM-like memory models maintain memory accessing every timesteps, thus they do not effectively leverage the short-term memory held in the controller. We hypothesize t…
CAggNet improves medical image segmentation by fusing coarse and fine features.
Federated Learning aims to train distributed deep models without sharing the raw data with the centralized server. Similarly, in distributed inference of neural networks, by partitioning the network and distributing it across several physical nodes, activations and gradients are exchanged between physical nodes, rather…
Recent work has demonstrated that embeddings of tree-like graphs in hyperbolic space surpass their Euclidean counterparts in performance by a large margin. Inspired by these results and scale-free structure in the word co-occurrence graph, we present an algorithm for learning word embeddings in hyperbolic space from fr…
Since the invention of word2vec, the skip-gram model has significantly advanced the research of network embedding, such as the recent emergence of the DeepWalk, LINE, PTE, and node2vec approaches. In this work, we show that all of the aforementioned models with negative sampling can be unified into the matrix factoriza…
Spacetimeformer learns spatiotemporal relationships from data alone.