arXiv research

The paper calculates sensitivities for financial derivatives using path weighting methods.

Study shows how feature weighting affects neural network regularization.

PSiLON Net uses $L_1$ weight normalization and 1-path-norm regularization for efficient learning and sparsity.

We revisit the choice of SGD for training deep neural networks by reconsidering the appropriate geometry in which to optimize the weights. We argue for a geometry invariant to rescaling of weights that does not affect the output of the network, and suggest Path-SGD, which is an approximate steepest descent method with …

Current algorithms for deep learning probably cannot run in the brain because they rely on weight transport, where forward-path neurons transmit their synaptic weights to a feedback path, in a way that is likely impossible biologically. An algorithm called feedback alignment achieves deep learning without weight transp…

Constructs weight 1/2 multiplier systems for a specific group and relates to geometric edge paths.

The paper proposes a method to improve random forest classification accuracy by weighting trees based on their decision path reliability.

Global approximation for piecewise linear paths via signatures.

New MCMC method improves sampling from multimodal distributions.

We study the use of power weighted shortest path distance functions for clustering high dimensional Euclidean data, under the assumption that the data is drawn from a collection of disjoint low dimensional manifolds. We argue, theoretically and experimentally, that this leads to higher clustering accuracy. We also pres…

URGE improves diffusion model quality without gradients or Hessian.

Consider a weighted or unweighted k-nearest neighbor graph that has been built on n data points drawn randomly according to some density p on R^d. We study the convergence of the shortest path distance in such graphs as the sample size tends to infinity. We prove that for unweighted kNN graphs, this distance converges …

Deep networks with path norm regularization can approximate analytic functions.

Optimal a priori estimates are derived for the population risk, also known as the generalization error, of a regularized residual network model. An important part of the regularized model is the usage of a new path norm, called the weighted path norm, as the regularization term. The weighted path norm treats the skip c…

The paper provides an efficient method to price path-dependent derivatives using multiscale stochastic volatility models.

Introduces Exponentially Weighted Signature for better path representation.

Deep learning has become a powerful and popular tool for a variety of machine learning tasks. However, it is challenging to understand the mechanism of deep learning from a theoretical perspective. In this work, we propose a random active path model to study collective properties of deep neural networks with binary syn…

New method adapts neural networks without losing prior knowledge.

Selecting important features in non-linear or kernel spaces is a difficult challenge in both classification and regression problems. When many of the features are irrelevant, kernel methods such as the support vector machine and kernel ridge regression can sometimes perform poorly. We propose weighting the features wit…

PS-IG improves feature attribution by reducing noise and variance.

The paper analyzes the role of ReLU gates in deep learning networks.

The solution path of the 1D fused lasso for an $n$-dimensional input is piecewise linear with $\mathcal{O}(n)$ segments (Hoefling et al. 2010 and Tibshirani et al 2011). However, existing proofs of this bound do not hold for the weighted fused lasso. At the same time, results for the generalized lasso, of which the wei…

It can be conjectured that the colored Jones function of a knot can be computed in terms of counting paths on the graph of a planar projection of a knot. On the combinatorial level, the colored Jones function can be replaced by its weight system. We give two curious formulas for the weight system of a colored Jones fun…

We present evolution equations for a family of paths that results from anisotropically weighting curve energies in non-linear statistics of manifold valued data. This situation arises when performing inference on data that have non-trivial covariance and are anisotropic distributed. The family can be interpreted as mos…

Proposes PGPS for efficient Bayesian inference.

New geometric analysis of PWSPDs balances density and geometry in high-dimensional data.

This work derives closed-form expressions computing the expectation of co-presence and of number of co-occurrences of nodes on paths sampled from a network according to general path weights (a bag of paths). The underlying idea is that two nodes are considered as similar when they often appear together on (preferably s…

Algorithm minimizes regret and converges to equilibria in Markov games.

Functional input neural networks approximate continuous functions on weighted spaces.

Dupire's functional Itô calculus provides an alternative approach to the classical Malliavin calculus for the computation of sensitivities, also called Greeks, of path-dependent derivatives prices. In this paper, we introduce a measure of path-dependence of functionals within the functional Itô calculus framework. Name…

Convolution operations designed for graph-structured data usually utilize the graph Laplacian, which can be seen as message passing between the adjacent neighbors through a generic random walk. In this paper, we propose PAN, a new graph convolution framework that involves every path linking the message sender and recei…

New Lipschitz bound for ReLU networks resists weight rescaling.

Universal approximation for stochastic processes using Brownian motion.

Guyon-Lekeufack model accurately predicts market volatility.

The state-of-art DNN structures involve high computation and great demand for memory storage which pose intensive challenge on DNN framework resources. To mitigate the challenges, weight pruning techniques has been studied. However, high accuracy solution for extreme structured pruning that combines different types of …

The paper identifies network bottlenecks using minimax paths in stochastic networks.

Signature portfolios approximate optimal wealth in non-Markovian markets.

Diagonal linear networks converge to lasso regularization path during training.

A hybrid framework prices options using neural networks and VAE latent space.

It is well known that neural networks with rectified linear units (ReLU) activation functions are positively scale-invariant. Conventional algorithms like stochastic gradient descent optimize the neural networks in the vector space of weights, which is, however, not positively scale-invariant. This mismatch may lead to…

In this work, we extend the SchNet architecture by using weighted skip connections to assemble the final representation. This enables us to study the relative importance of each interaction block for property prediction. We demonstrate on both the QM9 and MD17 dataset that their relative weighting depends strongly on t…

The study reveals how attention paths in Transformers influence learning outcomes.

Improves inference-time alignment for diffusion models without updating weights.

The recently developed bag-of-paths (BoP) framework consists in setting a Gibbs-Boltzmann distribution on all feasible paths of a graph. This probability distribution favors short paths over long ones, with a free parameter (the temperature $T$) controlling the entropic level of the distribution. This formalism enables…

Improved sampling efficiency for molecular systems using path gradients after Flow Matching.

For any ReLU network there is a representation in which the sum of the absolute values of the weights into each node is exactly $1$, and the input layer variables are multiplied by a value $V$ coinciding with the total variation of the path weights. Implications are given for Gaussian complexity, Rademacher complexity,…

A new slicing method speeds up sliced Wasserstein estimation.

A new method to rescale ReLU neural networks based on path-lifting.