Current algorithms for deep learning probably cannot run in the brain because they rely on weight transport, where forward-path neurons transmit their synaptic weights to a feedback path, in a way that is likely impossible biologically. An algorithm called feedback alignment achieves deep learning without weight transp…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “weight transport”
Neural networks trained with backpropagation, the standard algorithm of deep learning which uses weight transport, are easily fooled by existing gradient-based adversarial attacks. This class of attacks are based on certain small perturbations of the inputs to make networks misclassify them. We show that less biologica…
Enhances flexibility in data reweighting with optimal transport and maximum entropy principles.
problem Adapting empirical distributions to predefined constraints on moments, tail behavior, etc.
method Nonparametric distributional constraints, maximum entropy principle, optimal transport.
result Maximum entropy weight adjusted empirical distribution close to a specified distribution in optimal transport metric.
Paper uses GNN and conformal prediction for accurate edge weight prediction.
problem Predicting edge weights on graphs for various applications.
method Graph Neural Network (GNN) with conformal prediction and error reweighting.
result Our method provides better coverage and efficiency than baselines.
A new method aligns source and target distributions by tuning their weights.
problem Domain adaptation on unlabeled target datasets using labeled source datasets.
method Weighted Joint Distribution Optimal Transport (WJDOT) method that finds alignment between source and target distributions and re-weighting of source distributions.
result Achieves state-of-the-art performance on simulated and real-life datasets.
Two local learning rules are investigated to avoid weight transport in neural networks.
problem Local learning rules that avoid weight transport are unstable and require tuning.
method Investigated two non-local learning rules and a more robust local rule.
result Non-local learning rules match state-of-the-art performance and operate effectively in noisy updates.
ROTS improves sentence similarity by incorporating structural information.
problem Measuring sentence similarity with theoretical insights and structural awareness.
method Recursive Optimal Transport (ROT) framework to incorporate structural information.
result ROTS outperforms weakly supervised approaches in sentence similarity tasks.
A Lagrangian perspective on nonautonomous advection-diffusion processes in the low-diffusivity limitmath.AP
Study mass transport in low-diffusivity using Lagrangian coordinates.
problem Mass preserving transport of passive tracers in low-diffusivity limit.
method Lagrangian coordinates, time-averaged diffusion equation, weighted manifold structure.
result Leading order asymptotics extend to dominant nontrivial singular value in low-diffusivity limit.
This paper establishes rates of universal approximation for the shallow neural tangent kernel (NTK): network weights are only allowed microscopic changes from random initialization, which entails that activations are mostly unchanged, and the network is nearly equivalent to its linearization. Concretely, the paper has …
New method uses optimal transport for better covariate matching in causal effect estimation.
problem Estimating causal effects in observational studies with high-dimensional covariates.
method Multimarginal unbalanced optimal transport for interpretable matching.
result Method provides interpretable weights and competitive performance with k-nearest neighbors.
A new method corrects bias in causal inference by balancing covariate distributions.
problem Bias in causal effect estimates due to covariate imbalance.
method Causal Optimal Transport, a nonparametric approach targeting distributional balance.
result Causal Optimal Transport outperforms other methods in misspecified models.
New bounds for PDA using partial optimal transport improve domain alignment.
problem Scarcity of labeled target data with abundant source data.
method Derive theoretical bounds based on partial optimal transport.
result Theoretical bounds support partial Wasserstein distance for domain alignment.
Optimal transport aligns source and target distributions for domain adaptation.
problem Unsupervised domain adaptation with joint class-conditional and label shifts.
method Minimizes importance weighted loss and Wasserstein distance for aligned marginals and class-conditional distributions.
result Our method outperforms competitors on various domain adaptation tasks.
Synthetic Ricci flows defined for metric measure spaces.
problem Characterizing Ricci flows for non-smooth spaces.
method Heat flow, optimal transport, volume asymptotics.
result Equivalent characterizations of weighted Ricci flow.
Transformer models align words through attention weights, closely approximating Optimal Transport.
problem Understanding the internal mechanism of transformer models in language processing.
method Empirical evidence and theoretical analysis of attention weights and their relation to Optimal Transport.
result Transformer models can simulate gradient descent on the dual of entropy-regularized OT problem, providing a theoretical foundation for token alignment.
Unified formula for arbitrary liquidity operations in weighted AMMs
problem Decentralized resource allocation in intelligent transportation systems
method Weighted invariant adapted from Balancer-type AMMs
result Unified formula for four resource allocation operations
Study optimal transport on null hypersurfaces and null energy condition.
problem Optimal transport degeneracy on null hypersurfaces.
method Developed tools to characterize null energy condition using convexity properties of entropy.
result Optimal transport characterization of null energy condition.
We prove in this paper that the weighted volume of the set of integral transportation matrices between two integral histograms r and c of equal sum is a positive definite kernel of r and c when the set of considered weights forms a positive definite matrix. The computation of this quantity, despite being the subject of…
A new metric for comparing probability measures on graphs, scalable and negative definite.
problem Optimal transport's high complexity and indefiniteness for kernel machines.
method Sobolev transport metric for graph metrics, closed-form formula, negative definiteness.
result Sobolev transport yields a scalable and negative definite metric.
Stein transport improves Bayesian inference with faster convergence and reduced variance.
problem Efficiently approximating posterior distributions in Bayesian inference.
method A novel Bayesian inference method using Stein transport, which pushes particles along a curve of tempered distributions.
result Stein transport reaches posterior approximations faster and more accurately than Stein variational gradient descent (SVGD).
A method for optimal Bayesian filtering using progressive particle flow and optimal transport maps.
problem Optimizing Bayesian filtering with deterministic particles to avoid degeneration.
method Progressive flow of particles through a sequence of sub-steps, each using an optimal transport map to replace non-equally weighted particles with equally weighted ones.
result The method avoids particle degeneration and simplifies the filtering process by not requiring inversions or monotonicity constraints.
Wasserstein Dictionary Learning: Optimal Transport-based unsupervised non-linear dictionary learningstat.ML
This paper introduces a new nonlinear dictionary learning method for histograms in the probability simplex. The method leverages optimal transport theory, in the sense that our aim is to reconstruct histograms using so-called displacement interpolations (a.k.a. Wasserstein barycenters) between dictionary atoms; such at…
New method enhances adversarial robustness of deep learning models.
problem Improving the robustness of deep learning models against adversarial attacks.
method Optimal transport regularized divergences applied to distributionally robust optimization.
result Improved adversarial robustness on CIFAR-10 and CIFAR-100 datasets.
We present a notion of super Ricci flow for time-dependent finite weighted graphs. A challenging feature is that these flows typically encounter singularities where the underlying graph structure changes. Our notion is robust enough to allow the flow to continue past these singularities. As a crucial tool for this purp…
Survey of diffusion and optimal transport methods in machine learning.
problem Design and analysis of time-evolving probability distributions in machine learning.
method Switch from Eulerian to Lagrangian representation through vector fields.
result Both diffusion methods and optimal transport offer computational advantages.
Partial Fusion of Neural Networks: Efficient Tradeoffs Between Ensembles and Weight Aggregationcs.LG
Partial fusion combines neural networks to balance accuracy and efficiency.
problem Balancing accuracy and computational cost in neural networks.
method Extending weight aggregation methods based on neuron-level similarity, using partial optimal transport to match similar neurons.
result Achieves a flexible tradeoff between computational cost and performance.
The study characterizes straight-line flows in dynamic measure transport.
problem Tackles the challenge of designing flows that are easy to integrate.
method Characterizes straight-line flows using a PDE and Reynolds tensor.
result Characterizes affine-in-time interpolants and necessary conditions for flow geometry.
Paper introduces a novel framework for supervised graph prediction using Optimal Transport.
problem Supervised labeled graph prediction.
method Fused Gromov-Wasserstein (FGW) loss and FGW barycenter with neural network weights and learned graphs.
result The method can interpolate in the labeled graph space and achieve good performance on difficult problems.
This paper proposes a new method for conditional sampling using optimal transport.
problem Sampling conditional distributions in Bayesian inference and density estimation.
method Iterative block-triangular transport maps solving an optimal transport problem with a weighted L2 cost function.
result The proposed method extends the data-driven approach for conditional sampling.
FedAVOT improves federated learning by aligning user distributions.
problem Partial client participation leads to biased and unstable updates in federated learning.
method Formulates aggregation as masked optimal transport to align availability and importance distributions.
result Achieves a standard O(1/√T) rate, independent of the number of participating users per round.
The aim of this paper is to prove isoperimetric inequalities on submanifolds of the Euclidean space using mass transportation methods. We obtain a sharp ?weighted isoperimetric inequality? and a nonsharp classical inequality similar to the one obtained by J. Michael and L. Simon. The proof relies on the description of …
Sliced Optimal Transport simplifies OT for fast computation.
problem Efficient computation of distances and barycenters for probability measures.
method Combines OT, integral geometry, and statistics for fast computation.
result Retains rich geometric structure while speeding up computations.
Develops new synthetic Ricci flow concepts for metric measure spaces.
problem No specific problem stated; focuses on new mathematical concepts.
method Formulated in terms of dynamic convexity and local concavity of entropy, and global/short-time asymptotic transport cost estimates.
result Shows these properties characterise smooth (weighted) Ricci flows.
New method uses Ricci curvature for hypergraph clustering, outperforming existing techniques.
problem Community detection in hypergraphs with large hyperedges.
method Extending Ricci flow to hypergraphs by defining edge probability measures and transporting them on the line expansion.
result Enhanced sensitivity to hypergraph structure, especially in large hyperedges.
Algorithm classifies point clouds using deep set linearized optimal transport.
problem Classifying point clouds efficiently and accurately.
method Deep Set Linearized Optimal Transport, ICNNs, and a discriminator network.
result Efficiently distinguishes between various classes of point clouds.
sEM uses optimal transport to improve EM algorithm for better convergence and avoiding local optima.
problem Improving the EM algorithm for better convergence and avoiding local optima.
method sEM uses entropic optimal transport to compute responsibilities in the expectation step, leading to better global convergence guarantees and avoiding local optima.
result sEM learns cell labels significantly better than other approaches, improving convergence and avoiding local optima.
Extends martingale Schrödinger bridge to arbitrary dimensions and characterizes it.
problem Tackles the martingale Schrödinger bridge in arbitrary dimensions.
method Identifies continuous-time counterpart and relates to variational problems.
result Continuous martingale Schrödinger bridge coincides with Föllmer martingale in irreducible case.
Paper reformulates UOT as non-negative penalized linear regression for efficient algorithms.
problem Optimal transport with relaxed marginal conditions.
method Reformulate UOT as non-negative penalized linear regression, propose multiplicative updates.
result Efficient algorithms for UOT with quadratic penalties, continuity of solutions.
Theory of learning with weight-distribution constraints.
problem Understanding how structure influences function in neural networks.
method Statistical mechanical theory and optimal transport.
result Reduction in capacity due to constrained weight-distribution is related to Wasserstein distance.
Paper defines a new distance metric for comparing learning tasks.
problem Comparing difficulty of learning tasks between source and target.
method Information geometry, optimal transport, coupled transfer distance.
result Coupled transfer distance correlates with fine-tuning difficulty.
New scalable algorithm for non-negative linear regression with entropy-regularized OT loss.
problem Generalizing task-specific linear models to broader applications.
method Sinkhorn-like scaling iterations for convex penalty and datafit terms.
result Simple multiplicative updates for various penalty and datafit terms.
Optimizes transport in Finsler spacetimes with lower Ricci bounds.
problem Optimizing transport in Finsler spacetimes with lower Ricci bounds.
method Using optimal transport and weighted Ricci curvature bounds.
result Proves timelike curvature-dimension condition for Finsler spacetimes.
Traffic forecasting problem remains a challenging task in the intelligent transportation system due to its spatio-temporal complexity. Although temporal dependency has been well studied and discussed, spatial dependency is relatively less explored due to its large variations, especially in the urban environment. In thi…
Neural Local Wasserstein Regression models distribution-on-distribution regression with flexible, localized transport maps.
problem Estimating distribution-on-distribution regression with global optimal transport maps or linearization limitations.
method Proposes Neural Local Wasserstein Regression, a flexible nonparametric framework using locally defined transport maps in Wasserstein space.
result Demonstrates effective capture of nonlinear and high-dimensional distributional relationships.
Extends DAMs to Gaussian distributions for efficient pattern storage and retrieval.
problem Limited storage capacity and retrieval methods for non-vector pattern representations.
method Introduces a log-sum-exp energy function over Gaussian distributions, using optimal transport maps for retrieval dynamics.
result Proves exponential storage capacity and provides quantitative retrieval guarantees.
Two methods for interpolating manifold-valued data are presented.
problem Interpolating manifold-valued functions with derivative constraints.
method Two approaches: weighted Riemannian barycenters and tangent space interpolation.
result Both methods are valid and perform well with numerical examples.
While the backpropagation of error algorithm enables deep neural network training, it implies (i) bidirectional synaptic weight transport and (ii) update locking until the forward and backward passes are completed. Not only do these constraints preclude biological plausibility, but they also hinder the development of l…
Optimal transport simplifies machine learning by comparing probability measures.
problem Comparing and manipulating probability distributions in machine learning.
method Uses optimal transport to compare and manipulate probability distributions, combining statistical and geometric perspectives.
result Optimal transport provides a unified framework for various machine learning tasks.