Study connects Gaussian processes and regularization for sequence-function mappings.
problem Understanding and interpreting sequence-function maps in biology.
method Relates Gaussian process priors, regularization, and gauge fixing in overparameterized weight space.
result Established the relationship between regularized regression and Gaussian processes in function space.
The paper studies gravitational instantons with flat limits and finds elliptic regularity estimates.
problem Analyzing gravitational instantons with flat limits using elliptic analysis.
method Establishing elliptic regularity estimates and showing uniform constants for a family of metrics.
result The Laplacian is Fredholm and an isomorphism between specific weighted spaces.
Improved loss functions adapt to weight-space anisotropy, outperforming isotropic counterparts.
problem Adapting to the anisotropic nature of deep weight spaces for better performance.
method Refined local entropic loss functions restricted to a subset of weights, exploiting anisotropy.
result Partial local entropies outperform isotropic counterparts on image classification tasks.
LaLoRA prevents forgetting in LoRA fine-tuning.
problem Catastrophic forgetting in fine-tuned models.
method LaLoRA applies Laplace approximation to LoRA weights for regularization.
result Improved learning-forgetting trade-off with controllable regularization strength.
Functional input neural networks approximate continuous functions on weighted spaces.
problem Approximating continuous functions on infinite-dimensional weighted spaces.
method Additive family mapping, non-linear activation, linear readouts, Stone-Weierstrass theorem.
result Global universal approximation of continuous functions on weighted spaces.
The paper defines function spaces on manifolds with bounded or singular geometries.
problem Defining function spaces on manifolds with various geometries.
method Introduces and analyzes Sobolev, Besov, and Bessel potential spaces on uniformly regular and singular Riemannian manifolds.
result Demonstrates maximal regularity for a linear parabolic problem on singular manifolds.
We propose regularizing the empirical loss for semi-supervised learning by acting on both the input (data) space, and the weight (parameter) space. We show that the two are not equivalent, and in fact are complementary, one affecting the minimality of the resulting representation, the other insensitivity to nuisance va…
MLDS dataset reveals hidden model behavior via weight-space analysis.
problem Neural networks' opacity makes them hard to evaluate.
method Presented MLDS dataset of trained neural networks.
result Weight-space analysis reveals meaningful divergence with small changes in training data.
Paper addresses variational inference issues in Bayesian neural networks.
problem Negative infinite ELBO for function-space priors in BNNs.
method Regularized KL divergence for well-defined function-space variational inference.
result Method provides competitive uncertainty estimates for BNNs.
Data-driven model shows deep learning weights behave like a liquid.
problem Understanding the structure of deep neural network optimization landscapes.
method Statistical mechanics framework to model high-dimensional weight spaces.
result Deep networks' weight spaces are well-connected, not hierarchical, unlike shallow networks.
The paper investigates how neural network weights evolve to monitor training progress.
problem Monitoring the training progress of neural networks in a cost-effective manner.
method Investigates the evolution of neural network weights in weight space.
result DNN models evolve on unique, smooth trajectories in weight space that can be used to track training progress.
Paper solves CR positive mass and Yamabe problems on weighted spaces.
problem CR positive mass and Yamabe problems on weighted spaces.
method Analyzes sub-Laplacian on Folland-Stein spaces.
result CR positive mass and Yamabe problems resolved.
Variational inference struggles with weight symmetries in neural networks, leading to biased posteriors.
problem Weight space symmetries in neural networks cause multimodal posteriors, challenging variational inference.
method Developed a symmetrization mechanism to create permutation invariant variational posteriors.
result Symmetrized variational posteriors have a better fit to the true posterior and improved predictive performance.
Proposes learning invariances in neural networks using a weight-space approach.
problem Learning invariances from data in neural networks remains an open problem.
method Minimizes a lower bound on the marginal likelihood in weight space.
result Results in higher performing models with naturally learned invariances.
Study on Bayesian transformers finds issues with weight-space inference and prior specification.
problem Challenges in obtaining meaningful uncertainty estimates for transformer models.
method Proposed a novel method based on implicit reparameterization of the Dirichlet distribution for variational inference on attention weights.
result Proposed method performs competitively with baselines in estimating predictive uncertainty.
Extends Feller theory to non-locally compact spaces for stochastic equations.
problem Stochastic partial differential equations and fractional processes.
method Extended Feller processes and proofs of folklore results.
result No condition of generalized Feller semigroups can be dropped.
In this note we prove a new ε-regularity theorem for the Ricci flow. Let (M^n,g(t)) with t\in [-T,0] be a Ricci flow and H_{x} the conjugate heat kernel centered at a point (x,0) in the final time slice. Substituting H_{x} into Perelman's W-functional produces a monotone function W_{x}(s) of s \in [-T,0], the pointed e…
Recently, Neural networks have seen a huge surge in its adoption due to their ability to provide high accuracy on various tasks. On the other hand, the existence of adversarial examples have raised suspicions regarding the generalization capabilities of neural networks. In this work, we focus on the weight matrix learn…
New method approximates curvature from symmetries in deep networks.
problem Hard to approximate curvature in large deep networks.
method Analytically averaging over group actions that leave the loss invariant to construct structured Hessian approximations.
result Structured Hessian approximations from single gradients can be estimated, stored, and inverted.
The study analyzes local minima in ReLU networks and finds low probability of bad local minima.
problem Understanding the existence and probability of local minima in ReLU networks.
method Theoretical analysis combined with linear programming and experiments on MNIST and CIFAR-10 datasets.
result No bad differentiable local minima found almost everywhere in weight space.
New method adapts neural networks without losing prior knowledge.
problem Understanding and enabling flexible adaptation of neural networks.
method Differential geometry framework, functionally invariant paths (FIP).
result Achieves comparable state-of-the-art performance on continual learning and sparsification tasks.
Dropout increases the generalization of neural networks by expanding the weight space.
problem Understanding and improving the generalization of neural networks.
method Introducing weight expansion and showing that dropout leads to it.
result Dropout increases the generalization of neural networks by expanding the weight space.
Proposes NRS to find flat minima in deep neural networks.
problem Finding optimal solutions in deep neural networks with overparameterization.
method NRS leverages the concept of flat minima and uses Kullback-Leibler divergence to regularize the neighborhood region in weight space.
result NRS drives optimizers towards flat minima, improving generalization ability across various model architectures.
Study nonnegative solutions on Riemannian manifolds using fractional porous medium equation.
problem Analyzing solutions to fractional porous medium equation on noncompact Riemannian manifolds.
method Existence and smoothing estimates for weak solutions in L1 and weighted spaces. result Results hold for Euclidean and hyperbolic spaces, including larger data classes.
For a 2-periodic link L~ in the thickened annulus and its quotient link L, we exhibit a spectral sequence with E1≅AKh(L~)⊗F2F2[θ,θ−1]⇉E∞≅AKh(L)⊗F2F2[θ,θ−1]. This spectral sequence splits along qu…
Characterizes deep neural network weight space for adversarial attacks.
problem Poor performance of deep learning models in adversarial examples.
method Characterizes deep neural network solution space using two paradigms.
result Adversarial attacks are less successful against Associative Memory Models.
Soft diamond regularizers improve deep learning performance and sparsity.
problem Improving deep learning performance and sparsity of trained weights.
method New soft diamond synaptic weight priors based on thick-tailed symmetric alpha stable probability curves.
result Soft diamond regularizers outperform state-of-the-art methods in deep learning tasks.
Let g be a complex, simple Lie algebra with Cartan subalgebra h and Weyl group W. We construct a one-parameter family of flat connections D on h with values in any finite-dimensional h-module V and simple poles on the root hyperplanes. The corresponding monodromy representation of the braid group B of type g is a defor…
LGV boosts adversarial attacks by improving surrogate models.
problem Improving the transferability of black-box adversarial attacks.
method LGV uses a pretrained surrogate model and multiple weight sets from additional training epochs to generate an effective surrogate ensemble.
result LGV outperforms other test-time transformations by significant margins.
The paper shows neural networks can approximate functions over non-compact domains with non-polynomial activation.
problem Approximating functions over non-compact domains using neural networks.
method Using single-hidden-layer feedforward neural networks with non-polynomial activation functions over non-compact subsets of Euclidean spaces.
result Neural networks can approximate functions in weighted Ck-spaces and weighted Sobolev spaces over unbounded domains. Paper studies solutions to a specific equation in conformal geometry with singular sets.
problem Singular solutions to a fully non-linear equation in conformal geometry.
method Uses a classical gluing method adapted to the fully non-linear setting.
result Shows the classical gluing method can be applied to the σ2--Yamabe equation. Bayesian deep learning uses function-space priors to improve model uncertainty and robustness.
problem Bayesian deep learning struggles with model-specific weight-space priors that are hard to interpret and specify.
method Apply a Dirichlet prior in predictive space and perform approximate function-space variational inference.
result The approach improves uncertainty quantification, scalability, and adversarial robustness in large-scale image classification.
The paper studies the Heath-Jarrow-Morton-Musiela equation of the bond market. The equation is analyzed in weighted spaces of functions defined on [0,+∞). Sufficient conditions for local and global existence are obtained . For equation with the linear diffusion term the conditions for global existence are close …
Noise injection regularizes Hessian, improving neural network training and generalization.
problem Regularizing over-parameterized neural networks with nonconvex and nonlinear geometry.
method Injecting isotropic Gaussian noise into weight matrices and designing a two-point estimate of the Hessian penalty.
result Effective regularization of Hessian improves generalization, achieving up to 2.4% test accuracy increase.
We propose Radial Bayesian Neural Networks (BNNs): a variational approximate posterior for BNNs which scales well to large models while maintaining a distribution over weight-space with full support. Other scalable Bayesian deep learning methods, like MC dropout or deep ensembles, have discrete support-they assign zero…
High order splitting schemes with complex timesteps are applied to Kolmogorov backward equations stemming from stochastic differential equations in Stratonovich form. In the setting of weighted spaces, the necessary analyticity of the split semigroups can be easily proved. A numerical example from interest rate theory,…
We give a topological interpretation of the space of L2-harmonic forms on finite-volume manifolds with sufficiently pinched negative curvature. We give examples showing that this interpretation fails if the curvature is not sufficiently pinched and that our result is sharp with respect to the pinching constants. The me…
Bayesian Neural ODEs improve vessel trajectory prediction with better uncertainty estimates.
problem Challenges in predicting vessel trajectories from irregular AIS data.
method Adopted a Gaussian process (GP) kernel-based prior on the vector field evaluated at measurement points, combined with probabilistic multiple shooting for long trajectories.
result Improved accuracy and uncertainty quantification in vessel trajectory predictions.
Uniform elliptic theory for Dirac operators on orbifold resolutions.
problem Analyzing Dirac operators on orbifold resolutions.
method Viewing orbifolds as conically fibred singular spaces and resolving them by gluing asymptotically conical fibrations.
result Uniform index formula for Dirac operators on orbifold resolutions.
We introduce a quotient of the affine Temperley-Lieb category that encodes all weight-preserving linear maps between finite-dimensional sl(2)-representations. We study the diagrammatic idempotents that correspond to projections onto extremal weight spaces and find that they satisfy similar properties as Jones-Wenzl pro…
DeepWeightFlow generates diverse neural network weights efficiently.
problem Generating complete neural network weights efficiently and accurately.
method Flow Matching in weight space with Git Re-Basin and TransFusion.
result DeepWeightFlow generates high-accuracy neural networks without fine-tuning.
This paper is devoted to obtaining a wellposedness result for multidimensional BSDEs with possibly unbounded random time horizon and driven by a general martingale in a filtration only assumed to satisfy the usual hypotheses, i.e. the filtration may be stochastically discontinuous. We show that for stochastic Lipschitz…
This paper explores Bayesian Neural Network posteriors, uncovering symmetries and their impact.
problem Understanding the complex posterior distribution of deep Bayesian Neural Networks.
method Investigates optimal approaches for approximating posteriors, analyzes modes, and explores visualizations.
result Uncovered weight-space symmetries and their impact on the posterior, particularly scaling symmetries.
In this paper, we investigate the behavior of ADM mass and Einstein-Hilbert functional under the Yamabe flow. Through studying the Yamabe flow by weighted spaces, we show that ADM mass and Einstein-Hilbert functional are well-defined and monotone non-increasing under the Yamabe flow on n-dimensional, n≥3, asymp…
Koopman mode analysis applied to neural networks for training optimization.
problem Optimizing neural network training, identifying issues, and speeding up learning.
method Koopman operator analysis of neural network dynamics.
result Spectral analysis of Koopman operator aids in determining network depth, initialization quality, and training termination.
Hypernetworks are neural networks that generate weights for another neural network. We formulate the hypernetwork training objective as a compromise between accuracy and diversity, where the diversity takes into account trivial symmetry transformations of the target network. We explain how this simple formulation gener…
The Dirac operator d+delta on the Hodge complex of a Riemannian manifold is regarded as an annihilation operator A. On a weighted space L_mu^2 Omega, [A,A*] acts as multiplication by a positive constant on excited states if and only if the logarithm of the measure density of mu satisfies a pair of equations. The equati…
The graph-based semi-supervised label propagation algorithm has delivered impressive classification results. However, the estimated soft labels typically contain mixed signs and noise, which cause inaccurate predictions due to the lack of suitable constraints. Moreover, available methods typically calculate the weights…