arXiv research

For neural networks (NNs) with rectified linear unit (ReLU) or binary activation functions, we show that their training can be accomplished in a reduced parameter space. Specifically, the weights in each neuron can be trained on the unit sphere, as opposed to the entire space, and the threshold can be trained in a boun…

Paper reduces movement primitive dimensionality in parameter space.

We introduce two approaches for combining neural evolution strategy (NES) and proximal policy optimization (PPO): parameter transfer and parameter space noise. Parameter transfer is a PPO agent with parameters transferred from a NES agent. Parameter space noise is to directly add noise to the PPO agent`s parameters. We…

We study the Euler-Lagrange equations for a parameter dependent $G$-invariant Lagrangian on a homogeneous $G$-space. We consider the pullback of the parameter dependent Lagrangian to the Lie group $G$, emphasizing the special invariance properties of the associated Euler-Poincaré equations with advected parameters.

Black-box optimizers that explore in parameter space have often been shown to outperform more sophisticated action space exploration methods developed specifically for the reinforcement learning problem. We examine these black-box methods closely to identify situations in which they are worse than action space explorat…

Gradient flow in parameters equals linear interpolation in outputs.

The paper defines and studies canonical parameters on surfaces in 4D space.

Empirical study shows removing neural parameter symmetries impacts model performance.

Sharp bounds on hyperbolic metrics in Ptolemaic spaces are derived.

Improved Bayesian optimization for conditional parameter spaces.

The paper introduces canonical parameters for marginally trapped surfaces in Minkowski space.

Study geometric flows with varying parameters and prove continuous dependence.

A new method reduces high-dimensional parameter spaces for faster numerical tasks.

We define a family of kernels for mixed continuous/discrete hierarchical parameter spaces and show that they are positive definite.

Let $G$ be a group acting properly and by isometries on a metric space $X$; it follows that the quotient or orbit space $X/G$ is also a metric space. We study the Vietoris-Rips and Čech complexes of $X/G$. Whereas (co)homology theories for metric spaces let the scale parameter of a Vietoris-Rips or Čech complex go to z…

Conditions for uniquely identifying parameters of deep ReLU networks.

Gaussian kernel fails on circle and related spaces.

This paper proves Hitchin moduli spaces are ALG gravitational instantons.

This work formalizes and extends parameter sharing in multi-agent reinforcement learning.

Singularities of a statistical model are the elements of the model's parameter space which make the corresponding Fisher information matrix degenerate. These are the points for which estimation techniques such as the maximum likelihood estimator and standard Bayesian procedures do not admit the root-$n$ parametric rate…

Generatability in metric spaces studied with novel novelty parameters.

Teichmüller space and hyperelliptic surfaces parametrized by angles.

This Ph.D. thesis is devoted to the constructions of Lagrangian formulation on Finsler and Kawaguchi manifolds. While Finsler geometry is a natural extension of Riemannian geometry, Kawaguchi geometry is the extension of Finsler geometry to higher order derivatives and to k-dimensional parameter space. The latter exten…

New method encodes function preferences into neural nets for better generalization.

As machine learning systems get widely adopted for high-stake decisions, quantifying uncertainty over predictions becomes crucial. While modern neural networks are making remarkable gains in terms of predictive accuracy, characterizing uncertainty over the parameters of these models is challenging because of the high d…

Sliced Inverse Regression reduces parameter space for estimating complex financial models.

Improved likelihood-free inference for high-dimensional models.

Dual Bayesian Affine Estimators for Wiener-type state-space models

Indecomposable symmetric Lorentzian manifolds of non-constant curvature are called Cahen-Wallach spaces. Their isometry classes are described by continuous families of real parameters. We derive necessary and sufficient conditions for the existence of compact quotients of Cahen-Wallach spaces in terms of these paramete…

New Riemannian radial distributions help estimate parameters on symmetric spaces.

A projective parameter of a geodesic on a Finsler space is defined to be solution of a certain ODE. Using projective parameter and Funk metric, one can construct a projectively invariant intrinsic pseudo-distance on a Finsler space. In the present work, solutions of the projective parameter's ODE are characterized with…

We study geodesics of the form $γ(t)=π(\exp(tX)\exp(tY))$, $X,Y\in \fr{g}=\operatorname{Lie}(G)$, in homogeneous spaces $G/K$, where $π:G\rightarrow G/K$ is the natural projection. These curves naturally generalise homogeneous geodesics, that is orbits of one-parameter subgroups of $G$ (i.e. $γ(t)=π(\exp (tX))$, $X\in …

Function-space MAP estimation leads to better generalization and robustness.

Using the navigation data (h,W) of a Kropina space, we characterize weakly-Berwald Kropina spaces and Berwald Kropina spaces by means of the Killing vector field W and the parallel vector field W, respectively. Moreover, the local 1-parameter group of Finslerian local isometries of the Kropina space coincides with the …

A non-elementary Möbius group generated by two-parabolics is determined up to conjugation by one complex parameter and the parameter space has been extensively studied. In this paper, we use the results of \cite{GW} to obtain an additional structure for the parameter space, which we term the {\sl two-parabolic space}. …

Deep reinforcement learning (RL) methods generally engage in exploratory behavior through noise injection in the action space. An alternative is to add noise directly to the agent's parameters, which can lead to more consistent exploration and a richer set of behaviors. Methods such as evolutionary strategies use param…

Develops methods to measure and set function-space learning rates in neural networks.

In this paper we construct a compactification for the parameter space of convex projective structures on a fixed n-manifold M. This parameter space is a closed semi-algebraic subset of the variety of characters of representations of the fundamental group of M in SL_{n+1}(R). The boundary is the inverse limit of an inve…

Trains neural networks to efficiently solve Navier-Stokes equations across parameter space.

In practical Bayesian optimization, we must often search over structures with differing numbers of parameters. For instance, we may wish to search over neural network architectures with an unknown number of layers. To relate performance data gathered for different architectures, we define a new kernel for conditional p…

We compute the Betti numbers of the moduli space of rank 3 parabolic Higgs bundles, using Morse theory. A key point is that certain critical submanifolds of the Morse function can be identified with moduli spaces of parabolic triples. These moduli spaces come in families depending on a real parameter and we study their…

In many tasks, in particular in natural science, the goal is to determine hidden system parameters from a set of measurements. Often, the forward process from parameter- to measurement-space is a well-defined function, whereas the inverse problem is ambiguous: one measurement may map to multiple different sets of param…

New PG samplers improve inference in coupled state-space models.

One popular hypothesis of neural network generalization is that the flat local minima of loss surface in parameter space leads to good generalization. However, we demonstrate that loss surface in parameter space has no obvious relationship with generalization, especially under adversarial settings. Through visualizing …

Functional dimension varies in ReLU networks, with implications for symmetry and connectivity.

A new model approximates complex functions in parameter space.

Subspace identification is a classical and very well studied problem in system identification. The problem was recently posed as a convex optimization problem via the nuclear norm relaxation. Inspired by robust PCA, we extend this framework to handle outliers. The proposed framework takes the form of a convex optimizat…

In this paper, we introduce a new parameter, the affine twist parameter for the affine deformation of a sphere with holes. We show that the affine deformation space can be parametrized by Margulis invariants and affine twist parameters. The affine twist parameter is canonically regarded as a correspondence to the Fench…