arXiv research

Measures neural network complexity via effective degrees of freedom.

The paper proposes a thermodynamic potential to guide training of generative models, breaking ergodicity to improve functionality.

In this paper, we explore degrees of freedom in deep sigmoidal neural networks. We show that the degrees of freedom in these models is related to the expected optimism, which is the expected difference between test error and training error. We provide an efficient Monte-Carlo method to estimate the degrees of freedom f…

The derivation of statistical properties for Partial Least Squares regression can be a challenging task. The reason is that the construction of latent components from the predictor variables also depends on the response variable. While this typically leads to good performance and interpretable models in practice, it ma…

This paper investigates the model degrees of freedom in k-means clustering. An extension of Stein's lemma provides an expression for the effective degrees of freedom in the k-means model. Approximating the degrees of freedom in practice requires simplifications of this expression, however empirical studies evince the a…

Regularization aims to improve prediction performance of a given statistical modeling approach by moving to a second approach which achieves worse training error but is expected to have fewer degrees of freedom, i.e., better agreement between training and prediction error. We show here, however, that this expected beha…

A new distribution family extends the $\alpha$-stable distribution with a degree of freedom parameter.

Unified finetuning of all quantization degrees of freedom achieves state-of-the-art 4-bit quantization.

Generalised Degrees of Freedom (GDF), as defined by Ye (1998 JASA 93:120-131), represent the sensitivity of model fits to perturbations of the data. As such they can be computed for any statistical model, making it possible, in principle, to derive the number of parameters in machine-learning approaches. Defined origin…

New method calculates degrees of freedom for sparse estimation in continuous models.

We re-examine classical mechanics with both commuting and anticommuting degrees of freedom. We do this by defining the phase dynamics of a general Lagrangian system as an implicit differential equation in the spirit of Tulczyjew. Rather than parametrising our basic degrees of freedom by a specified Grassmann algebra, w…

Fewer degrees of freedom can train deep networks, showing a sharp phase transition.

The paper argues for using more degrees of freedom in empirical financial analysis to improve conclusions.

EPGP surrogate outperforms finite elements in solving wave equations.

We discuss normal forms and symplectic invariants of parabolic orbits and cuspidal tori in integrable Hamiltonian systems with two degrees of freedom. Such singularities appear in many integrable systems in geometry and mathematical physics and can be considered as the simplest example of degenerate singularities. We a…

Developed a new thresholding method that connects soft and hard thresholding.

Develops unisolvent weights for Nédélec second family finite elements in 2D.

Physical systems differring in their microscopic details often display strikingly similar behaviour when probed at macroscopic scales. Those universal properties, largely determining their physical characteristics, are revealed by the powerful renormalization group (RG) procedure, which systematically retains "slow" de…

Directly simulates squared Bessel processes efficiently.

Deep neural networks reduce loan portfolio risk.

A new method joins two arcs with a degree of freedom.

A neural network with a single hidden layer can't represent certain multivariable functions.

In this paper we are investigating variational homogeneous second order differential equations by considering the questions of how many different variational principles exist for a given spray. We focus our attention on h(2)-variationality; that is, the regular Lagrange function is homogeneous of degree two in the dire…

We simplify supergravity in 10D using geometric insights.

A new method detects and displays pairwise dependence between variates.

For an integrable Hamiltonian with $d\ (d\geq 2)$ degrees of freedom, we show the conditions on perturbations, for which invariant tori can be destructed.

This paper studies schemes to de-bias the Lasso in a linear model $y=Xβ+ε$ where the goal is to construct confidence intervals for $a_0^Tβ$ in a direction $a_0$, where $X$ has iid $N(0,Σ)$ rows. We show that previously analyzed propositions to de-bias the Lasso require a modification in order to enjoy efficiency in a f…

Transformers reduce redundancy by focusing on invariant relational quantities.

AI agents improve forecast combination in empirical economics.

Paper develops methods for estimating and simulating a Student-t Lévy regression model.

A method to automatically choose feature dimensions in linear attention for better approximation quality.

Choosing appropriate architectures and regularization strategies for deep networks is crucial to good predictive performance. To shed light on this problem, we analyze the analogous problem of constructing useful priors on compositions of functions. Specifically, we study the deep Gaussian process, a type of infinitely…

AI agents improve forecast combination but require transparency.

The runtime for Kernel Partial Least Squares (KPLS) to compute the fit is quadratic in the number of examples. However, the necessity of obtaining sensitivity measures as degrees of freedom for model selection or confidence intervals for more detailed analysis requires cubic runtime, and thus constitutes a computationa…

In this note, we consider generalizations of the asymptotic Hopf invariant, or helicity, for Hamiltonian systems with one-and-a-half degrees of freedom and symplectic diffeomorphisms of a two-disk to itself.

New complexity measures explain overparameterized models' surprising performance.

Corrects GCV for inconsistent risk estimation in finite ensembles of penalized estimators.

Simple model finds high correlation in retail crypto returns.

Novel digital twin for complex systems improves performance.

In sparse regression modeling via regularization such as the lasso, it is important to select appropriate values of tuning parameters including regularization parameters. The choice of tuning parameters can be viewed as a model selection and evaluation problem. Mallows' $C_p$ type criteria may be used as a tuning param…

Complex geometry and symplectic geometry are mirrors in string theory. The recently developed generalised complex geometry interpolates between the two of them. On the other hand, the classical and quantum mechanics of a finite number of degrees of freedom are respectively described by a symplectic structure and a comp…

We investigate local configuration controllability for mechanical control systems within the affine connection formalism. Extending the work by Lewis for the single-input case, we are able to characterize local configuration controllability for systems with $n$ degrees of freedom and $n-1$ input forces.

The paper surveys open problems and questions related to different aspects of integrable systems with finitely many degrees of freedom. Many of the open problems were suggested by the participants of the conference "Finite-dimensional Integrable Systems, FDIS 2017" held at CRM, Barcelona in July 2017.

New forms generalize Whitney forms with rational coefficients for numerical analysis.

It is shown that the equation which describes constant mean curvature surface via the generalized Weierstrass-Enneper inducing has Hamiltonian form. Its simplest finite-dimensional reduction has two degrees of freedom, integrable and its trajectories correspond to well-known Delaunay and do Carmo-Dajzcer surfaces (i.e.…

We suggest several mathematical counterparts to the idea of "effective degrees of freedom" and formulate specific questions, much of which are inspired by Larry Guth's results and ideas on the Hermann Weyl kind of asymptotics of the Morse (co)homology spectra of the volume energy function on the spaces of cycles in bal…

Derives log-corrections in AdS4/CFT3 using supergravity localization.

Proposes a boosting framework for sparsity in grouped covariates.