In this paper, we explore degrees of freedom in deep sigmoidal neural networks. We show that the degrees of freedom in these models is related to the expected optimism, which is the expected difference between test error and training error. We provide an efficient Monte-Carlo method to estimate the degrees of freedom f…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “degrees of freedom”
The derivation of statistical properties for Partial Least Squares regression can be a challenging task. The reason is that the construction of latent components from the predictor variables also depends on the response variable. While this typically leads to good performance and interpretable models in practice, it ma…
Measures neural network complexity via effective degrees of freedom.
problem Challenges in quantifying neural network complexity.
method Adapts generalized degrees of freedom (GDF) for binary outcomes and compares with cross-validation and null degrees of freedom.
result GDF provides a robust measure of model complexity for neural networks.
This paper investigates the model degrees of freedom in k-means clustering. An extension of Stein's lemma provides an expression for the effective degrees of freedom in the k-means model. Approximating the degrees of freedom in practice requires simplifications of this expression, however empirical studies evince the a…
Regularization aims to improve prediction performance of a given statistical modeling approach by moving to a second approach which achieves worse training error but is expected to have fewer degrees of freedom, i.e., better agreement between training and prediction error. We show here, however, that this expected beha…
A new distribution family extends the -stable distribution with a degree of freedom parameter.
problem Lack of moments in the -stable distribution.
method Wright function framework to combine and extend distribution families.
result Generalized -stable distribution with valid moments.
Unified finetuning of all quantization degrees of freedom achieves state-of-the-art 4-bit quantization.
problem Achieving high accuracy in quantized neural networks while maintaining speed and resource constraints.
method Quantization-aware finetuning (QFT) that jointly optimizes all quantization degrees of freedom.
result 4-bit weight quantization results on-par with state-of-the-art (SoTA) within PTQ constraints.
We re-examine classical mechanics with both commuting and anticommuting degrees of freedom. We do this by defining the phase dynamics of a general Lagrangian system as an implicit differential equation in the spirit of Tulczyjew. Rather than parametrising our basic degrees of freedom by a specified Grassmann algebra, w…
Fewer degrees of freedom can train deep networks, showing a sharp phase transition.
problem Training deep networks with fewer degrees of freedom than parameters.
method Examined success probability of hitting training loss sub-level sets within random subspaces.
result Threshold training dimension increases as desired final loss decreases.
The paper argues for using more degrees of freedom in empirical financial analysis to improve conclusions.
problem Improving trustworthiness of financial analysis conclusions.
method Using more degrees of freedom and forking paths in multiple testing.
result Forking paths raises the bar for significance in multiple testing.
We discuss normal forms and symplectic invariants of parabolic orbits and cuspidal tori in integrable Hamiltonian systems with two degrees of freedom. Such singularities appear in many integrable systems in geometry and mathematical physics and can be considered as the simplest example of degenerate singularities. We a…
EPGP surrogate outperforms finite elements in solving wave equations.
problem Benchmarking Gaussian Process surrogates vs. finite elements for wave equation solutions.
method EPGP uses penalized least squares and exponential-polynomial bases; CN-FEM employs Crank--Nicolson time stepping.
result EPGP achieves lower error than CN-FEM under matched degrees-of-freedom.
Developed a new thresholding method that connects soft and hard thresholding.
problem Connecting soft and hard thresholding methods in data analysis.
method Scaled soft thresholding method with empirical scaling values.
result Found two sources of over-fitting in the scaled soft thresholding method.
A central question in modern machine learning and imaging sciences is to quantify the number of effective parameters of vastly over-parameterized models. The degrees of freedom is a mathematically convenient way to define this number of parameters. Its computation and properties are well understood when dealing with di…
Develops unisolvent weights for Nédélec second family finite elements in 2D.
problem Finding efficient degrees of freedom for Nédélec second family finite elements.
method Uses techniques of homological algebra to obtain degrees of freedom for differential forms.
result Provides a family of unisolvent and minimal physical degrees of freedom for Nédélec second family finite elements.
Physical systems differring in their microscopic details often display strikingly similar behaviour when probed at macroscopic scales. Those universal properties, largely determining their physical characteristics, are revealed by the powerful renormalization group (RG) procedure, which systematically retains "slow" de…
Directly simulates squared Bessel processes efficiently.
problem Simulating squared Bessel processes accurately and efficiently.
method Two-dimensional Chebyshev expansion for non-central chi-square distribution inverse.
result Accurate and efficient simulation for various degrees of freedom.
Deep neural networks reduce loan portfolio risk.
problem Minimizing risk in peer-to-peer lending portfolios.
method Proposed DeNN and DSNN models to predict default probability and time.
result DeNN model significantly reduces portfolio VaRs at various confidence levels.
Symplectifying Biarcsmath.SG
A new method joins two arcs with a degree of freedom.
problem Joining two arcs with a precise point.
method Geometric approach using tangent vectors and points.
result A novel method to determine the join point.
In this paper we are investigating variational homogeneous second order differential equations by considering the questions of how many different variational principles exist for a given spray. We focus our attention on h(2)-variationality; that is, the regular Lagrange function is homogeneous of degree two in the dire…
Geometry of supergravity and the Batalin--Vilkovisky formulation of the theory in ten dimensionshep-th
We simplify supergravity in 10D using geometric insights.
problem Formulating supergravity in 10D without Lorentz degrees of freedom.
method Using generalised geometry, we describe the fibred structure of field space.
result Our action satisfies the classical master equation without Lorentz terms.
For an integrable Hamiltonian with degrees of freedom, we show the conditions on perturbations, for which invariant tori can be destructed.
This paper studies schemes to de-bias the Lasso in a linear model where the goal is to construct confidence intervals for in a direction , where has iid rows. We show that previously analyzed propositions to de-bias the Lasso require a modification in order to enjoy efficiency in a f…
Transformers reduce redundancy by focusing on invariant relational quantities.
problem Substantial internal redundancy in Transformer models due to coordinate-dependent representations and continuous symmetries.
method Reformulate representations, attention mechanisms, and optimization dynamics in terms of invariant relational quantities, eliminating redundant degrees of freedom by construction.
result Architectures that operate directly on relational structures, providing a principled geometric framework for reducing parameter redundancy and analyzing optimization.
AI agents improve forecast combination in empirical economics.
problem Hidden researcher degrees of freedom in AI-generated code.
method Adapted agent-loop architecture to empirical economics, added holdout evaluation.
result Independent agent searches find better forecast methods than benchmarks.
Paper develops methods for estimating and simulating a Student-t Lévy regression model.
problem Estimation and simulation of Student-t Lévy process with arbitrary degrees of freedom.
method Develops a two-step estimation procedure and simulates increments using inverse Fourier transform.
result Efficient estimation and simulation methods for Student-t Lévy process.
A method to automatically choose feature dimensions in linear attention for better approximation quality.
problem Choosing the feature dimension in linear attention to balance quality and efficiency.
method Statistical degrees of freedom for determining feature dimension, layer-wise training strategy.
result Our method achieves smaller approximation error compared to fixed dimensions and improves model performance.
Choosing appropriate architectures and regularization strategies for deep networks is crucial to good predictive performance. To shed light on this problem, we analyze the analogous problem of constructing useful priors on compositions of functions. Specifically, we study the deep Gaussian process, a type of infinitely…
AI agents improve forecast combination but require transparency.
problem AI coding agents increase flexibility in empirical economics, leading to hidden degrees of freedom.
method Adapted open-source agent-loop architecture to empirical economics workflow, adding post-search holdout evaluation.
result Multiple agent runs outperform standard benchmarks in rolling evaluation but not all on post-search holdout.
The runtime for Kernel Partial Least Squares (KPLS) to compute the fit is quadratic in the number of examples. However, the necessity of obtaining sensitivity measures as degrees of freedom for model selection or confidence intervals for more detailed analysis requires cubic runtime, and thus constitutes a computationa…
In this note, we consider generalizations of the asymptotic Hopf invariant, or helicity, for Hamiltonian systems with one-and-a-half degrees of freedom and symplectic diffeomorphisms of a two-disk to itself.
New complexity measures explain overparameterized models' surprising performance.
problem Understanding why overparameterized models generalize well despite fitting training data.
method Reinterpreting classical degrees of freedom in a random-X setting.
result Random-X prediction error better explains generalization in complex models.
Corrects GCV for inconsistent risk estimation in finite ensembles of penalized estimators.
problem Inconsistent risk estimation of GCV for finite ensembles of penalized estimators.
method Identifies a correction involving an additional scalar correction based on degrees of freedom adjusted training errors from each ensemble component.
result CGCV maintains computational advantages of GCV and is model-free uniformly consistent for ridge regression.
Simple model finds high correlation in retail crypto returns.
problem Discerning correlation in retail cryptocurrency markets without factors.
method Used N*(N) statistic to compare models of daily returns.
result High average pairwise correlation (60%) found, supports isotropic model.
Novel digital twin for complex systems improves performance.
problem Lack of practical implementation details for stochastic nonlinear MDOF systems.
method Decouples time-scales, uses physics-based model, Bayesian filtering, and machine learning.
result Excellent performance of proposed digital twin framework validated by examples.
The paper proposes a thermodynamic potential to guide training of generative models, breaking ergodicity to improve functionality.
problem Improving generative model functionality while limiting access to underrepresented patterns.
method Constructing a thermodynamic potential that guides training, leading to multiple minima in the free energy.
result Training a generative model breaks ergodicity, preventing escape into the high-temperature phase.
Efficient algorithm to select tuning parameters in sparse regression modeling with regularizationstat.ME
In sparse regression modeling via regularization such as the lasso, it is important to select appropriate values of tuning parameters including regularization parameters. The choice of tuning parameters can be viewed as a model selection and evaluation problem. Mallows' type criteria may be used as a tuning param…
Complex geometry and symplectic geometry are mirrors in string theory. The recently developed generalised complex geometry interpolates between the two of them. On the other hand, the classical and quantum mechanics of a finite number of degrees of freedom are respectively described by a symplectic structure and a comp…
We investigate local configuration controllability for mechanical control systems within the affine connection formalism. Extending the work by Lewis for the single-input case, we are able to characterize local configuration controllability for systems with degrees of freedom and input forces.
The paper surveys open problems and questions related to different aspects of integrable systems with finitely many degrees of freedom. Many of the open problems were suggested by the participants of the conference "Finite-dimensional Integrable Systems, FDIS 2017" held at CRM, Barcelona in July 2017.
New forms generalize Whitney forms with rational coefficients for numerical analysis.
problem Numerical problems with singularities near simplex faces.
method Introduce shadow forms and degrees of freedom for integration over faces of blow-up simplices.
result Obtain isomorphism between shadow forms cohomology and cellular cohomology of blow-up simplices.
It is shown that the equation which describes constant mean curvature surface via the generalized Weierstrass-Enneper inducing has Hamiltonian form. Its simplest finite-dimensional reduction has two degrees of freedom, integrable and its trajectories correspond to well-known Delaunay and do Carmo-Dajzcer surfaces (i.e.…
We suggest several mathematical counterparts to the idea of "effective degrees of freedom" and formulate specific questions, much of which are inspired by Larry Guth's results and ideas on the Hermann Weyl kind of asymptotics of the Morse (co)homology spectra of the volume energy function on the spaces of cycles in bal…
Derives log-corrections in AdS4/CFT3 using supergravity localization.
problem Factorizing log-corrections in AdS4/CFT3.
method Supergravity localization, Atiyah-Singer index theorem, fixed points (NUTs), fixed two-manifolds (Bolts).
result General fixed-point formula for log-corrections in large N expansion.
Proposes a boosting framework for sparsity in grouped covariates.
problem Sparsity and selection bias in grouped covariates.
method Component-wise and group-wise gradient boosting with adjusted degrees of freedom.
result Reduces bias and improves predictability in variable selection.
Proposes a balanced multi-component and multi-layer neural network for efficient function approximation.
problem Accurately and efficiently approximating complex functions with high degrees of freedom and computational cost.
method Inspired by a multi-component approach, MMNN combines single-layer networks with a multi-layer decomposition strategy.
result Significant reduction in training parameters, more efficient training process, and improved accuracy compared to FCNNs or MLPs.
Paper analyzes a three-loop linkage, showing it's overconstrained and shaky.
problem Analyzing a three-loop spatial linkage's degree of freedom and configuration space.
method Local analysis of differential degrees of freedom, computation of kinematic tangent cone, and c-space approximation.
result The linkage has a finite degree of freedom 3 and is locally a smooth manifold, making it shaky.
Ray-Singer torsion measures light degrees of freedom in black hole entropy.
problem Entropy of small black holes with many light particles.
method Computes partition function at genus one using mirror symmetry and Ray-Singer torsion.
result Ray-Singer torsion provides an effective quantum gravity cutoff known as the species scale.