The paper reveals that deep neural networks have fewer degrees of freedom than parameters, impacting model performance.
problem Understanding the relationship between degrees of freedom and model performance in deep neural networks.
method Developed an efficient Monte-Carlo method to estimate degrees of freedom for multi-class classification methods.
result Degrees of freedom in deep networks are dramatically smaller than the number of parameters, often by several orders of magnitude.
We give the first example of systolic freedom over torsion coefficients. The phenomenon is a bit unexpected (contrary to a conjecture of Gromov's) and more delicate than systolic freedom over the integers.
Study gauge freedoms in elastic wave equations and Dirichlet-to-Neumann map.
problem Recover stiffness tensor and density from Dirichlet-to-Neumann map.
method Analyze invariance under coordinate transformations and gauge freedoms.
result Present gauge freedoms in the Dirichlet-to-Neumann map for Riemannian elastic wave equation.
The derivation of statistical properties for Partial Least Squares regression can be a challenging task. The reason is that the construction of latent components from the predictor variables also depends on the response variable. While this typically leads to good performance and interpretable models in practice, it ma…
Measures neural network complexity via effective degrees of freedom.
problem Challenges in quantifying neural network complexity.
method Adapts generalized degrees of freedom (GDF) for binary outcomes and compares with cross-validation and null degrees of freedom.
result GDF provides a robust measure of model complexity for neural networks.
Measuring supernova neutrinos removes spacetime's conformal freedom.
problem Determining the conformal factor of spacetime's visible part.
method Measuring neutrino cones in addition to light cones.
result The conformal factor can now be determined.
The paper introduces a new method to select high-quality clustering solutions in k-means.
problem Selecting the optimal number of clusters in k-means clustering.
method The paper introduces a new method to estimate the degrees of freedom in k-means clustering, which is used for model selection.
result The proposed method for selecting high-quality clustering solutions is competitive and reliable.
Classifies solutions in multisymplectic field theories using geometric gauge freedom.
problem Classifying solutions in multisymplectic field theories.
method Using the kernel of a premultisymplectic form and equivalence relations.
result Equivalence relations and reduction procedures for sections.
Regularization aims to improve prediction performance of a given statistical modeling approach by moving to a second approach which achieves worse training error but is expected to have fewer degrees of freedom, i.e., better agreement between training and prediction error. We show here, however, that this expected beha…
Re-examines classical mechanics with superdegrees of freedom.
problem Classical mechanics with both commuting and anticommuting degrees of freedom.
method Defines phase dynamics as an implicit differential equation on supermanifolds.
result Defines phase dynamics on arbitrary supermanifolds.
A new distribution family extends the α-stable distribution with a degree of freedom parameter.
problem Lack of moments in the α-stable distribution. method Wright function framework to combine and extend distribution families.
result Generalized α-stable distribution with valid moments. Mathematical counterparts to effective degrees of freedom inspired by Guth's results.
problem Understanding effective degrees of freedom in mathematical contexts.
method Formulating specific questions inspired by Guth's results and Weyl asymptotics.
result New mathematical counterparts to effective degrees of freedom.
Sharp analysis of isotonic regression for binary data, improving calibration bounds.
problem Improving the calibration of probabilistic predictors using isotonic regression.
method Sharp finite-sample characterization of isotonic regression's degrees of freedom using analytic number theory.
result First nontrivial distribution-free guarantee on Expected Calibration Error (ECE) of isotonic regression.
Unified finetuning of all quantization degrees of freedom achieves state-of-the-art 4-bit quantization.
problem Achieving high accuracy in quantized neural networks while maintaining speed and resource constraints.
method Quantization-aware finetuning (QFT) that jointly optimizes all quantization degrees of freedom.
result 4-bit weight quantization results on-par with state-of-the-art (SoTA) within PTQ constraints.
New method calculates degrees of freedom for sparse estimation in continuous models.
problem Quantifying effective parameters in over-parameterized models with large continuous parameter spaces.
method Develops a continuous Lasso method for sparsity-inducing optimization over measure spaces.
result Proof of a continuous degrees of freedom formula for Beurling Lasso.
Machine learning identifies key degrees of freedom in physical systems.
problem Identifying important degrees of freedom in complex systems.
method Artificial neural network based on mutual information and RG procedure.
result Extracted Ising critical exponent using machine learning.
The paper explores the freedom of h(2)-variationality in sprays.
problem Investigating the number of variational principles for sprays.
method Analyzing h(2)-variationality and using the holonomy distribution to calculate freedom.
result The holonomy distribution can be used to calculate the freedom of h(2)-variationality of a spray.
Fewer degrees of freedom can train deep networks, showing a sharp phase transition.
problem Training deep networks with fewer degrees of freedom than parameters.
method Examined success probability of hitting training loss sub-level sets within random subspaces.
result Threshold training dimension increases as desired final loss decreases.
The paper examines parallel one forms on Riemannian and Finslerian manifolds.
problem Existence of parallel one forms on Riemannian and Finslerian manifolds.
method Using Finslerian settings, the paper investigates the existence of parallel one forms on Riemannian manifolds and Finslerian manifolds, proving conditions for their existence and non-existence.
result Conditions for the existence and non-existence of parallel one forms on Riemannian and Finslerian manifolds.
The paper argues for using more degrees of freedom in empirical financial analysis to improve conclusions.
problem Improving trustworthiness of financial analysis conclusions.
method Using more degrees of freedom and forking paths in multiple testing.
result Forking paths raises the bar for significance in multiple testing.
EPGP surrogate outperforms finite elements in solving wave equations.
problem Benchmarking Gaussian Process surrogates vs. finite elements for wave equation solutions.
method EPGP uses penalized least squares and exponential-polynomial bases; CN-FEM employs Crank--Nicolson time stepping.
result EPGP achieves lower error than CN-FEM under matched degrees-of-freedom.
Geometrically describes pseudo-gauge freedom in relativistic hydrodynamics.
problem Pseudo-gauge ambiguity in relativistic hydrodynamics.
method Develops k-contact geometry to describe thermodynamic states and conservation laws. result Identifies pseudo-gauge freedom as the non-uniqueness of thermodynamic equilibrium redefinition.
Directly simulates squared Bessel processes efficiently.
problem Simulating squared Bessel processes accurately and efficiently.
method Two-dimensional Chebyshev expansion for non-central chi-square distribution inverse.
result Accurate and efficient simulation for various degrees of freedom.
The paper improves Lasso de-biasing methods to enhance confidence interval efficiency.
problem Improving confidence intervals for Lasso in high-dimensional linear models.
method Degrees-of-freedom adjustment to modify Lasso de-biasing schemes.
result The degrees-of-freedom adjustment ensures asymptotic efficiency for any direction a0 under certain conditions. Deep neural networks reduce loan portfolio risk.
problem Minimizing risk in peer-to-peer lending portfolios.
method Proposed DeNN and DSNN models to predict default probability and time.
result DeNN model significantly reduces portfolio VaRs at various confidence levels.
The center of mass in General Relativity is hard to define due to coordinate freedom.
problem Defining the center of mass in General Relativity rigorously and consistently.
method Analyzing the challenges in Newtonian Gravity and using Bartnik's asymptotic harmonic coordinates.
result Examples of initial data sets in General Relativity that do not satisfy center of mass definitions.
A new method joins two arcs with a degree of freedom.
problem Joining two arcs with a precise point.
method Geometric approach using tangent vectors and points.
result A novel method to determine the join point.
We developed a perturbation model for affine gravity theories.
problem Cosmological perturbations in theories without metric.
method Segregated perturbations into symmetric and antisymmetric components, decomposing into irreducible elements.
result Fully addressed gauge freedom in affine gravity theories.
A new property fixes look-ahead bias in backtesting and trading pipelines.
problem Fixing look-ahead bias in backtesting and trading pipelines.
method Developed a pipeline calculus separating availability from reference time, and a type-and-effect system for the value-independent fragment.
result The check scales linearly and catches all leaks, including those missed by differential and tiling detectors.
Researchers explore gauge freedom in entropies of q-Gaussian measures.
problem Exploring the gauge freedom of entropies in q-Gaussian measures. method Introducing a refined q-logarithmic function to demonstrate gauge freedom. result Different escort expectations can lead to the same entropy but different relative entropies.
Given a pair of integers m and n such that 1 < m < n, we show that every n-dimensional manifold admits metrics of arbitrarily small total volume, and possessing the following property: every m-dimensional submanifold of less than unit m-volume is necessarily torsion in homology. This result is different from the case o…
Developed a new thresholding method that connects soft and hard thresholding.
problem Connecting soft and hard thresholding methods in data analysis.
method Scaled soft thresholding method with empirical scaling values.
result Found two sources of over-fitting in the scaled soft thresholding method.
Develops unisolvent weights for Nédélec second family finite elements in 2D.
problem Finding efficient degrees of freedom for Nédélec second family finite elements.
method Uses techniques of homological algebra to obtain degrees of freedom for differential forms.
result Provides a family of unisolvent and minimal physical degrees of freedom for Nédélec second family finite elements.
We prove the simultaneous (k,n-k)-systolic freedom, for a pair of adjacent integers k smaller than n/2, of a simply connected n-manifold X. Our construction, related to recent results of I. Babenko, is concentrated in a neighborhood of suitable k-dimensional submanifolds of X. We employ calibration by differential form…
Study symplectic invariants of parabolic orbits and cuspidal tori in integrable systems.
problem Understanding symplectic invariants of degenerate singularities in integrable systems.
method Normal forms and new techniques for studying symplectic invariants.
result New insights into symplectic invariants of degenerate singularities.
We simplify supergravity in 10D using geometric insights.
problem Formulating supergravity in 10D without Lorentz degrees of freedom.
method Using generalised geometry, we describe the fibred structure of field space.
result Our action satisfies the classical master equation without Lorentz terms.
For an integrable Hamiltonian with d (d≥2) degrees of freedom, we show the conditions on perturbations, for which invariant tori can be destructed.
Transformers reduce redundancy by focusing on invariant relational quantities.
problem Substantial internal redundancy in Transformer models due to coordinate-dependent representations and continuous symmetries.
method Reformulate representations, attention mechanisms, and optimization dynamics in terms of invariant relational quantities, eliminating redundant degrees of freedom by construction.
result Architectures that operate directly on relational structures, providing a principled geometric framework for reducing parameter redundancy and analyzing optimization.
AI agents improve forecast combination in empirical economics.
problem Hidden researcher degrees of freedom in AI-generated code.
method Adapted agent-loop architecture to empirical economics, added holdout evaluation.
result Independent agent searches find better forecast methods than benchmarks.
Paper develops methods for estimating and simulating a Student-t Lévy regression model.
problem Estimation and simulation of Student-t Lévy process with arbitrary degrees of freedom.
method Develops a two-step estimation procedure and simulates increments using inverse Fourier transform.
result Efficient estimation and simulation methods for Student-t Lévy process.
Spectral pruning compresses deep networks by reducing degrees of freedom.
problem Efficiently compress deep neural networks for edge devices.
method Develops a new theoretical framework and spectral pruning method.
result Shows a sharp generalization error bound for compressed models.
GDF approach compared to cross-validation for AICc in machine learning models.
problem Estimating model complexity for machine learning models, especially for binary data.
method Generalised Degrees of Freedom (GDF) for model sensitivity, compared to cross-validation.
result GDF-based AICc is similar to cross-validation but unstable for binary data.
A method to automatically choose feature dimensions in linear attention for better approximation quality.
problem Choosing the feature dimension in linear attention to balance quality and efficiency.
method Statistical degrees of freedom for determining feature dimension, layer-wise training strategy.
result Our method achieves smaller approximation error compared to fixed dimensions and improves model performance.
AI agents improve forecast combination but require transparency.
problem AI coding agents increase flexibility in empirical economics, leading to hidden degrees of freedom.
method Adapted open-source agent-loop architecture to empirical economics workflow, adding post-search holdout evaluation.
result Multiple agent runs outperform standard benchmarks in rolling evaluation but not all on post-search holdout.
Choosing appropriate architectures and regularization strategies for deep networks is crucial to good predictive performance. To shed light on this problem, we analyze the analogous problem of constructing useful priors on compositions of functions. Specifically, we study the deep Gaussian process, a type of infinitely…
The runtime for Kernel Partial Least Squares (KPLS) to compute the fit is quadratic in the number of examples. However, the necessity of obtaining sensitivity measures as degrees of freedom for model selection or confidence intervals for more detailed analysis requires cubic runtime, and thus constitutes a computationa…
This paper analyzes the generalization risk of unrolled neural networks using Stein's Unbiased Risk Estimator.
problem Analyzing the generalization risk of unrolled neural networks and its relationship to network design and train sample size.
method Using Stein's Unbiased Risk Estimator (SURE), the paper analyzes the generalization risk with bias and variance components for recurrent unrolled networks, focusing on the degrees-of-freedom (DOF) component and the trace of the end-to-end network Jacobian.
result DOF is well-approximated by the weighted path sparsity of the network under incoherence conditions on the trained weights, and DOF increases with train sample size and converges to the generalization risk for both recurrent and non-recurrent schemes.
New complexity measures explain overparameterized models' surprising performance.
problem Understanding why overparameterized models generalize well despite fitting training data.
method Reinterpreting classical degrees of freedom in a random-X setting.
result Random-X prediction error better explains generalization in complex models.