In this paper, we explore degrees of freedom in deep sigmoidal neural networks. We show that the degrees of freedom in these models is related to the expected optimism, which is the expected difference between test error and training error. We provide an efficient Monte-Carlo method to estimate the degrees of freedom f…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,291 papers · 148 categories
Trend · papers per month
48 results for “generalized degrees of freedom”
Measures neural network complexity via effective degrees of freedom.
problem Challenges in quantifying neural network complexity.
method Adapts generalized degrees of freedom (GDF) for binary outcomes and compares with cross-validation and null degrees of freedom.
result GDF provides a robust measure of model complexity for neural networks.
A new distribution family extends the -stable distribution with a degree of freedom parameter.
problem Lack of moments in the -stable distribution.
method Wright function framework to combine and extend distribution families.
result Generalized -stable distribution with valid moments.
Regularization aims to improve prediction performance of a given statistical modeling approach by moving to a second approach which achieves worse training error but is expected to have fewer degrees of freedom, i.e., better agreement between training and prediction error. We show here, however, that this expected beha…
The derivation of statistical properties for Partial Least Squares regression can be a challenging task. The reason is that the construction of latent components from the predictor variables also depends on the response variable. While this typically leads to good performance and interpretable models in practice, it ma…
The paper introduces a new method to select high-quality clustering solutions in k-means.
problem Selecting the optimal number of clusters in k-means clustering.
method The paper introduces a new method to estimate the degrees of freedom in k-means clustering, which is used for model selection.
result The proposed method for selecting high-quality clustering solutions is competitive and reliable.
We re-examine classical mechanics with both commuting and anticommuting degrees of freedom. We do this by defining the phase dynamics of a general Lagrangian system as an implicit differential equation in the spirit of Tulczyjew. Rather than parametrising our basic degrees of freedom by a specified Grassmann algebra, w…
Mathematical counterparts to effective degrees of freedom inspired by Guth's results.
problem Understanding effective degrees of freedom in mathematical contexts.
method Formulating specific questions inspired by Guth's results and Weyl asymptotics.
result New mathematical counterparts to effective degrees of freedom.
An analysis of binary isotonic regression: degrees of freedom and implications for calibrationstat.ML
Sharp analysis of isotonic regression for binary data, improving calibration bounds.
problem Improving the calibration of probabilistic predictors using isotonic regression.
method Sharp finite-sample characterization of isotonic regression's degrees of freedom using analytic number theory.
result First nontrivial distribution-free guarantee on Expected Calibration Error (ECE) of isotonic regression.
Unified finetuning of all quantization degrees of freedom achieves state-of-the-art 4-bit quantization.
problem Achieving high accuracy in quantized neural networks while maintaining speed and resource constraints.
method Quantization-aware finetuning (QFT) that jointly optimizes all quantization degrees of freedom.
result 4-bit weight quantization results on-par with state-of-the-art (SoTA) within PTQ constraints.
New method calculates degrees of freedom for sparse estimation in continuous models.
problem Quantifying effective parameters in over-parameterized models with large continuous parameter spaces.
method Develops a continuous Lasso method for sparsity-inducing optimization over measure spaces.
result Proof of a continuous degrees of freedom formula for Beurling Lasso.
The paper argues for using more degrees of freedom in empirical financial analysis to improve conclusions.
problem Improving trustworthiness of financial analysis conclusions.
method Using more degrees of freedom and forking paths in multiple testing.
result Forking paths raises the bar for significance in multiple testing.
The paper explores the freedom of h(2)-variationality in sprays.
problem Investigating the number of variational principles for sprays.
method Analyzing h(2)-variationality and using the holonomy distribution to calculate freedom.
result The holonomy distribution can be used to calculate the freedom of h(2)-variationality of a spray.
Directly simulates squared Bessel processes efficiently.
problem Simulating squared Bessel processes accurately and efficiently.
method Two-dimensional Chebyshev expansion for non-central chi-square distribution inverse.
result Accurate and efficient simulation for various degrees of freedom.
Machine learning identifies key degrees of freedom in physical systems.
problem Identifying important degrees of freedom in complex systems.
method Artificial neural network based on mutual information and RG procedure.
result Extracted Ising critical exponent using machine learning.
Developed a new thresholding method that connects soft and hard thresholding.
problem Connecting soft and hard thresholding methods in data analysis.
method Scaled soft thresholding method with empirical scaling values.
result Found two sources of over-fitting in the scaled soft thresholding method.
Fewer degrees of freedom can train deep networks, showing a sharp phase transition.
problem Training deep networks with fewer degrees of freedom than parameters.
method Examined success probability of hitting training loss sub-level sets within random subspaces.
result Threshold training dimension increases as desired final loss decreases.
Study symplectic invariants of parabolic orbits and cuspidal tori in integrable systems.
problem Understanding symplectic invariants of degenerate singularities in integrable systems.
method Normal forms and new techniques for studying symplectic invariants.
result New insights into symplectic invariants of degenerate singularities.
The paper improves Lasso de-biasing methods to enhance confidence interval efficiency.
problem Improving confidence intervals for Lasso in high-dimensional linear models.
method Degrees-of-freedom adjustment to modify Lasso de-biasing schemes.
result The degrees-of-freedom adjustment ensures asymptotic efficiency for any direction under certain conditions.
EPGP surrogate outperforms finite elements in solving wave equations.
problem Benchmarking Gaussian Process surrogates vs. finite elements for wave equation solutions.
method EPGP uses penalized least squares and exponential-polynomial bases; CN-FEM employs Crank--Nicolson time stepping.
result EPGP achieves lower error than CN-FEM under matched degrees-of-freedom.
Spectral pruning compresses deep networks by reducing degrees of freedom.
problem Efficiently compress deep neural networks for edge devices.
method Develops a new theoretical framework and spectral pruning method.
result Shows a sharp generalization error bound for compressed models.
Develops unisolvent weights for Nédélec second family finite elements in 2D.
problem Finding efficient degrees of freedom for Nédélec second family finite elements.
method Uses techniques of homological algebra to obtain degrees of freedom for differential forms.
result Provides a family of unisolvent and minimal physical degrees of freedom for Nédélec second family finite elements.
This paper analyzes the generalization risk of unrolled neural networks using Stein's Unbiased Risk Estimator.
problem Analyzing the generalization risk of unrolled neural networks and its relationship to network design and train sample size.
method Using Stein's Unbiased Risk Estimator (SURE), the paper analyzes the generalization risk with bias and variance components for recurrent unrolled networks, focusing on the degrees-of-freedom (DOF) component and the trace of the end-to-end network Jacobian.
result DOF is well-approximated by the weighted path sparsity of the network under incoherence conditions on the trained weights, and DOF increases with train sample size and converges to the generalization risk for both recurrent and non-recurrent schemes.
Deep neural networks reduce loan portfolio risk.
problem Minimizing risk in peer-to-peer lending portfolios.
method Proposed DeNN and DSNN models to predict default probability and time.
result DeNN model significantly reduces portfolio VaRs at various confidence levels.
AI agents improve forecast combination in empirical economics.
problem Hidden researcher degrees of freedom in AI-generated code.
method Adapted agent-loop architecture to empirical economics, added holdout evaluation.
result Independent agent searches find better forecast methods than benchmarks.
Symplectifying Biarcsmath.SG
A new method joins two arcs with a degree of freedom.
problem Joining two arcs with a precise point.
method Geometric approach using tangent vectors and points.
result A novel method to determine the join point.
New complexity measures explain overparameterized models' surprising performance.
problem Understanding why overparameterized models generalize well despite fitting training data.
method Reinterpreting classical degrees of freedom in a random-X setting.
result Random-X prediction error better explains generalization in complex models.
In this note, we consider generalizations of the asymptotic Hopf invariant, or helicity, for Hamiltonian systems with one-and-a-half degrees of freedom and symplectic diffeomorphisms of a two-disk to itself.
Corrects GCV for inconsistent risk estimation in finite ensembles of penalized estimators.
problem Inconsistent risk estimation of GCV for finite ensembles of penalized estimators.
method Identifies a correction involving an additional scalar correction based on degrees of freedom adjusted training errors from each ensemble component.
result CGCV maintains computational advantages of GCV and is model-free uniformly consistent for ridge regression.
The paper proposes a thermodynamic potential to guide training of generative models, breaking ergodicity to improve functionality.
problem Improving generative model functionality while limiting access to underrepresented patterns.
method Constructing a thermodynamic potential that guides training, leading to multiple minima in the free energy.
result Training a generative model breaks ergodicity, preventing escape into the high-temperature phase.
Geometry of supergravity and the Batalin--Vilkovisky formulation of the theory in ten dimensionshep-th
We simplify supergravity in 10D using geometric insights.
problem Formulating supergravity in 10D without Lorentz degrees of freedom.
method Using generalised geometry, we describe the fibred structure of field space.
result Our action satisfies the classical master equation without Lorentz terms.
For an integrable Hamiltonian with degrees of freedom, we show the conditions on perturbations, for which invariant tori can be destructed.
Transformers reduce redundancy by focusing on invariant relational quantities.
problem Substantial internal redundancy in Transformer models due to coordinate-dependent representations and continuous symmetries.
method Reformulate representations, attention mechanisms, and optimization dynamics in terms of invariant relational quantities, eliminating redundant degrees of freedom by construction.
result Architectures that operate directly on relational structures, providing a principled geometric framework for reducing parameter redundancy and analyzing optimization.
Paper develops methods for estimating and simulating a Student-t Lévy regression model.
problem Estimation and simulation of Student-t Lévy process with arbitrary degrees of freedom.
method Develops a two-step estimation procedure and simulates increments using inverse Fourier transform.
result Efficient estimation and simulation methods for Student-t Lévy process.
Efficient algorithm to select tuning parameters in sparse regression modeling with regularizationstat.ME
In sparse regression modeling via regularization such as the lasso, it is important to select appropriate values of tuning parameters including regularization parameters. The choice of tuning parameters can be viewed as a model selection and evaluation problem. Mallows' type criteria may be used as a tuning param…
New forms generalize Whitney forms with rational coefficients for numerical analysis.
problem Numerical problems with singularities near simplex faces.
method Introduce shadow forms and degrees of freedom for integration over faces of blow-up simplices.
result Obtain isomorphism between shadow forms cohomology and cellular cohomology of blow-up simplices.
A method to automatically choose feature dimensions in linear attention for better approximation quality.
problem Choosing the feature dimension in linear attention to balance quality and efficiency.
method Statistical degrees of freedom for determining feature dimension, layer-wise training strategy.
result Our method achieves smaller approximation error compared to fixed dimensions and improves model performance.
Choosing appropriate architectures and regularization strategies for deep networks is crucial to good predictive performance. To shed light on this problem, we analyze the analogous problem of constructing useful priors on compositions of functions. Specifically, we study the deep Gaussian process, a type of infinitely…
AI agents improve forecast combination but require transparency.
problem AI coding agents increase flexibility in empirical economics, leading to hidden degrees of freedom.
method Adapted open-source agent-loop architecture to empirical economics workflow, adding post-search holdout evaluation.
result Multiple agent runs outperform standard benchmarks in rolling evaluation but not all on post-search holdout.
The runtime for Kernel Partial Least Squares (KPLS) to compute the fit is quadratic in the number of examples. However, the necessity of obtaining sensitivity measures as degrees of freedom for model selection or confidence intervals for more detailed analysis requires cubic runtime, and thus constitutes a computationa…
It is shown that the equation which describes constant mean curvature surface via the generalized Weierstrass-Enneper inducing has Hamiltonian form. Its simplest finite-dimensional reduction has two degrees of freedom, integrable and its trajectories correspond to well-known Delaunay and do Carmo-Dajzcer surfaces (i.e.…
Derives log-corrections in AdS4/CFT3 using supergravity localization.
problem Factorizing log-corrections in AdS4/CFT3.
method Supergravity localization, Atiyah-Singer index theorem, fixed points (NUTs), fixed two-manifolds (Bolts).
result General fixed-point formula for log-corrections in large N expansion.
Simple model finds high correlation in retail crypto returns.
problem Discerning correlation in retail cryptocurrency markets without factors.
method Used N*(N) statistic to compare models of daily returns.
result High average pairwise correlation (60%) found, supports isotropic model.
Novel digital twin for complex systems improves performance.
problem Lack of practical implementation details for stochastic nonlinear MDOF systems.
method Decouples time-scales, uses physics-based model, Bayesian filtering, and machine learning.
result Excellent performance of proposed digital twin framework validated by examples.
Estimator of Prediction Error Based on Approximate Message Passing for Penalized Linear Regressionstat.ML
New estimator for prediction error using AMP algorithm for sparse penalties.
problem Estimating prediction error for penalized linear regression models.
method Approximate message passing (AMP) algorithm for calculating generalized degrees of freedom.
result Asymptotically unbiased estimator for Gaussian distributed predictors.
Complex geometry and symplectic geometry are mirrors in string theory. The recently developed generalised complex geometry interpolates between the two of them. On the other hand, the classical and quantum mechanics of a finite number of degrees of freedom are respectively described by a symplectic structure and a comp…
We investigate local configuration controllability for mechanical control systems within the affine connection formalism. Extending the work by Lewis for the single-input case, we are able to characterize local configuration controllability for systems with degrees of freedom and input forces.
An adaptive ensemble filter for heavy-tailed distributions: tuning-free inflation and localizationstat.CO
A new filter adapts to heavy-tailed data without tuning, improving performance in challenging conditions.
problem Degraded performance of Kalman and EnKF in heavy-tailed distributions.
method Generalizes EnKF using t-distributions, estimating parameters via EM algorithm.
result Improves performance on challenging filtering problems with heavy-tailed noise.