Study proposes local effective dimension to measure model capacity and generalization error.
problem Capturing the generalization power of machine learning models.
method Proposes local effective dimension as a capacity measure.
result Local effective dimension bounds the generalization error and correlates well with it.
Model complexity is an important factor to consider when selecting among graphical models. When all variables are observed, the complexity of a model can be measured by its standard dimension, i.e. the number of independent parameters. When hidden variables are present, however, standard dimension might no longer be ap…
Generalization in nonlinear least squares can be studied via algorithmic stability and effective dimension.
problem Generalization in nonlinear least squares models
method Deriving error bounds for local minimizers using algorithmic stability and effective dimension
result Bounds depend on learned geometry rather than parameter count
SGD generalizes well in high dimensions without regularization.
problem Generalization of overparameterized models in high dimensions.
method Stochastic Gradient Descent (SGD) for convex and locally convex loss functions.
result Generalization error is independent of the ambient dimension p under certain conditions. NGD models have higher effective dimension than SGD models.
problem Measuring model complexity accurately.
method Comparison of NGD and SGD models using effective dimension measures.
result NGD models have a higher effective dimension than SGD models.
Paper analyzes ensemble Kalman updates for effective dimension and localization.
problem Why small ensemble sizes work well in inverse problems and data assimilation.
method Non-asymptotic analysis of ensemble Kalman updates, focusing on effective dimension and localization.
result Rigorously explains why a small ensemble size is sufficient when prior covariance has moderate effective dimension.
Paper improves learning efficiency by focusing on effective dimensionality.
problem Dimensionality bottleneck in modern learning tasks.
method Developed tools to reduce dimensional costs using effective dimensionality.
result Uniform concentration bounds involving effective dimensionality, improving over existing results.
New stability analysis improves generalization of multipass SGD.
problem Improper preconditioning affects generalization in multipass SGD.
method Developed on-average stability analysis for multipass SGD.
result Proper preconditioning yields optimal effective dimension dependence.
A faster method for estimating effects in large data using fixed-point trees.
problem Estimating heterogeneous effects in large dimensions with computational efficiency.
method Fixed-point approximation to eliminate Jacobian estimation and speed up GRFs.
result Significant computational efficiency improvement without sacrificing statistical accuracy.
Estimates mean dimension of neural networks to reveal interaction effects.
problem Understanding interaction effects in neural networks.
method Estimation procedure for mean dimension from datasets, analyzing layer-by-layer evolution and impact of activation functions.
result Mean dimension reveals differences in interaction magnitude across neural network architectures.
SignSGD analysis quantifies its effects in high dimensions.
problem Understanding signSGD's effects in high-dimensional settings.
method High-dimensional analysis of signSGD, deriving SDE and ODE for risk.
result Quantification of signSGD's effects: effective learning rate, noise compression, diagonal preconditioning, gradient noise reshaping.
The paper improves confidence set construction for statistical inference.
problem Constructing reliable confidence sets in statistical inference.
method Establishes a finite-sample bound using effective dimension and generalized self-concordance.
result Developed a confidence set adapted to optimization landscapes.
A new measure of model complexity based on Fisher Information.
problem Model complexity measurement in statistical models.
method Effective dimension defined by the number of cubes needed to cover the model space.
result The effective dimension is scale-dependent and measures model complexity.
Study on VC dimension of GCNNs with input resolution effects.
problem Understanding the generalization capabilities of GCNNs.
method Derived upper and lower bounds for VC dimension, analyzed factors affecting it.
result Extended previous results on VC dimension of GCNNs, providing insights into input resolution dependence.
Kernel balancing weights are generalized as KRRR, providing better confidence intervals for treatment effects.
problem Lack of generalization error, correct feature specification, and limited to average effects.
method Interpreting kernel balancing weights as KRRR, relaxing feature specification, and extending Gaussian approximation.
result KRRR provides strong generalization properties and justifies confidence sets for causal functions.
New method corrects Laplace/BIC errors in singular models, revealing effective dimension.
problem Laplace/BIC errors in singular models due to incorrect effective dimension assumption.
method RLCT (real log canonical threshold) to correct effective dimension in linear models.
result Correct evidence slope and effective dimension estimation in linear settings.
Study bandit problems under censorship, estimating performance loss.
problem Estimating performance loss in bandit problems with censored feedback.
method Introduced a broad class of censorship models and analyzed their effective dimension.
result Effective dimension naturally leads to results analogous to uncensored settings.
Study shows how to effectively predict functions on manifolds using kernel methods.
problem Regression on manifolds with limited data.
method Reproducing kernel Hilbert space methods, Weyl law, effective dimension.
result Kernel regression estimator yields minimax-optimal error bounds controlled by effective dimension.
New algorithm reduces sketching dimension to effective problem size.
problem Solving L2-regularized least-squares problems efficiently.
method Randomized algorithm using Gaussian and SRHT embeddings.
result Preserves convergence guarantees with reduced embedding dimension.
Sliced inverse regression (SIR) is a pioneer tool for supervised dimension reduction. It identifies the effective dimension reduction space, the subspace of significant factors with intrinsic lower dimensionality. In this paper, we propose to refine the SIR algorithm through an overlapping slicing scheme. The new algor…
Generative model handles varying data dimensions using jump diffusion processes.
problem Handling data of varying dimensionality in generative models.
method Formulated as a jump diffusion process, learning to approximate the process with a novel evidence lower bound.
result Effective sampling of data of varying dimensionality, better compatibility with test-time diffusion guidance imputation tasks.
This work improves understanding of reinforcement learning state representations.
problem Lack of precise characterization of how and when state representations generalize.
method Developed a bound on the generalization error based on effective dimension.
result Bound quantifies the tension between generalization and approximation.
Augmented KRnet improves flow-based generative modeling by maintaining exact invertibility.
problem Maintaining exact invertibility in flow-based generative models.
method Integrates augmented dimensions into KRnet to achieve full nonlinear updates in two iterations, keeping exact invertibility.
result Augmented KRnet achieves full nonlinear updates in two iterations, maintaining exact invertibility.
A mathematical model describes deforming manifolds with precise vectors and fields.
problem Modeling and describing the deformation of complex manifolds in practical applications.
method Proposes a modified differential dynamic model with constraints on spatial and temporal continuity, presenting deforming vector and field.
result Demonstrates the effectiveness of an autonomous deforming field in data dimension reduction tasks.
In this text we give a decomposition result on polynomial poly-vector fields generalizing a result on the decomposition of homogeneous Poisson structures. We discuss consequences of this decomposition result in particular for low dimensions and low degrees. We provide the tools to calculate simple cubic Poisson structu…
We obtain an Einstein metric of constant negative curvature given an arbitrary boundary metric in three dimensions, and a conformally flat one given an arbitrary conformally flat boundary metric in other dimensions. In order to compute the on-shell value of the gravitational action for these solutions, we propose to in…
The correspondence between Riemann-Finsler geometries and effective field theories with spin-independent Lorentz violation is explored. We obtain the general quadratic action for effective scalar field theories in any spacetime dimension with Lorentz-violating operators of arbitrary mass dimension. Classical relativist…
New theory shows deep networks adapt to data's intrinsic dimensionality even when data isn't on a low-dimensional manifold.
problem Existing theories on deep nonparametric regression assume data lie on a low-dimensional manifold, which is often not the case in real-world applications.
method Introduces effective Minkowski dimension to characterize the intrinsic dimension of data subsets and proves sample complexity depends on this new complexity notation.
result Deep neural networks can adapt to the effective Minkowski dimension of data, circumventing the curse of dimensionality for moderate sample sizes.
A new measure predicts deep learning model performance.
problem Predicting the generalization error of deep learning models.
method 2sED measure based on effective dimension, layerwise iterative approximation.
result 2sED correlates well with training error and generalization error.
Study extends compactness theorems to weighted manifolds with integral curvature bounds.
problem Estimating diameter of weighted manifolds under curvature constraints.
method Extended Sprouse's compactness theorems to weighted manifolds with integral curvature bounds. Used ε-range to handle specific cases. Extended segment inequality to weighted manifolds.
result Proved theorems for weighted manifolds with effective dimension ≤ 1 and ≥ dimension.
This paper evaluates fractal dimension and persistent homology for neural network generalization.
problem Bounding and predicting the generalization gap of neural networks.
method Empirical evaluation of fractal dimension and persistent homology as generalization measures.
result Fractal dimension and persistent homology fail to predict generalization of models trained from poor initializations.
Deconfounding scores improve causal effect estimation with weak overlap.
problem Challenges in causal treatment effect estimation due to weak overlap in high-dimensional data.
method Propose deconfounding scores to preserve identification and target estimation while improving overlap.
result Prognostic scores are overlap-optimal under a broad family of generalized linear models with Gaussian features.
Characterizes learnability of forgiving 0-1 loss functions in multiclass settings.
problem Understanding when multiclass learning with forgiving 0-1 loss functions is possible.
method Introduces a new combinatorial dimension based on Natarajan Dimension to determine learnability.
result A hypothesis class is learnable if and only if the Generalized Natarajan Dimension is finite.
The versatility of exponential families, along with their attendant convexity properties, make them a popular and effective statistical model. A central issue is learning these models in high-dimensions, such as when there is some sparsity pattern of the optimal parameter. This work characterizes a certain strong conve…
In this paper several examples of gaps (lacunes) between dimensions of maximal and submaximal symmetric models are considered, which include investigation of number of independent linear and quadratic integrals of metrics and counting the symmetries of geometric structures and differential equations. A general result c…
Proposes a deep learning method for effective data representation.
problem Constructing effective data representations for prediction.
method A deep dimension reduction approach to learning representations with sufficiency, low dimensionality, and disentanglement.
result The proposed deep nonparametric representation is consistent and performs better than existing methods.
Proposes a method for evaluating multiple dimensions of organizational effectiveness using DEA.
problem Evaluating multiple dimensions of organizational effectiveness in large data sets.
method Introduces two regularized DEA models (SBM and GP-SBM) to estimate both dimension-specific and aggregate efficiency scores.
result Demonstrates improved efficiency and validity compared to conventional methods.
Diffusion models achieve high-quality samples from complex high-dimensional Gaussian mixtures without scaling with dimension.
problem Achieving accurate sampling from high-dimensional distributions using diffusion models.
method Investigates the effectiveness of diffusion models in sampling from Gaussian Mixture Models (GMMs) without scaling with dimension.
result DDPM requires at most O(1/ε) iterations to attain an ε-accurate distribution in total variation distance, independent of dimension and number of components. Paper proposes adaptive parameter selection for KGD algorithms.
problem Improving parameter selection for kernel-based gradient descent.
method Integrates bias-variance analysis with splitting method, introduces empirical effective dimension.
result Adaptive parameter selection strategy achieves optimal generalization error bound.
We show, using a theorem of Milnor and Margulis, that string theory on compact negatively curved spaces grows new effective dimensions as the space shrinks, generalizing and contextualizing the results in hep-th/0510044. Milnor's theorem relates negative sectional curvature on a compact Riemannian manifold to exponenti…
We explicitly classify all pairs (M,G), where M is a connected complex manifold of dimension n≥2 and G is a connected Lie group acting properly and effectively on M by holomorphic transformations and having dimension dG satisfying n2+2≤dG<n2+2n. These results extend -- in the complex case -- the…
SCBMs model causal effects using low-dimensional bottlenecks.
problem Causal effect estimation in high-dimensional systems.
method Structural causal models with low-dimensional summary statistics.
result SCBMs provide a flexible framework for task-specific dimension reduction.
Estimates manifold dimension from random samples.
problem Estimating the dimension of a manifold from random samples.
method Explicit theoretical and heuristic bounds for data set size.
result Data set needs to be sufficiently large for accurate dimension estimation.
We prove the existence of Sasakian-Einstein metrics on infinitely many rational homology spheres in all odd dimensions greater than 3. In dimension 5 we obain somewhat sharper results. There are examples where the number of effective parameters in the Einstein metric grows exponentially with dimension.
New research shows that the dimension gap between intrinsic and ambient dimensions affects adversarial vulnerability of machine learning models.
problem The mystery of adversarial attacks on machine learning models.
method Introducing two types of adversarial attacks and proving their relationship to the dimension gap.
result The dimension gap between intrinsic and ambient dimensions makes clean-trained models more vulnerable to off-manifold adversarial perturbations.
Chart autoencoders learn latent features preserving manifold topology and geometry, with robust denoising capabilities.
problem Learning low-dimensional latent features of high-dimensional data sampled near a manifold.
method Chart autoencoders encode data into latent features on charts, preserving manifold topology and geometry.
result Chart autoencoders achieve a squared generalization error of n−d+22log4n under proper network architectures. Deep generative modeling using flows has gained popularity owing to the tractable exact log-likelihood estimation with efficient training and synthesis process. However, flow models suffer from the challenge of having high dimensional latent space, the same in dimension as the input space. An effective solution to the …
Paper introduces a new regularization method for kernel gradient descent learning.
problem Preventing overfitting in kernel gradient descent learning.
method Random smoothing regularization as novel convolution-based smoothing kernels.
result Optimal convergence rates achieved in various function spaces.