Paper investigates conditions for independence of weak gradients on metric spaces.
problem Dependence of weak gradients on p in arbitrary metric measure spaces. method Investigates the Bounded Interpolation Property to ensure independence of weak gradients.
result Bounded Interpolation Property guarantees independence of weak gradients.
The paper improves generalization bounds using interpolation between various divergences.
problem Improving generalization bounds in machine learning.
method Derives new PAC-Bayes generalization bounds based on (f,Γ)-divergence and interpolates between various divergences. result Connects derived bounds to earlier statistical learning results and provides practical training objectives.
Paper analyzes mistake and generalization of MNIC classifiers.
problem Understanding the performance of interpolating classifiers.
method Elementary analyses of MNIC's regret and generalization.
result MNIC generalizes with a rate proportional to the norm of the interpolating solution and inversely proportional to the number of data points.
NODEs with explicit time dependence can interpolate and generalize like piecewise-constant estimators.
problem Learning from finite datasets with neural ODEs.
method Control-theoretic perspective applied to semi-autonomous NODEs.
result SA-NODEs can interpolate and satisfy SCC, leading to generalization rates similar to histogram and nearest-neighbor estimators.
Study on learning properties of scale-dependent kernels controlling stability and error.
problem Understanding the learning properties of scale-dependent kernels in nonparametric ridge-less least squares.
method Combines probabilistic results with interpolation theory to analyze stability and error.
result Different regimes of learning error depending on sample size and data dimension.
This note is the updated outline of the article "Interpolational properties of planar spiral curves", Fund. and Applied Math., 2001, Vol.7, N.2, 441-463, published in Russian. The main result establishes boundary regions for spiral and piecewise spiral splines, matching given data. The width of such region can serve as…
Uniform convergence of interpolators proven for Gaussian data.
problem Interpolation learning in high-dimensional linear regression with Gaussian data.
method Generic uniform convergence guarantee in terms of Gaussian width.
result Consistency of interpolators for minimum-norm and near-minimal-norm cases.
Kernel interpolation is inconsistent for norms with smoothness above a constant.
problem Inconsistency of kernel interpolation in reproducing kernel Hilbert spaces.
method Lower bounds for generalization error in Sobolev norms.
result Kernel interpolation is always inconsistent for norms with smoothness above a constant.
In the absence of explicit regularization, Kernel "Ridgeless" Regression with nonlinear kernels has the potential to fit the training data perfectly. It has been observed empirically, however, that such interpolated solutions can still generalize well on test data. We isolate a phenomenon of implicit regularization for…
Interpolation can prevent classifiers from having desired invariance properties.
problem Invariance properties in over-parameterized models are often ineffective.
method Theoretical analysis and algorithm design for non-interpolating classifiers.
result Interpolating classifiers cannot satisfy desired invariance properties.
Study shows interpolating predictor's risk is optimal in low-dimensional factor regression models.
problem Understanding the risk of interpolating predictors in high-dimensional factor regression models.
method Detailed finite-sample analysis of minimum-norm interpolating predictor's risk in factor regression models.
result The risk of the minimum-norm interpolating predictor approaches optimal benchmarks in low-dimensional factor regression models.
Study tightens bounds for interpolating noisy data using minimum l1-norm.
problem Predicting noisy data with minimum l1-norm interpolation.
method Provided matching upper and lower bounds for prediction error.
result Tight consistency up to negligible terms for d≫n. New algorithm interpolates data with neural nets, independent of sample size.
problem Understanding neural networks' ability to memorize training data.
method Randomized algorithm for constructing interpolating neural networks.
result Guarantees that are independent of the number of samples, moving beyond worst-case memorization capacity bounds.
Proves a new law of robustness for interpolating arbitrary data distributions.
problem Understanding robust interpolation for arbitrary data distributions.
method Proves a Lipschitzness lower bound for robust interpolation.
result Demonstrates a two-fold law of robustness for interpolating functions.
New approach uses interpolation models and error bounds for verifiable scientific machine learning.
problem Challenges in verifying and validating modern scientific machine learning workflows.
method Combines multiple standard interpolation techniques with error bounds for efficient computation and comparative performance analysis.
result Error bounds for interpolation techniques can be computed or estimated efficiently, aiding in validation goals.
Loose bounds found for least-norm interpolant in over-parameterized settings.
problem Failures of model-dependent generalization bounds for least-norm interpolation.
method Analysis of generalization performance of least-norm linear regressor in over-parameterized regime.
result Generalization bounds for least-norm interpolant can be very loose, even when true excess risk goes to zero.
This paper introduces a new bound to explain generalization in over-parameterized models.
problem Understanding why some over-parameterized models generalize well while others do not.
method PAC-Chernoff bounds and smoothness measures based on large deviation theory.
result Interpolators with smoother structures generalize better, according to the new theoretical framework.
Monotonic Linear Interpolation property in neural networks persists despite non-convexity.
problem Understanding the geometric properties of neural network loss landscapes.
method Tools from differential geometry to analyze the monotonicity of neural network weights.
result Sufficient conditions for the Monotonic Linear Interpolation property under mean squared error.
Deep linear networks can closely approximate interpolants without improving risk.
problem Understanding the risk bounds of deep linear networks compared to minimum ℓ2-norm solutions. method Bounding excess risk of interpolating deep linear networks trained using gradient flow.
result Deep linear networks can closely approximate or match minimum ℓ2-norm solutions in terms of risk. Gradient descent on shallow neural networks achieves near-optimal generalization error.
problem Optimizing shallow neural networks with minimal width for generalization and stability.
method Gradient descent in the interpolating regime with minimal width.
result Gradient descent achieves near-optimal generalization error with minimal width.
We study the risk of minimum-norm interpolants of data in Reproducing Kernel Hilbert Spaces. Our upper bounds on the risk are of a multiple-descent shape for the various scalings of d=nα, α∈(0,1), for the input dimension d and sample size n. Empirical evidence supports our finding that minimum-norm interpo…
Study shows gap between uniform convergence and test error in random feature models.
problem Understanding the gap between uniform convergence and test error in random feature models.
method Analytical expressions for uniform convergence over norm balls, interpolators, and minimum norm interpolator risk derived and proved.
result Uniform convergence over interpolators still gives a non-trivial bound of test error even when classical uniform convergence is vacuous.
DALMC provides non-asymptotic error bounds for generative models.
problem Efficiently generating samples from complex data distributions.
method Analysis of diffusion paths and Langevin Monte Carlo.
result Theoretical guarantees for a class of generative models.
Efficiently computes quasiconcave envelope with limited data.
problem Approximating unknown quasiconcave function with partial information.
method Solves value problem and interpolation problem with polynomial and logarithmic LPs.
result Efficiently computes quasiconcave envelope with limited data.
Bagging stabilizes linear interpolators, improving their generalization performance.
problem Unstable linear interpolators fail on noisy data.
method Introduced multiplier-bootstrap-based bagged least square estimator.
result Bagging effectively mitigates variance, leading to bounded prediction risk.
Recurrent tasks such as pricing, calibration and risk assessment need to be executed accurately and in real-time. Simultaneously we observe an increase in model sophistication on the one hand and growing demands on the quality of risk management on the other. To address the resulting computational challenges, it is nat…
We construct geodesics in the Wasserstein space of probability measure along which all the measures have an upper bound on their density that is determined by the densities of the endpoints of the geodesic. Using these geodesics we show that a local Poincaré inequality and the measure contraction property follow from t…
Manifold learning has been successfully applied to a variety of medical imaging problems. Its use in real-time applications requires fast projection onto the low-dimensional space. To this end, out-of-sample extensions are applied by constructing an interpolation function that maps from the input space to the low-dimen…
Paper analyzes Gibbs and Langevin Monte Carlo for interpolation regime, showing generalization from low errors.
problem Analyzing Gibbs and Langevin Monte Carlo in overparameterized interpolation regime.
method Data-dependent bounds and stability under approximation with Langevin Monte Carlo.
result Generalization is signaled by small training errors in noisy regime, with bounds stable under approximation.
Deep ReLU networks need Ω(N) parameters to interpolate at irregularly spaced points.
problem Interpolating at irregularly spaced data points with deep ReLU networks.
method Analyzing the number of parameters required for interpolation.
result Ω(N) parameters are necessary for interpolation when δ is exponentially small in N.
Lower bounds show OLS outperforms basis pursuit in overparameterized linear regression.
problem Excess risk of sparse interpolating procedures in overparameterized linear regression.
method Proved lower bounds on excess risk for OLS and basis pursuit.
result Excess risk of basis pursuit can converge at an exponentially slower rate than OLS.
Private optimization faster on interpolation problems with quadratic growth.
problem Private optimization in interpolation problems.
method Adaptive algorithm with improved sample complexity.
result Exponential improvement in private sample complexity for quadratic growth.
IIC provides a PAC-Bayes bound for interpolating models, revealing factors affecting generalization.
problem Theoretical challenges in understanding overparameterized models and their performance.
method PAC-Bayesian perspective applied to the Interpolating Information Criterion (IIC).
result Test error for overparameterized models achieving zero training error depends on various factors.
Kernel interpolation improved with continuous volume sampling.
problem Approximating functions from RKHS using weighted sums of kernel translates.
method Continuous volume sampling for choosing node locations.
result Proved almost optimal bounds for interpolation and quadrature under VS.
The paper analyzes the robustness of a minimum ℓ2 interpolator in high-dimensional linear regression.
problem Analyzing the robustness of a minimum ℓ2 interpolator in high-dimensional linear regression. method The paper analyzes the interpolator with minimal ℓ2-norm in a general high-dimensional linear regression framework, proving bounds on prediction loss. result The paper shows that the prediction loss of the interpolator is bounded by (∥β∗∥22rcn(Σ)∨∥ξ∥2)/n with high probability, revealing a transition in rates. Gromov has shown how to construct holomorphic maps of the plane to a complex manifold with prescribed values on a lattice. In the present paper, a similar interpolation theorem for pseudo-holomorphic maps from the cylinder S to an almost-complex manifold (M,J) is proved. Properties of the space of pseudo-holomorphic ma…
CNFs learn distributions from samples with error bounds.
problem Learning probability distributions from finite samples.
method Continuous normalizing flows with linear interpolation and flow matching objective function.
result Non-asymptotic error bounds for distribution estimator in Wasserstein-2 distance.
New bounds for linear interpolators show how they generalize under covariate shifts.
problem Understanding how linear interpolators generalize under covariate shifts.
method Proved non-asymptotic excess risk bounds for benignly-overfit linear interpolators in transfer learning.
result Identified beneficial and malignant covariate shifts based on overparameterization degree.
Study on Gaussian interpolation flows for generative modeling.
problem Theoretical properties and regularizing effect of Gaussian denoising in continuous normalizing flows.
method Unified framework of Gaussian interpolation flow, Lipschitz regularity, existence and uniqueness of flow, stability analysis.
result Established theoretical properties of Gaussian interpolation flows, including Lipschitz continuity and existence of flow.
Defines CAMC discrete nets and their properties.
problem Understanding CAMC discrete nets and their properties.
method Defining CAMC discrete nets and proving properties.
result Properties of CAMC discrete nets are equivalent to properties of compatible interpolating quadrics.
The paper establishes bounds on the smoothness parameter in Gaussian process interpolation.
problem Estimating the smoothness parameter in Gaussian process models.
method Approximation theory in Sobolev spaces and general theorems on parameter estimation.
result Maximum likelihood estimation recovers the true smoothness for certain classes of functions.
Near-interpolating models grow norms quickly, affecting generalization.
problem Understanding the trade-off between interpolation and generalization in near-interpolating models.
method Random matrix theory and eigendecay analysis of data covariance matrix.
result Near-interpolating models exhibit rapid norm growth and worse generalization trade-offs.
The paper explores why a specific type of predictor works well in noisy data.
problem Understanding why a specific type of predictor (minimum-norm interpolator) works well in noisy data.
method The paper uses uniform convergence and zero-error predictors in a norm ball to explain the success of the minimum-norm interpolator.
result The minimum-norm interpolator is consistent, and this can be explained by uniform convergence of zero-error predictors in a norm ball.
We prove comparison theorems for the sub-Riemannian distortion coefficients appearing in interpolation inequalities. These results, which are equivalent to a sub-Laplacian comparison theorem for the sub-Riemannian distance, are obtained by introducing a suitable notion of sub-Riemannian Bakry-Émery curvature. The model…
This note is devoted to optimal spectral estimates for Schrödinger operators on compact connected Riemannian manifolds without boundary. These estimates are based on the use of appropriate interpolation inequalities and on some recent rigidity results for nonlinear elliptic equations on those manifolds.
Study on RF regression with SGD shows double descent phenomenon.
problem Understanding generalization in RF models trained with SGD.
method Precise non-asymptotic error bounds derived for RF regression under constant and polynomial-decay step-size SGD.
result RF regression generalizes well for interpolation learning and exhibits double descent behavior.
Strong inductive biases prevent harmless interpolation in overparameterized models.
problem Understanding the conditions under which overparameterized models can interpolate noise without overfitting.
method Theoretical analysis of high-dimensional kernel regression and deep neural networks, focusing on the role of inductive biases.
result The strength of an estimator's inductive bias determines whether interpolation is harmless or requires fitting noise for good generalization.
Gradient descent with logistic loss can interpolate deep networks with smoothed ReLU activations under certain conditions.
problem Conditions for gradient descent to drive logistic loss to zero in deep networks with smoothed ReLU activations.
method Gradient descent applied to fixed-width deep networks with smoothed ReLU approximations (e.g., Swish, Huberized ReLU).
result Gradient descent can drive logistic loss to zero under specific conditions, providing bounds on convergence rate.