Deep neural networks can interpolate any dataset in the overparametrized regime.
problem Interpolating any dataset with deep neural networks in the overparametrized regime.
method Proving universal approximations and interpolating any dataset with deep neural networks, considering specific conditions on activation functions.
result Interpolation of any dataset is possible in the overparametrized regime with deep neural networks.
Piecewise polynomial interpolation-based gradient descent reduces oracle complexity for smooth loss functions.
problem Optimizing empirical risk minimization loss functions
method Piecewise polynomial interpolation-based gradient descent
result Oracle complexity is reduced for smooth loss functions
Proves a new law of robustness for interpolating arbitrary data distributions.
problem Understanding robust interpolation for arbitrary data distributions.
method Proves a Lipschitzness lower bound for robust interpolation.
result Demonstrates a two-fold law of robustness for interpolating functions.
Motivated from the action functional for bosonic strings with extrinsic curvature term we introduce an action functional for maps between Riemannian manifolds that interpolates between the actions for harmonic and biharmonic maps. Critical points of this functional will be called interpolating sesqui-harmonic maps. In …
Develops interpolation methods for matrix functions in statistics and machine learning.
problem Estimating matrix functions in statistics and machine learning.
method Interpolates log-determinant and trace of matrix powers using modified sharp bounds.
result Accuracy and performance demonstrated in numerical examples.
In this article we study various analytic aspects of interpolating sesqui-harmonic maps between Riemannian manifolds where we mostly focus on the case of a spherical target. The latter are critical points of an energy functional that interpolates between the functionals for harmonic and biharmonic maps. In the case of …
A physics-based method improves data interpolators and regression tasks.
problem Improving accuracy and efficiency in function learning.
method Inspired by statistical mechanics, introduces corrections to minimize energy.
result Improves performance in interpolation and regression tasks, especially in high-dimensional spaces.
We show that stochastic interpolation flow maps are Lipschitz with a sharp constant.
problem High dimensional sampling and transport problems.
method Investigating stochastic interpolation flow for generating data samples.
result Stochastic interpolation flow maps are Lipschitz with a sharp constant matching optimal transport maps.
Estimates box dimension of fractal interpolation surfaces using oscillation vectors.
problem Estimating the complexity of fractal interpolation surfaces.
method Defined vertical scaling matrices and used them to relate oscillation vectors of different levels.
result Obtained the box dimension of generalized affine fractal interpolation surfaces.
This work bridges stochastic interpolants to infinite-dimensional Hilbert spaces.
problem Limited flexibility in generating arbitrary distributions for function-valued data.
method Establishes a rigorous framework for stochastic interpolants in infinite-dimensional Hilbert spaces.
result Achieves state-of-the-art results in conditional generation for complex PDE-based benchmarks.
The study identifies conditions under which algorithmic stability explains generalization in interpolating learning systems.
problem Understanding when algorithmic stability explains generalization in interpolating learning systems.
method Modeling training as a function-space trajectory and measuring sensitivity to single-sample perturbations.
result There exist interpolating regimes with small risk where contractive sensitivity cannot hold, showing that stability is not a universal explanation.
The paper extends a variance gamma model to quadratic functions, reducing arbitrage and computational costs.
problem Creating an arbitrage-free interpolation for option pricing models.
method Generalizing the local variance gamma model to a piecewise quadratic local variance function.
result The quadratic model results in an arbitrage-free interpolation of class C3, reducing knots and computational cost.
We replace the output layer of deep neural nets, typically the softmax function, by a novel interpolating function. And we propose end-to-end training and testing algorithms for this new architecture. Compared to classical neural nets with softmax function as output activation, the surrogate with interpolating function…
A common approach for defining a reward function for Multi-objective Reinforcement Learning (MORL) problems is the weighted sum of the multiple objectives. The weights are then treated as design parameters dependent on the expertise (and preference) of the person performing the learning, with the typical result that a …
Ridgeless ReLU networks interpolate datasets and extrapolate based on curvature signs.
problem Interpolating and extrapolating 1D datasets with ReLU networks.
method Minimizes ℓ2-norm of weights, extrapolates based on curvature signs. result Ridgeless ReLU interpolants extrapolate as nearest neighbor curvature extrapolation.
Two methods for interpolating manifold-valued data are presented.
problem Interpolating manifold-valued functions with derivative constraints.
method Two approaches: weighted Riemannian barycenters and tangent space interpolation.
result Both methods are valid and perform well with numerical examples.
Study on biharmonic and interpolating sesqui-harmonic vector fields on para-Kähler--Norden manifolds.
problem Investigating higher-order harmonicity in pseudo-Riemannian geometry.
method Deriving first variations of bienergy and interpolating sesqui-energy functionals, characterizing biharmonic and interpolating sesqui-harmonic vector fields.
result Explicit characterizations and examples of vector fields satisfying biharmonic and interpolating sesqui-harmonic conditions.
We introduce a new wavelet transform suitable for analyzing functions on point clouds and graphs. Our construction is based on a generalization of the average interpolating refinement scheme of Donoho. The most important ingredient of the original scheme that needs to be altered is the choice of the interpolant. Here, …
A new GP interpolation method for better predictive distributions in ranges of interest.
problem Improving predictive distributions in specific ranges of interest.
method Relaxed Gaussian process interpolation, relaxing interpolation constraints outside ranges of interest.
result Better predictive distributions in ranges of interest, especially in non-stationary cases.
The paper studies optimal transport in linear quadratic systems and derives interpolation inequalities.
problem Optimal transport problem in Linear Quadratic optimal control systems.
method Well-posedness of the Monge problem, regularity of optimal transport map, displacement interpolation of measures.
result Derivation of general interpolation inequalities for entropy functionals.
Stochastic second-order methods converge fast under interpolation conditions.
problem Minimizing smooth and strongly-convex functions efficiently.
method Regularized subsampled Newton method (R-SSN) and stochastic BFGS algorithms.
result R-SSN achieves global linear convergence and quadratic rate in a local neighbourhood.
Monotone neural networks can approximate and interpolate functions efficiently.
problem Understanding the efficiency and expressiveness of monotone neural networks.
method Solving the monotone interpolation problem using depth-4 networks and comparing size bounds with arbitrary networks.
result Monotone neural networks can approximate and interpolate functions efficiently, but may require exponential size in high dimensions.
GNNs outperform NNs in interpolating bandlimited functions on Euclidean cubes.
problem Interpolating bandlimited functions on Euclidean cubes using GNNs vs. NNs.
method Investigates optimal GNN configurations and weights for function interpolation.
result GNNs require fewer weights and samples to interpolate bandlimited functions compared to NNs.
Deep networks can interpolate noisy data without losing generalization.
problem Characterizing the relationship between interpolation and generalization in overparameterized deep networks.
method Analyzing the loss landscape of neural network functions over volumes around training data points, varying model parameters and training epochs.
result Loss sharpness in the input space follows a double descent, with large models predicting noisy targets over larger volumes around training data points.
Diffusion models improve creativity by smoothing the score function, leading to interpolated data.
problem Improving creativity in diffusion models.
method Analyzing the effect of score smoothing on diffusion model dynamics.
result Score smoothing causes diffusion models to generate data that interpolate the training set.
Efficiently prices American options with multiple assets using sparse grids.
problem Pricing American options with multiple underlying assets efficiently.
method Dynamic programming formulation followed by sparse grid interpolation.
result Sparse grids reduce the number of interpolation points and maintain function smoothness.
Paper investigates optimal interpolation methods in linear regression.
problem Understanding when interpolating methods generalize well in linear regression.
method Investigates optimal response-linear interpolators using functions linear in the response variable.
result Provides a closed-form expression for the optimal interpolator and shows it can be derived as the limit of gradient descent.
Interpolates curves using maximal and minimal surfaces in different spaces.
problem Interpolating curves in spacelike and Euclidean spaces.
method Using maximal and minimal surfaces, interpolates curves based on the Björling problem.
result Constructs surfaces to interpolate curves in a specified manner.
A new training method for normalizing flows without samples.
problem Training normalizing flows without samples but with energy functions.
method Interpolates energy functions to find a transport vector field.
result Optimizes transport vector field and energy function to satisfy continuity equation.
Private optimization faster on interpolation problems with quadratic growth.
problem Private optimization in interpolation problems.
method Adaptive algorithm with improved sample complexity.
result Exponential improvement in private sample complexity for quadratic growth.
SKI accelerates GP inference with sparse grids to handle higher dimensions.
problem SKI scales poorly in high dimensions due to dense grid size.
method Sparse grids within SKI framework, novel matrix-vector multiplication algorithm.
result SKI can be scaled to higher dimensions while maintaining accuracy.
This paper presents an efficient algorithm for evolving point cloud data on smooth manifolds using B-Splines.
problem Evolution of point cloud data on smooth manifolds in higher dimensions.
method Lagrangian approach using adaptive B-Spline interpolation.
result Demonstrates the convergence of geometric quantities and the effectiveness of the approach.
Modern supervised learning techniques, particularly those using deep nets, involve fitting high dimensional labelled data sets with functions containing very large numbers of parameters. Much of this work is empirical. Interesting phenomena have been observed that require theoretical explanations; however the non-conve…
A method to construct fractal surfaces by recurrent fractal curves is provided. First we construct fractal interpolation curves using a recurrent iterated functions system(RIFS) with function scaling factors and estimate their box-counting dimension. Then we present a method of construction of wider class of fractal su…
New law explains why deep learning models often have more parameters than needed.
problem Why deep learning models often have more parameters than classical theory suggests.
method Proved a universal law of robustness for smooth interpolation.
result Smooth interpolation requires d times more parameters than mere interpolation.
Paper develops SINNOs for approximating stochastic processes.
problem Approximating stochastic processes with neural networks.
method Developed stochastic interpolation neural network operators (SINNOs) with random coefficients.
result Established boundedness, interpolation accuracy, and approximation capabilities of SINNOs.
Geometric theory connects machine learning classifiers to differential geometry.
problem Classifying data points in machine learning.
method Mapping binary classification to vector bundles and differential geometry.
result Harmonic interpolation solves RKHS interpolation problems.
We present a neural network architecture based upon the Autoencoder (AE) and Generative Adversarial Network (GAN) that promotes a convex latent distribution by training adversarially on latent space interpolations. By using an AE as both the generator and discriminator of a GAN, we pass a pixel-wise error function acro…
Recent works have shown that stochastic gradient descent (SGD) achieves the fast convergence rates of full-batch gradient descent for over-parameterized models satisfying certain interpolation conditions. However, the step-size used in these works depends on unknown quantities and SGD's practical performance heavily re…
Paper analyzes mistake and generalization of MNIC classifiers.
problem Understanding the performance of interpolating classifiers.
method Elementary analyses of MNIC's regret and generalization.
result MNIC generalizes with a rate proportional to the norm of the interpolating solution and inversely proportional to the number of data points.
The study examines deep convolutional neural networks and their learning ability.
problem Understanding the learning ability of deep convolutional neural networks (DCNNs).
method Examines DCNNs under both underparameterized and overparameterized settings, using a novel network deepening scheme.
result Establishes the first learning rates of underparameterized DCNNs and shows how adding layers can create interpolating DCNNs with good learning rates.
Managing and hedging the risks associated with Variable Annuity (VA) products require intraday valuation of key risk metrics for these products. The complex structure of VA products and computational complexity of their accurate evaluation have compelled insurance companies to adopt Monte Carlo (MC) simulations to valu…
This paper introduces a new bound to explain generalization in over-parameterized models.
problem Understanding why some over-parameterized models generalize well while others do not.
method PAC-Chernoff bounds and smoothness measures based on large deviation theory.
result Interpolators with smoother structures generalize better, according to the new theoretical framework.
Deep neural networks can approximate rough functions with high accuracy.
problem Approximating rough functions with neural networks.
method Proved that ENO interpolation can be cast as a deep ReLU neural network, transferring ENO's high-order accuracy.
result Deep neural networks can achieve high-order accuracy in approximating Lipschitz functions.
Regression models can interpolate noisy data and still perform well, contrary to the bias-variance tradeoff.
problem Understanding why overparametrized models can generalize well despite the bias-variance tradeoff.
method Analysis of minimum norm solutions and ridge regression, focusing on the smallest singular value of the regression matrix.
result Testing error exhibits double descent behavior as model order increases, contrary to the classical bias-variance tradeoff.
The paper proposes a new method for density estimation using spline quasi-interpolation for clustering.
problem Density estimation and clustering modeling for multivariate data.
method Spline quasi-interpolation for mono-variate approximation, copulas for multivariate modeling.
result The proposed method achieves accurate clustering of data using copulas and spline quasi-interpolation.
We consider the class of affine LIBOR models with multiple curves, which is an analytically tractable class of discrete tenor models that easily accommodates positive or negative interest rates and positive spreads. By introducing an interpolating function, we extend the affine LIBOR models to a continuous tenor and de…
This paper introduces an interpolation-based method, called the reconstruction approach, for nonparametric regression. Based on the fact that interpolation usually has negligible errors compared to statistical estimation, the reconstruction approach uses an interpolator to parameterize the regression function with its …