The paper improves interpolation in generative models by using specific base distributions.
problem Unexpected side effects in linear interpolations of normalizing flows.
method Enforces a specific manifold using Dirichlet and von Mises-Fisher base distributions.
result Superior performance in terms of bits per dimension, FID, and KID scores for interpolation.
Interpolating between points is a problem connected simultaneously with finding geodesics and study of generative models. In the case of geodesics, we search for the curves with the shortest length, while in the case of generative models we typically apply linear interpolation in the latent space. However, this interpo…
Deep networks can interpolate noisy data without losing generalization.
problem Characterizing the relationship between interpolation and generalization in overparameterized deep networks.
method Analyzing the loss landscape of neural network functions over volumes around training data points, varying model parameters and training epochs.
result Loss sharpness in the input space follows a double descent, with large models predicting noisy targets over larger volumes around training data points.
Study investigates overparametrization in survival models, revealing complex loss behavior.
problem Understanding overparametrization in survival models through interpolation.
method Defined interpolation and finite-norm interpolation, rigorously analyzed four survival models.
result Overparametrization can lead to improved performance in survival models, contrary to classical learning theory.
New optimization method helps models generalize better after achieving near-perfect training performance.
problem Models can achieve near-perfect training performance but fail to generalize well to unseen examples.
method GROKtimizer combines rapid convergence to interpolation with post-interpolation norm minimization using Critically Damped Momentum.
result GROKtimizer provides a quadratic speedup over classical gradient descent, offering a natural solution for selecting low-norm interpolating solutions.
Study interpolating estimators for causal learning from observational data.
problem Learning causal models from observational data in complex model classes.
method Investigate min-norm interpolators and ridge-regularized regressors in a linearly confounded model.
result Interpolators cannot be optimal for causal learning under the principle of independent causal mechanisms, requiring stronger regularization.
Near-interpolating models grow norms quickly, affecting generalization.
problem Understanding the trade-off between interpolation and generalization in near-interpolating models.
method Random matrix theory and eigendecay analysis of data covariance matrix.
result Near-interpolating models exhibit rapid norm growth and worse generalization trade-offs.
Study shows gap between uniform convergence and test error in random feature models.
problem Understanding the gap between uniform convergence and test error in random feature models.
method Analytical expressions for uniform convergence over norm balls, interpolators, and minimum norm interpolator risk derived and proved.
result Uniform convergence over interpolators still gives a non-trivial bound of test error even when classical uniform convergence is vacuous.
New approach uses interpolation models and error bounds for verifiable scientific machine learning.
problem Challenges in verifying and validating modern scientific machine learning workflows.
method Combines multiple standard interpolation techniques with error bounds for efficient computation and comparative performance analysis.
result Error bounds for interpolation techniques can be computed or estimated efficiently, aiding in validation goals.
Exploiting the deep generative model's remarkable ability of learning the data-manifold structure, some recent researches proposed a geometric data interpolation method based on the geodesic curves on the learned data-manifold. However, this interpolation method often gives poor results due to a topological difference …
The paper connects flatness to generalization in learning multi-index models with neural networks.
problem Understanding the generalization of non-convex neural networks using flatness measures.
method Analyzes 2-layer non-convex homogeneous neural networks and their connection to multi-index models.
result Flattest interpolators achieve small population loss and generalize well, establishing a direct link between flatness and generalization.
Strong inductive biases prevent harmless interpolation in overparameterized models.
problem Understanding the conditions under which overparameterized models can interpolate noise without overfitting.
method Theoretical analysis of high-dimensional kernel regression and deep neural networks, focusing on the role of inductive biases.
result The strength of an estimator's inductive bias determines whether interpolation is harmless or requires fitting noise for good generalization.
The over-parameterized models attract much attention in the era of data science and deep learning. It is empirically observed that although these models, e.g. deep neural networks, over-fit the training data, they can still achieve small testing error, and sometimes even {\em outperform} traditional algorithms which ar…
Gaussian Processes improve data interpolation from diverse experiments.
problem Interpolation of sparse and inconsistent datasets from various experiments.
method Used Gaussian Processes (GP) for data interpolation, including uncertainty quantification.
result GPs successfully interpolate data and quantify uncertainties, demonstrating consistency across different sources.
A new image interpolation model using sparse representation and nonlocal linear regression.
problem Image interpolation without blurring and noise.
method Sparse representation, nonlocal self-similarity, nonlocal linear regression, adaptive sub-dictionary learning, weighted encoding.
result Our method outperforms state-of-the-art methods in quantitative measures and visual quality.
Regression models can interpolate noisy data and still perform well, contrary to the bias-variance tradeoff.
problem Understanding why overparametrized models can generalize well despite the bias-variance tradeoff.
method Analysis of minimum norm solutions and ridge regression, focusing on the smallest singular value of the regression matrix.
result Testing error exhibits double descent behavior as model order increases, contrary to the classical bias-variance tradeoff.
A new tradeoff between regularization and sharpness improves model performance in overparameterized settings.
problem Improving model performance in overparameterized settings with minimum-norm interpolators.
method Proposes a regularization-sharpness tradeoff for overparameterized linear regression with an ℓ^p penalty.
result Empirical validation shows the tradeoff terms can distinguish performant linear interpolators.
Paper shows faster convergence to local-minimizers in over-parametrized models under interpolation-like conditions.
problem Escaping saddle-points in over-parametrized models.
method Stochastic and deterministic optimization algorithms under interpolation-like conditions.
result Oracle complexity of PSGD and SCRN algorithms to reach ε-local-minimizer matches or improves upon deterministic rates. Randomly sampled interpolators achieve zero generalization error with enough data.
problem Understanding the high generalization ability of machine learning models.
method Algebraic geometry tools to prove zero generalization error for random interpolators.
result Generalization error of randomly sampled interpolators becomes zero once the number of training samples exceeds a geometric threshold.
The study finds flaws in methods used to estimate foreign exchange option prices.
problem Flaws in estimating foreign exchange option prices.
method Provided counterexamples of popular FX option interpolation methods.
result Popular FX option interpolation methods fail in certain scenarios.
Combines spline interpolation and ARIMA for stock market forecasting.
problem Limited predictive performance of ARIMA in noisy data.
method Integrates cubic spline interpolation and ARIMA for time series forecasting.
result Demonstrates guidance for short-term stock market forecasting.
In this paper, we propose a method to learn a minimizing geodesic within a data manifold. Along the learned geodesic, our method can generate high-quality interpolations between two given data samples. Specifically, we use an autoencoder network to map data samples into latent space and perform interpolation via an int…
We introduce a new structured kernel interpolation (SKI) framework, which generalises and unifies inducing point methods for scalable Gaussian processes (GPs). SKI methods produce kernel approximations for fast computations through kernel interpolation. The SKI framework clarifies how the quality of an inducing point a…
This paper shows how forward rate interpolations are equivalent to discount factor interpolations in yield curve construction.
problem The challenge of choosing between different interpolation methods for yield curve construction.
method Demonstrates the equivalence between forward rate interpolations and discount factor interpolations.
result Some popular interpolation methods on forward rates are equivalent to classical interpolation methods on discount factors.
Paper investigates optimal interpolation methods in linear regression.
problem Understanding when interpolating methods generalize well in linear regression.
method Investigates optimal response-linear interpolators using functions linear in the response variable.
result Provides a closed-form expression for the optimal interpolator and shows it can be derived as the limit of gradient descent.
This paper introduces a new bound to explain generalization in over-parameterized models.
problem Understanding why some over-parameterized models generalize well while others do not.
method PAC-Chernoff bounds and smoothness measures based on large deviation theory.
result Interpolators with smoother structures generalize better, according to the new theoretical framework.
The paper studies the minimum ℓ₁-norm interpolator's risk behavior in over-parameterized settings.
problem Understanding the risk behavior of minimum ℓ₁-norm interpolators in high-dimensional settings.
method Exact characterization of the risk behavior through a system of two non-linear equations.
result Observation of a multi-descent phenomenon in the generalization risk of the minimum ℓ₁-norm interpolator.
IIC provides a PAC-Bayes bound for interpolating models, revealing factors affecting generalization.
problem Theoretical challenges in understanding overparameterized models and their performance.
method PAC-Bayesian perspective applied to the Interpolating Information Criterion (IIC).
result Test error for overparameterized models achieving zero training error depends on various factors.
In implicit models, one often interpolates between sampled points in latent space. As we show in this paper, care needs to be taken to match-up the distributional assumptions on code vectors with the geometry of the interpolating paths. Otherwise, typical assumptions about the quality and semantics of in-between points…
Interpolation can prevent classifiers from having desired invariance properties.
problem Invariance properties in over-parameterized models are often ineffective.
method Theoretical analysis and algorithm design for non-interpolating classifiers.
result Interpolating classifiers cannot satisfy desired invariance properties.
Noise affects the effectiveness of interpolating models, especially those with strong inductive biases.
problem The impact of noise on interpolating models with strong inductive biases.
method Analyzing linear and classification models with sparse ground truths, proving fast rates for interpolators.
result Strong inductive biases can lead to faster but noisier interpolators, contrary to intuition.
We show that stochastic interpolation flow maps are Lipschitz with a sharp constant.
problem High dimensional sampling and transport problems.
method Investigating stochastic interpolation flow for generating data samples.
result Stochastic interpolation flow maps are Lipschitz with a sharp constant matching optimal transport maps.
Enhances interpolation paths in latent space using particle filters.
problem Generating meaningful interpolations between data points in latent space.
method Introduces a discriminator network to guide particle filter sampling of interpolation paths.
result Improved variability and stronger drift towards high data density areas.
The paper studies optimal transport in linear quadratic systems and derives interpolation inequalities.
problem Optimal transport problem in Linear Quadratic optimal control systems.
method Well-posedness of the Monge problem, regularity of optimal transport map, displacement interpolation of measures.
result Derivation of general interpolation inequalities for entropy functionals.
The paper extends a variance gamma model to quadratic functions, reducing arbitrage and computational costs.
problem Creating an arbitrage-free interpolation for option pricing models.
method Generalizing the local variance gamma model to a piecewise quadratic local variance function.
result The quadratic model results in an arbitrage-free interpolation of class C3, reducing knots and computational cost.
This paper explores how deep learning models can fit data exactly and why this is important.
problem Understanding why deep learning models can fit data exactly and generalize well.
method Interpolation and over-parameterization as key themes to understand deep learning.
result Interpolation and over-parameterization are crucial for deep learning models to fit data exactly and generalize well.
A new method uses denoising diffusion models to improve seismic data interpolation.
problem Improving the accuracy of seismic data interpolation to enhance imaging and interpretation.
method The approach combines denoising diffusion probabilistic models with coherence-corrected resampling strategies.
result The proposed method achieves superior performance and generalization to various missing patterns and noise levels.
In order to generate novel 3D shapes with machine learning, one must allow for interpolation. The typical approach for incorporating this creative process is to interpolate in a learned latent space so as to avoid the problem of generating unrealistic instances by exploiting the model's learned structure. The process o…
This work proposes a novel method for interpolating ROMs without solving FEM models.
problem Interpolating ROMs for unseen parameter values without solving FEM models.
method Non-intrusive Space-Time POD interpolation on compact Stiefel manifolds.
result Robust ROMs derived for unseen parameter values with strong correlations to high-fidelity simulations.
We focus on an interpolation method referred to Bayesian reconstruction in this paper. Whereas in standard interpolation methods missing data are interpolated deterministically, in Bayesian reconstruction, missing data are interpolated probabilistically using a Bayesian treatment. In this paper, we address the framewor…
Study on Gaussian interpolation flows for generative modeling.
problem Theoretical properties and regularizing effect of Gaussian denoising in continuous normalizing flows.
method Unified framework of Gaussian interpolation flow, Lipschitz regularity, existence and uniqueness of flow, stability analysis.
result Established theoretical properties of Gaussian interpolation flows, including Lipschitz continuity and existence of flow.
This work bridges stochastic interpolants to infinite-dimensional Hilbert spaces.
problem Limited flexibility in generating arbitrary distributions for function-valued data.
method Establishes a rigorous framework for stochastic interpolants in infinite-dimensional Hilbert spaces.
result Achieves state-of-the-art results in conditional generation for complex PDE-based benchmarks.
Models which postulate lognormal dynamics for interest rates which are compounded according to market conventions, such as forward LIBOR or forward swap rates, can be constructed initially in a discrete tenor framework. Interpolating interest rates between maturities in the discrete tenor structure is equivalent to ext…
Generative models learn manifold structure; new approach uses atlas and geodesic interpolation.
problem Challenges in representing manifolds with topology different from Euclidean space.
method Atlas Generative Models (AGMs) with hybrid latent spaces and geodesic interpolation.
result Geodesic interpolation can be extended to AGMs, improving manifold representation.
The paper analyzes the generalization error of min-norm interpolators in transfer learning with limited test samples.
problem Characterizing the generalization error of min-norm interpolators in transfer learning with limited test samples.
method Characterizes the bias and variance of pooled min-ℓ2-norm interpolation under covariate shift and model shift. result Shows that adding data can hurt when SNR is low and is beneficial at higher SNR levels under certain conditions.
Study on clustering in high dimensions with anisotropic Gaussian mixtures, showing interpolation can be optimal and robust.
problem Clustering in high-dimensional anisotropic Gaussian mixtures.
method Derive minimax bounds, analyze ℓ2-regularized classifiers, and investigate interpolation's robustness. result Interpolating solutions can be optimal and robust under certain conditions.
FFRK automatically extracts features for spatial interpolation without external variables.
problem Spatial interpolation challenges, especially nonstationarity and lack of explanatory variables.
method Feature-Free Regression Kriging (FFRK) method that extracts geospatial features.
result FFRK outperforms classical methods in predicting heavy metal concentrations.
Fair Mixup improves fairness in classifiers by interpolating between groups.
problem Ensuring fairness in classifiers during training and evaluation.
method Fair Mixup uses interpolation of samples between groups to enforce fairness constraints.
result Fair Mixup ensures better generalization of fairness in various benchmarks.