Study shows interpolating predictor's risk is optimal in low-dimensional factor regression models.
problem Understanding the risk of interpolating predictors in high-dimensional factor regression models.
method Detailed finite-sample analysis of minimum-norm interpolating predictor's risk in factor regression models.
result The risk of the minimum-norm interpolating predictor approaches optimal benchmarks in low-dimensional factor regression models.
The paper explores why a specific type of predictor works well in noisy data.
problem Understanding why a specific type of predictor (minimum-norm interpolator) works well in noisy data.
method The paper uses uniform convergence and zero-error predictors in a norm ball to explain the success of the minimum-norm interpolator.
result The minimum-norm interpolator is consistent, and this can be explained by uniform convergence of zero-error predictors in a norm ball.
New insights into Nadaraya-Watson interpolators show varied generalization behaviors.
problem Understanding generalization of interpolating predictors, especially in noisy data.
method Revisiting Nadaraya-Watson estimator with a single hyperparameter.
result Multiple overfitting behaviors exist, ranging from catastrophic to tempered.
Study shows gap between uniform convergence and test error in random feature models.
problem Understanding the gap between uniform convergence and test error in random feature models.
method Analytical expressions for uniform convergence over norm balls, interpolators, and minimum norm interpolator risk derived and proved.
result Uniform convergence over interpolators still gives a non-trivial bound of test error even when classical uniform convergence is vacuous.
We propose to study the generalization error of a learned predictor h^ in terms of that of a surrogate (potentially randomized) predictor that is coupled to h^ and designed to trade empirical risk for control of generalization error. In the case where h^ interpolates the data, it is interesting to con…
It is widely believed that the prediction accuracy of decision tree models is invariant under any strictly monotone transformation of the individual predictor variables. However, this statement may be false when predicting new observations with values that were not seen in the training-set and are close to the location…
Proposes bivariate DeepKriging for efficient wind field prediction.
problem Challenges in predicting large-scale bivariate wind fields with high spatial variability and heterogeneity.
method Spatially dependent deep neural network (DNN) with embedding layer using spatial radial basis functions.
result Outperforms traditional cokriging predictors and reduces computation time.
Study calibrates high-dimensional binary classifiers using angle between estimator and true weights.
problem Calibrating high-dimensional binary classifiers with provable properties.
method Interpolates with a chance classifier to construct well-calibrated predictor based on angle between estimator and true weights.
result Angular calibration approach is provably well-calibrated in high dimensions, minimizing Bregman divergence.
Study optimizes linear regression analysis for high-dimensional settings.
problem Understanding high-dimensional linear regression with interpolation and regularization.
method Localized uniform convergence analysis of optimistic rates for linear regression.
result Recover guarantees for ridge and LASSO regression under random designs.
The study finds a trade-off between model size, test loss, and training loss for linear predictors.
problem Finding the optimal balance between model size, test loss, and training loss for linear predictors.
method Established an algorithm and distribution-independent trade-off using non-asymptotic analysis.
result Models with low test loss are either classical (close to noise level training loss) or modern (large number of parameters).
Neural networks can interpolate noisy data and still generalize well.
problem Generalization of neural networks trained on noisy data.
method Two-layer neural networks trained to interpolation by gradient descent on corrupted labels.
result Neural networks can achieve zero training error and optimal test error.
Machine learning is used to approximate density functionals. For the model problem of the kinetic energy of non-interacting fermions in 1d, mean absolute errors below 1 kcal/mol on test densities similar to the training set are reached with fewer than 100 training densities. A predictor identifies if a test density is …
Task shift from classification to regression is possible in overparameterized linear models with limited additional data.
problem Transferability of latent knowledge from classification to regression in overparameterized linear models.
method Investigation of task shift in overparameterized linear regression, zero-shot and few-shot cases, with a focus on minimum-norm interpolation.
result Minimum-norm interpolators can transfer latent knowledge from classification to regression with limited additional data.
New insights into when benign overfitting occurs in linear and classification tasks.
problem Understanding when benign overfitting happens in linear and classification models.
method Analysis of a generic data model and comparison of predictors (minimum-norm interpolating and max-margin).
result The minimum-norm interpolating predictor is biased towards an inconsistent solution, preventing benign overfitting in linear regression.
This study analyzes prediction risk for PCR method in latent factor regression models.
problem Prediction risk analysis in latent factor regression models.
method Adaptive PCR method with risk bounds established under factor regression model.
result Unified framework for analyzing various linear prediction methods under factor regression.
Neural network model forecasts extreme flood risk.
problem Accurately estimating high quantiles of extreme events.
method EQRN model combining neural networks and extreme value theory.
result Forecasting flood risk with improved adaptability.
We prove a new generalization bound that shows for any class of linear predictors in Gaussian space, the Rademacher complexity of the class and the training error under any continuous loss ℓ can control the test error under all Moreau envelopes of the loss ℓ. We use our finite-sample bound to directly recover…
Given a reproducing kernel Hilbert space H of real-valued functions and a suitable measure mu over the source space D (subset of R), we decompose H as the sum of a subspace of centered functions for mu and its orthogonal in H. This decomposition leads to a special case of ANOVA kernels, for which the functional ANOVA r…
This study argues for pruning trees in random forests to improve performance in low signal-to-noise scenarios.
problem Improving random forest performance in scenarios with low signal-to-noise ratio.
method Using regularization theory, the study re-examines the depth of trees in random forests and provides evidence that shallow trees are advantageous.
result Random forests with shallow trees are advantageous when the signal-to-noise ratio is low.
SGD converges to optimal solution in perfect data fitting problem.
problem Noiseless least-squares with perfect data fitting.
method Last iterate of SGD with constant step-size.
result Convergence of SGD final iterate for non-strongly convex problem.
Forecast stock return distributions using neural networks.
problem Accurately modeling non-Gaussian stock return features.
method Two-stage quantile neural network with spline interpolation.
result Improved mean and variance forecasts compared to standard models.
Paper develops an efficient method for conformal prediction in sparse linear models.
problem Computing conformal prediction sets for sparse linear models is computationally infeasible.
method Numerical continuation techniques to approximate the solution path efficiently.
result The method accurately approximates conformal prediction sets for sparse linear models.
This work falls within the context of predicting the value of a real function at some input locations given a limited number of observations of this function. The Kriging interpolation technique (or Gaussian process regression) is often considered to tackle such a problem but the method suffers from its computational b…
This paper shows how forward rate interpolations are equivalent to discount factor interpolations in yield curve construction.
problem The challenge of choosing between different interpolation methods for yield curve construction.
method Demonstrates the equivalence between forward rate interpolations and discount factor interpolations.
result Some popular interpolation methods on forward rates are equivalent to classical interpolation methods on discount factors.
Study shows that ridgeless Gaussian kernel regression overfits even with varying bandwidth or dimensionality.
problem Analyzing overfitting in Gaussian kernel ridgeless regression with varying bandwidth or dimensionality.
method Examined the behavior of minimum norm interpolating solutions for fixed and increasing dimensions under varying bandwidth and sample size.
result Ridgeless solutions are never consistent and can be worse than null predictor with large enough noise, even with varying bandwidth or dimensionality.
The paper proposes a method for generating uniform interpolations on data manifolds.
problem Generating high-quality interpolations between data samples on complex manifolds.
method Autoencoder network with interpolation network, regularized by a Riemannian metric.
result The method generates interpolations that remain within the manifold's distribution.
New FX option interpolations impact implied volatilities.
problem Different interpolations of FX option quotes lead to varying implied volatilities.
method Analysis of various exact interpolations of broker quotes.
result Different interpolations result in different implied volatilities.
The Absorption, Distribution, Metabolism, Elimination, and Toxicity (ADMET) properties of drug candidates are estimated to account for up to 50% of all clinical trial failures. Predicting ADMET properties has therefore been of great interest to the cheminformatics and medicinal chemistry communities in recent decades. …
Kernel interpolation is inconsistent for norms with smoothness above a constant.
problem Inconsistency of kernel interpolation in reproducing kernel Hilbert spaces.
method Lower bounds for generalization error in Sobolev norms.
result Kernel interpolation is always inconsistent for norms with smoothness above a constant.
Bayesian interpolants explain neural network inferences concisely.
problem Understanding neural network inferences.
method Adapting Craig interpolants for neural networks.
result Produces precise, understandable explanations.
Near-interpolating models grow norms quickly, affecting generalization.
problem Understanding the trade-off between interpolation and generalization in near-interpolating models.
method Random matrix theory and eigendecay analysis of data covariance matrix.
result Near-interpolating models exhibit rapid norm growth and worse generalization trade-offs.
The paper improves interpolation in generative models by using specific base distributions.
problem Unexpected side effects in linear interpolations of normalizing flows.
method Enforces a specific manifold using Dirichlet and von Mises-Fisher base distributions.
result Superior performance in terms of bits per dimension, FID, and KID scores for interpolation.
The problem of forecasting conditional probabilities of the next event given the past is considered in a general probabilistic setting. Given an arbitrary (large, uncountable) set C of predictors, we would like to construct a single predictor that performs asymptotically as well as the best predictor in C, on any data.…
Deep neural networks can interpolate any dataset in the overparametrized regime.
problem Interpolating any dataset with deep neural networks in the overparametrized regime.
method Proving universal approximations and interpolating any dataset with deep neural networks, considering specific conditions on activation functions.
result Interpolation of any dataset is possible in the overparametrized regime with deep neural networks.
Interpolation hurts robust generalization even without noise.
problem The challenge of robust generalization in the absence of noise.
method Avoiding interpolation through ridge regularization.
result Ridge regularization improves robust generalization.
Uniform convergence of interpolators proven for Gaussian data.
problem Interpolation learning in high-dimensional linear regression with Gaussian data.
method Generic uniform convergence guarantee in terms of Gaussian width.
result Consistency of interpolators for minimum-norm and near-minimal-norm cases.
The paper characterizes vector fields as interpolating sesqui-harmonic maps on Riemannian manifolds.
problem Characterizing vector fields as interpolating sesqui-harmonic maps on Riemannian manifolds.
method Characterization theorem and critical point condition for interpolating sesqui-harmonic vector fields.
result Conditions for vector fields to be interpolating sesqui-harmonic maps on compact manifolds.
SoftKI combines SKI and variational methods for scalable GP regression.
problem Scalable Gaussian Process regression on high-dimensional datasets.
method SoftKI approximates kernel via softmax interpolation from a smaller number of learned points.
result SoftKI is competitive with other approximated GP methods for modest data dimensions.
Proves a new law of robustness for interpolating arbitrary data distributions.
problem Understanding robust interpolation for arbitrary data distributions.
method Proves a Lipschitzness lower bound for robust interpolation.
result Demonstrates a two-fold law of robustness for interpolating functions.
Interpolating between points is a problem connected simultaneously with finding geodesics and study of generative models. In the case of geodesics, we search for the curves with the shortest length, while in the case of generative models we typically apply linear interpolation in the latent space. However, this interpo…
We consider interpolating sesqui-harmonic Legendre curves in Sasakian space forms. We find the necessary and sufficient conditions for Legendre curves in Sasakian space forms to be interpolating sesqui-harmonic. Finally, we obtain an example for an interpolating sesqui-harmonic Legendre curve in a Sasakian space form.
Interpolating estimators in nonparametric regression become suboptimal under adversarial attacks.
problem Adversarial robustness of interpolating estimators in nonparametric regression.
method Investigation of adversarial robustness of interpolating estimators in a nonparametric regression framework.
result Interpolating estimators must be suboptimal even under a subtle future X-attack. Deep networks retain initial bias after training, affecting generalization.
problem Understanding how much initial bias in neural networks survives training.
method Introduced initialization memory to measure initial bias's survival.
result SGD can preserve initial bias, while Adam-family methods erase it.
Paper investigates optimal interpolation methods in linear regression.
problem Understanding when interpolating methods generalize well in linear regression.
method Investigates optimal response-linear interpolators using functions linear in the response variable.
result Provides a closed-form expression for the optimal interpolator and shows it can be derived as the limit of gradient descent.
Covariance shrinkage via stochastic interpolation
problem High-dimensional covariance estimation
method Recasting shrinkage as empirical risk minimization
result Reduces statistical risk through scheduling, flow maps, and early stopping
Characterizes kernel interpolation in large dimensions, revealing optimal and sub-optimal regions.
problem Understanding the phase diagram of kernel interpolation in large dimensions.
method Characterization of variance and bias under various source conditions.
result Determined the (s,γ)-phase diagram of large-dimensional kernel interpolation. New bounds explain deterministic non-smooth deep nets without large Lipschitz constants.
problem Challenges in explaining generalization of deterministic non-smooth deep nets.
method De-randomized PAC-Bayes margin bounds for deterministic non-convex and non-smooth predictors.
result New bounds avoid large Lipschitz constants, providing generalization guarantees.
Study on biharmonic and interpolating sesqui-harmonic vector fields on para-Kähler--Norden manifolds.
problem Investigating higher-order harmonicity in pseudo-Riemannian geometry.
method Deriving first variations of bienergy and interpolating sesqui-energy functionals, characterizing biharmonic and interpolating sesqui-harmonic vector fields.
result Explicit characterizations and examples of vector fields satisfying biharmonic and interpolating sesqui-harmonic conditions.