New distributed algorithm for gradient descent converges linearly in the interpolation limit.
problem Analyzing convergence of stochastic gradient descent in the interpolation limit of high-dimensional data fitting.
method Introduced a distributed gradient descent algorithm for the penalized distributed loss function, showing linear convergence rate.
result Distributed SGD algorithm converges linearly with rate 1-η/nλ_min(H), where λ_min(H) is the smallest nonzero eigenvalue of the Hessian.
Gaussian Processes improve data interpolation from diverse experiments.
problem Interpolation of sparse and inconsistent datasets from various experiments.
method Used Gaussian Processes (GP) for data interpolation, including uncertainty quantification.
result GPs successfully interpolate data and quantify uncertainties, demonstrating consistency across different sources.
AutoEncoder smooths noisy sensor data and interpolates missing values.
problem Noisy sensor data and missing timepoints require interpolation.
method Uses AutoEncoder to denoise and interpolate data.
result AutoEncoder improves data quality and reveals dynamics.
The paper analyzes the generalization error of min-norm interpolators in transfer learning with limited test samples.
problem Characterizing the generalization error of min-norm interpolators in transfer learning with limited test samples.
method Characterizes the bias and variance of pooled min- ℓ 2 \ell_2 ℓ 2 -norm interpolation under covariate shift and model shift. result Shows that adding data can hurt when SNR is low and is beneficial at higher SNR levels under certain conditions.
This work bridges stochastic interpolants to infinite-dimensional Hilbert spaces.
problem Limited flexibility in generating arbitrary distributions for function-valued data.
method Establishes a rigorous framework for stochastic interpolants in infinite-dimensional Hilbert spaces.
result Achieves state-of-the-art results in conditional generation for complex PDE-based benchmarks.
Study finds exact limits for sparse regression with fewer observations than usual.
problem Understanding sparse linear regression with sublinear sparsity.
method Adaptive interpolation method and modified AMP algorithm.
result Exact asymptotic expressions for mutual information and MMSE in sublinear sparsity.
The paper improves generalization bounds using interpolation between various divergences.
problem Improving generalization bounds in machine learning.
method Derives new PAC-Bayes generalization bounds based on ( f , Γ ) (f, Γ) ( f , Γ ) -divergence and interpolates between various divergences. result Connects derived bounds to earlier statistical learning results and provides practical training objectives.
The study examines issues with latent distributions in generative models and proposes using Cauchy distribution.
problem Issues with latent distributions causing mismatch in sampled regions during linear interpolations.
method Proposed using multidimensional Cauchy distribution and two methods for creating non-linear interpolations.
result Linear interpolations may generate unrealistic data due to the Central Limit Theorem, and Cauchy distribution mitigates this issue.
Combines spline interpolation and ARIMA for stock market forecasting.
problem Limited predictive performance of ARIMA in noisy data.
method Integrates cubic spline interpolation and ARIMA for time series forecasting.
result Demonstrates guidance for short-term stock market forecasting.
Paper investigates optimal interpolation methods in linear regression.
problem Understanding when interpolating methods generalize well in linear regression.
method Investigates optimal response-linear interpolators using functions linear in the response variable.
result Provides a closed-form expression for the optimal interpolator and shows it can be derived as the limit of gradient descent.
Paper explores methods to improve web search ranking.
problem Improving web search ranking accuracy with varied test data.
method Model Interpolation and Boosting Algorithm.
result Model Interpolation achieves best results on open test sets.
Task shift from classification to regression is possible in overparameterized linear models with limited additional data.
problem Transferability of latent knowledge from classification to regression in overparameterized linear models.
method Investigation of task shift in overparameterized linear regression, zero-shot and few-shot cases, with a focus on minimum-norm interpolation.
result Minimum-norm interpolators can transfer latent knowledge from classification to regression with limited additional data.
Survey explains Kahler-Einstein metrics construction from interpolation problems.
problem Kahler-Einstein metrics on compact complex manifolds.
method Statistical mechanical construction from interpolation problems.
result Probabilistic construction of Kahler solutions to Einstein's equations.
Geometric families of low-rank covariances improve flexibility and tractability in high dimensions.
problem Interpolating and identifying covariance matrices in high dimensions with limited data.
method Differential geometric construction of low-rank covariance families, interpolation on manifolds, and distance minimization for identification.
result Differential geometric covariance families offer significant flexibility and computational tractability.
Interpolates mean shift and spectral clustering on graphs.
problem Data clustering algorithms.
method Fokker-Planck equations on data graphs.
result New theoretical insights on diffusion maps and mean shift dynamics.
Study improves sugarcane plot prediction using data interpolation.
problem Predicting adventive plants in sugarcane plots with limited data.
method Interpolation techniques (Gaussian processes, kriging) for geo-referenced data augmentation.
result GP-COMB outperforms other methods with less additional data.
Bagging stabilizes linear interpolators, improving their generalization performance.
problem Unstable linear interpolators fail on noisy data.
method Introduced multiplier-bootstrap-based bagged least square estimator.
result Bagging effectively mitigates variance, leading to bounded prediction risk.
This paper analyzes the interpolation error of nonlinear Attention compared to linear regression.
problem Understanding the interpolation error of nonlinear Attention in high-dimensional settings.
method Derives explicit expressions for mean-squared interpolation error using signal-plus-noise model and random matrix theory.
result Nonlinear Attention generally incurs a larger interpolation error than linear regression, but this gap can be reversed with structured signals.
Paper introduces a method to explain deep learning models and identify good generalization.
problem Limited interpretability of neural networks hinders progress and real-world applications.
method Polytope interpolation method for local explainability and generalization assessment.
result Developed a method to identify deep learning models with good generalization properties.
CNNs predict spatial fields from sparse data.
problem Predicting complete spatial fields from limited observations.
method Convolutional Neural Networks (CNNs) trained on a single partially observed field.
result CNNs can flexibly capture local spatial patterns without explicit covariance modeling.
Neural networks learn incrementally from orthogonal data, interpolating with minimal complexity.
problem Understanding the learning dynamics and implicit bias in ReLU networks with orthogonal data.
method Gradient flow analysis of two-layer ReLU networks from small initialization with orthogonal training data.
result The learned interpolator has a squared ℓ 2 \ell_2 ℓ 2 -norm scaling as n \sqrt{n} n , close to the minimal interpolator's complexity. NODEs with explicit time dependence can interpolate and generalize like piecewise-constant estimators.
problem Learning from finite datasets with neural ODEs.
method Control-theoretic perspective applied to semi-autonomous NODEs.
result SA-NODEs can interpolate and satisfy SCC, leading to generalization rates similar to histogram and nearest-neighbor estimators.
A new criterion selects models in overparameterized settings.
problem Model selection for overparameterized models with more parameters than data.
method Establishes Bayesian duality and introduces the Interpolating Information Criterion.
result The Interpolating Information Criterion selects models in overparameterized settings.
Hybrid framework merges data and domain knowledge for better spatial interpolation.
problem Spatial interpolation overlooks domain knowledge and limits to spatial coordinates.
method Integrates data-driven features with rule-assisted spatial dependency function mapping.
result Superior performance in two application scenarios, capturing localized features.
A simple GI loss improves temporal generalization without complex methods.
problem Temporal drift between train and test distributions in evolving data.
method Gradient Interpolation (GI) loss to regularize temporal complexity.
result GI loss outperforms complex methods on real-world datasets.
New insights into overfitting peaks in generalization error for l 2 l_2 l 2 and l 1 l_1 l 1 penalized interpolation.
problem Understanding the phenomenon of overfitting peaks in generalization error for modern machine learning models.
method Introducing a generative and fitting model pair (MiSpaR) and deriving analytical risk curves for l 2 l_2 l 2 and l 1 l_1 l 1 penalties. result The overfitting peak can be dissociated from the point of model flexibility, complicating the interpretation of overfitting as a boundary between classical and modern regimes.
We analyze ridge interpolators in correlated factor regression models using RDT.
problem Performance analysis of ridge interpolators in correlated factor regression models.
method Utilizing Random Duality Theory (RDT), we obtain precise closed form characterizations of optimization problems.
result Ridge interpolators can smooth out the excess prediction risk and exhibit double-descent behavior.
Improves latent space structure for better data representation.
problem Limited ability of conventional priors to encode data manifold structure.
method Introduces an Encoded Prior Sliced Wasserstein AutoEncoder with iterative training and geodesic interpolation.
result Learned manifold encoding preserves topological and geometric properties of data.
Topology-GS improves 3D GS for better structural and feature integrity.
problem Compromised pixel-level and feature-level integrity in 3D GS.
method Incorporates Local Persistent Voronoi Interpolation (LPVI) and PersLoss based on persistent homology.
result Topology-GS outperforms existing methods in PSNR, SSIM, and LPIPS metrics.
Efficiently computes quasiconcave envelope with limited data.
problem Approximating unknown quasiconcave function with partial information.
method Solves value problem and interpolation problem with polynomial and logarithmic LPs.
result Efficiently computes quasiconcave envelope with limited data.
RNGI model bridges two probability densities on Riemannian manifolds efficiently.
problem Limited applicability of Euclidean stochastic interpolants to Riemannian manifolds.
method Introduces RNGI model interpolating between Riemannian manifold probability densities along geodesics.
result Proves temporal marginal density solves transport equation on Riemannian manifold.
GNNs outperform NNs in interpolating bandlimited functions on Euclidean cubes.
problem Interpolating bandlimited functions on Euclidean cubes using GNNs vs. NNs.
method Investigates optimal GNN configurations and weights for function interpolation.
result GNNs require fewer weights and samples to interpolate bandlimited functions compared to NNs.
Study on optimal ReLU networks with weight decay for interpolation.
problem Interpolating data with radially symmetric distributions using shallow ReLU networks.
method Weight decay regularization in infinite neuron, infinite data limit; analysis of growth rates.
result Existence and growth rates of unique radially symmetric minimizers with weight decay.
Study on gradient dynamics of shallow ReLU networks for least-squares interpolation.
problem Understanding the gradient dynamics of shallow ReLU networks for interpolation.
method Theoretical and empirical analysis of gradient flow in non-redundant parameterization.
result Identification of two learning regimes: kernel and adaptive, with distinct interpolant shapes.
Geometric theory connects machine learning classifiers to differential geometry.
problem Classifying data points in machine learning.
method Mapping binary classification to vector bundles and differential geometry.
result Harmonic interpolation solves RKHS interpolation problems.
Study assesses neural nets for optimization problems, highlighting SiLU's effectiveness.
problem Using neural nets for optimization problems, especially for accurate approximations.
method Determined best activation function (SiLU) for nonlinear optimization problems. Analyzed function approximations using neural networks and interpolation/regression models.
result Neural nets can deliver competitive zero- and first-order approximations but underperform on second-order approximations.
Study geometric structures in transfer learning to avoid negative transfer.
problem Understanding information-theoretic limits of transfer learning without exploiting domain geometry.
method Integrates geometric structure into linear regression models, using Gram matrices of source and target domains.
result Proposes an interpolation estimator that matches minimax lower bound and outperforms existing methods.
MTDS improves sequence generation adaptability via latent code control.
problem Lack of adaptability in sequence generation models like RNNs.
method Hierarchical multi-task dynamical systems (MTDS) with latent code control.
result MTDS enables style transfer, interpolation, and morphing in generated sequences.
Study contractibility of boundaries in convex sets and limit sets of subgroups.
problem Understanding contractibility of boundaries and wildness of limit sets in geometric structures.
method Use sufficient conditions for contractibility, study coarse upper curvature bounds, and analyze interpolation in geodesic metric spaces.
result Conditions for contractibility of boundaries and properties of limit sets are established.
Transformers can interpolate between arbitrary measures.
problem Understanding the expressive power of Transformers as measure-to-measure maps.
method Provided an explicit choice of parameters for a single Transformer to match N arbitrary input measures to N arbitrary target measures.
result A single Transformer can interpolate between arbitrary measures.
This paper shows how forward rate interpolations are equivalent to discount factor interpolations in yield curve construction.
problem The challenge of choosing between different interpolation methods for yield curve construction.
method Demonstrates the equivalence between forward rate interpolations and discount factor interpolations.
result Some popular interpolation methods on forward rates are equivalent to classical interpolation methods on discount factors.
The paper proposes a method to learn 3D object pose manifolds using GANs and elasticae.
problem Learning image manifolds of 3D objects with limited data.
method Geom-SGAN and elasticae for geometry-preserving image interpolation.
result The method outperforms state-of-the-art GANs and VAEs in learning rotation paths.
The paper proposes a method for generating uniform interpolations on data manifolds.
problem Generating high-quality interpolations between data samples on complex manifolds.
method Autoencoder network with interpolation network, regularized by a Riemannian metric.
result The method generates interpolations that remain within the manifold's distribution.
This paper derives a diffusion approximation for a sequence of discrete-time one-sided limit order book models with non-linear state dependent order arrival and cancellation dynamics. The discrete time sequences are specified in terms of an R + \R_+ R + -valued best bid price process and an L l o c 2 L^2_{loc} L l oc 2 -valued volume process. …
BWFlow improves graph generation by smoothly interpolating graph components.
problem Disjoint modeling of graph nodes and edges leads to irregular and non-smooth probability paths.
method Modeling graphs as MRFs and using optimal transport displacement for a smooth probability path.
result BWFlow achieves better training convergence and efficient sampling in graph generation.
Study finds conditions for Legendre curves to be interpolating sesqui-harmonic in Sasakian space forms.
problem Characterizing Legendre curves in Sasakian space forms.
method Analyzes necessary and sufficient conditions for Legendre curves to be interpolating sesqui-harmonic.
result Obtains an example of an interpolating sesqui-harmonic Legendre curve in a Sasakian space form.
SGD converges fast in over-parametrized models that interpolate data.
problem Understanding fast convergence of SGD in modern machine learning.
method Formal analysis of SGD convergence in over-parametrized models.
result SGD converges similarly to full gradient descent in certain regimes.
New FX option interpolations impact implied volatilities.
problem Different interpolations of FX option quotes lead to varying implied volatilities.
method Analysis of various exact interpolations of broker quotes.
result Different interpolations result in different implied volatilities.