Dreaming neural networks learn and consolidate patterns during sleep.
problem Maximizing information storage and critical capacity in neural networks.
method Daily routine of learning during awake state and consolidation during sleep, using Guerra's interpolation techniques.
result The network achieves perfect retrieval regime after sleep, storing the same number of patterns as neurons.
We study the fundamental limits of detecting the presence of an additive rank-one perturbation, or spike, to a Wigner matrix. When the spike comes from a prior that is i.i.d. across coordinates, we prove that the log-likelihood ratio of the spiked model against the non-spiked one is asymptotically normal below a certai…
Analyzes unsupervised neural networks using statistical mechanics and Monte Carlo simulations.
problem Understanding computational capabilities of unsupervised neural networks.
method Statistical mechanics approach and Monte Carlo simulations.
result Obtained a phase diagram summarizing network performance.
Paper introduces supervised and unsupervised TAM models for binary neurons.
problem Learning and retrieval of structured triplets of patterns in neural networks.
method Extends Hebbian paradigm to supervised and unsupervised protocols, using glassy statistical mechanical techniques.
result Obtained self-consistency equations for critical dataset sizes and retrieval performance.
Dense neural networks learn efficiently with large datasets and noise.
problem Training neural networks with large, noisy datasets.
method Statistical mechanics and Monte Carlo simulations.
result Dense neural networks can handle large amounts of patterns and recognize patterns at high signal-to-noise ratios.
Dense Associative Memories outperform classical networks in robustness and signal processing.
problem Improving neural network performance in adversarial attacks and weak signal processing.
method Relaxing replica symmetry in statistical mechanics of spin glasses to analyze unsupervised and supervised learning.
result Explicit analytical investigation of phase diagrams and storage capacities for Dense Associative Memories.
Study neural networks learning from noisy examples via reverberation.
problem Learning from noisy examples in neural networks.
method Adapted Guerra's interpolation technique to provide statistical mechanics of supervised and unsupervised learning.
result Full phase diagrams and thresholds for learning are analytically obtained.
This paper shows how forward rate interpolations are equivalent to discount factor interpolations in yield curve construction.
problem The challenge of choosing between different interpolation methods for yield curve construction.
method Demonstrates the equivalence between forward rate interpolations and discount factor interpolations.
result Some popular interpolation methods on forward rates are equivalent to classical interpolation methods on discount factors.
The paper proposes a method for generating uniform interpolations on data manifolds.
problem Generating high-quality interpolations between data samples on complex manifolds.
method Autoencoder network with interpolation network, regularized by a Riemannian metric.
result The method generates interpolations that remain within the manifold's distribution.
Study finds conditions for Legendre curves to be interpolating sesqui-harmonic in Sasakian space forms.
problem Characterizing Legendre curves in Sasakian space forms.
method Analyzes necessary and sufficient conditions for Legendre curves to be interpolating sesqui-harmonic.
result Obtains an example of an interpolating sesqui-harmonic Legendre curve in a Sasakian space form.
New FX option interpolations impact implied volatilities.
problem Different interpolations of FX option quotes lead to varying implied volatilities.
method Analysis of various exact interpolations of broker quotes.
result Different interpolations result in different implied volatilities.
Kernel interpolation is inconsistent for norms with smoothness above a constant.
problem Inconsistency of kernel interpolation in reproducing kernel Hilbert spaces.
method Lower bounds for generalization error in Sobolev norms.
result Kernel interpolation is always inconsistent for norms with smoothness above a constant.
Bayesian interpolants explain neural network inferences concisely.
problem Understanding neural network inferences.
method Adapting Craig interpolants for neural networks.
result Produces precise, understandable explanations.
Near-interpolating models grow norms quickly, affecting generalization.
problem Understanding the trade-off between interpolation and generalization in near-interpolating models.
method Random matrix theory and eigendecay analysis of data covariance matrix.
result Near-interpolating models exhibit rapid norm growth and worse generalization trade-offs.
The paper improves interpolation in generative models by using specific base distributions.
problem Unexpected side effects in linear interpolations of normalizing flows.
method Enforces a specific manifold using Dirichlet and von Mises-Fisher base distributions.
result Superior performance in terms of bits per dimension, FID, and KID scores for interpolation.
Paper presents a unified approach to interpolation and geodesics in latent spaces of generative models.
problem Finding geodesics and interpolating in latent spaces of non-Gaussian densities.
method General approach to interpolation and geodesics in latent space for arbitrary density.
result Maximizing quality measure of an interpolating curve is equivalent to finding geodesic.
Improved sparse-view CT images with deep learning sinogram interpolation.
problem Sparse-view CT images quality improvement with limited projection data.
method Combination of U-Net and residual learning for sinogram interpolation.
result Significantly improved CT image quality (RMSE and SSIM metrics) over standard methods.
Deep neural networks can interpolate any dataset in the overparametrized regime.
problem Interpolating any dataset with deep neural networks in the overparametrized regime.
method Proving universal approximations and interpolating any dataset with deep neural networks, considering specific conditions on activation functions.
result Interpolation of any dataset is possible in the overparametrized regime with deep neural networks.
Interpolation hurts robust generalization even without noise.
problem The challenge of robust generalization in the absence of noise.
method Avoiding interpolation through ridge regularization.
result Ridge regularization improves robust generalization.
Uniform convergence of interpolators proven for Gaussian data.
problem Interpolation learning in high-dimensional linear regression with Gaussian data.
method Generic uniform convergence guarantee in terms of Gaussian width.
result Consistency of interpolators for minimum-norm and near-minimal-norm cases.
Interpolation improves nearest neighbor algorithms' performance.
problem Improving nearest neighbor algorithms' performance.
method Considered a class of interpolated weighting schemes and characterized their asymptotic performances.
result Mild degree of data interpolation strictly improves prediction accuracy and statistical stability.
The paper characterizes vector fields as interpolating sesqui-harmonic maps on Riemannian manifolds.
problem Characterizing vector fields as interpolating sesqui-harmonic maps on Riemannian manifolds.
method Characterization theorem and critical point condition for interpolating sesqui-harmonic vector fields.
result Conditions for vector fields to be interpolating sesqui-harmonic maps on compact manifolds.
SoftKI combines SKI and variational methods for scalable GP regression.
problem Scalable Gaussian Process regression on high-dimensional datasets.
method SoftKI approximates kernel via softmax interpolation from a smaller number of learned points.
result SoftKI is competitive with other approximated GP methods for modest data dimensions.
Proves a new law of robustness for interpolating arbitrary data distributions.
problem Understanding robust interpolation for arbitrary data distributions.
method Proves a Lipschitzness lower bound for robust interpolation.
result Demonstrates a two-fold law of robustness for interpolating functions.
Interpolating estimators in nonparametric regression become suboptimal under adversarial attacks.
problem Adversarial robustness of interpolating estimators in nonparametric regression.
method Investigation of adversarial robustness of interpolating estimators in a nonparametric regression framework.
result Interpolating estimators must be suboptimal even under a subtle future X-attack. Paper investigates optimal interpolation methods in linear regression.
problem Understanding when interpolating methods generalize well in linear regression.
method Investigates optimal response-linear interpolators using functions linear in the response variable.
result Provides a closed-form expression for the optimal interpolator and shows it can be derived as the limit of gradient descent.
Covariance shrinkage via stochastic interpolation
problem High-dimensional covariance estimation
method Recasting shrinkage as empirical risk minimization
result Reduces statistical risk through scheduling, flow maps, and early stopping
Characterizes kernel interpolation in large dimensions, revealing optimal and sub-optimal regions.
problem Understanding the phase diagram of kernel interpolation in large dimensions.
method Characterization of variance and bias under various source conditions.
result Determined the (s,γ)-phase diagram of large-dimensional kernel interpolation. This work predicts and interpolates long-range videos using unsupervised landmarks.
problem Predicting and interpolating long-range video data with occlusions and appearance changes.
method Unsupervised latent structure inference followed by temporal prediction in a latent space.
result High-quality long-range video interpolation and extrapolation achieved through landmark representation.
Study on biharmonic and interpolating sesqui-harmonic vector fields on para-Kähler--Norden manifolds.
problem Investigating higher-order harmonicity in pseudo-Riemannian geometry.
method Deriving first variations of bienergy and interpolating sesqui-energy functionals, characterizing biharmonic and interpolating sesqui-harmonic vector fields.
result Explicit characterizations and examples of vector fields satisfying biharmonic and interpolating sesqui-harmonic conditions.
Study tightens bounds for interpolating noisy data using minimum l1-norm.
problem Predicting noisy data with minimum l1-norm interpolation.
method Provided matching upper and lower bounds for prediction error.
result Tight consistency up to negligible terms for d≫n. A novel density regularizer improves data interpolation on non-simply-connected manifolds.
problem Topological differences between model-defined simply-connected manifolds and dataset's non-simply-connected regions.
method Density regularizer to circumvent low-probability-density regions (holes).
result Consistently better interpolation results on real-world image datasets.
Deep networks can interpolate noisy data without losing generalization.
problem Characterizing the relationship between interpolation and generalization in overparameterized deep networks.
method Analyzing the loss landscape of neural network functions over volumes around training data points, varying model parameters and training epochs.
result Loss sharpness in the input space follows a double descent, with large models predicting noisy targets over larger volumes around training data points.
Autoencoders provide a powerful framework for learning compressed representations by encoding all of the information needed to reconstruct a data point in a latent code. In some cases, autoencoders can "interpolate": By decoding the convex combination of the latent codes for two datapoints, the autoencoder can produce …
AutoEncoder smooths noisy sensor data and interpolates missing values.
problem Noisy sensor data and missing timepoints require interpolation.
method Uses AutoEncoder to denoise and interpolate data.
result AutoEncoder improves data quality and reveals dynamics.
Enhances interpolation paths in latent space using particle filters.
problem Generating meaningful interpolations between data points in latent space.
method Introduces a discriminator network to guide particle filter sampling of interpolation paths.
result Improved variability and stronger drift towards high data density areas.
We introduce a new structured kernel interpolation (SKI) framework, which generalises and unifies inducing point methods for scalable Gaussian processes (GPs). SKI methods produce kernel approximations for fast computations through kernel interpolation. The SKI framework clarifies how the quality of an inducing point a…
The paper proves approximation and interpolation theorems for maxfaces with singularities.
problem Proving approximation and interpolation theorems for maxfaces with singularities.
method Surveying and applying Enneper--Weierstrass representation formula methods to maxfaces, incorporating singularity criteria.
result Existence of maxfaces with prescribed singularities and maxfaces with dense image singular set.
This research solves Hermite interpolation on manifolds using retractions.
problem Interpolating data on non-Euclidean spaces with matching derivatives.
method Proposes a novel procedure using retractions for Hermite interpolation on various manifolds.
result Establishes the well-posedness of the method and extends Hermite interpolation results to manifolds.
New optimization method helps models generalize better after achieving near-perfect training performance.
problem Models can achieve near-perfect training performance but fail to generalize well to unseen examples.
method GROKtimizer combines rapid convergence to interpolation with post-interpolation norm minimization using Critically Damped Momentum.
result GROKtimizer provides a quadratic speedup over classical gradient descent, offering a natural solution for selecting low-norm interpolating solutions.
Geometrically interpolates rigid body motions with initial and terminal twists.
problem Finding spatial trajectories between prescribed initial and terminal poses.
method Derives solutions for k-IV-TIP and k-BV-TIP for k=1,...,4.
result Automatic cubic interpolation identical to minimum acceleration curve when twists are zero.
A new tradeoff between regularization and sharpness improves model performance in overparameterized settings.
problem Improving model performance in overparameterized settings with minimum-norm interpolators.
method Proposes a regularization-sharpness tradeoff for overparameterized linear regression with an ℓ^p penalty.
result Empirical validation shows the tradeoff terms can distinguish performant linear interpolators.
Interpolation can prevent classifiers from having desired invariance properties.
problem Invariance properties in over-parameterized models are often ineffective.
method Theoretical analysis and algorithm design for non-interpolating classifiers.
result Interpolating classifiers cannot satisfy desired invariance properties.
Strong inductive biases prevent harmless interpolation in overparameterized models.
problem Understanding the conditions under which overparameterized models can interpolate noise without overfitting.
method Theoretical analysis of high-dimensional kernel regression and deep neural networks, focusing on the role of inductive biases.
result The strength of an estimator's inductive bias determines whether interpolation is harmless or requires fitting noise for good generalization.
Paper shows faster convergence to local-minimizers in over-parametrized models under interpolation-like conditions.
problem Escaping saddle-points in over-parametrized models.
method Stochastic and deterministic optimization algorithms under interpolation-like conditions.
result Oracle complexity of PSGD and SCRN algorithms to reach ε-local-minimizer matches or improves upon deterministic rates. We show that stochastic interpolation flow maps are Lipschitz with a sharp constant.
problem High dimensional sampling and transport problems.
method Investigating stochastic interpolation flow for generating data samples.
result Stochastic interpolation flow maps are Lipschitz with a sharp constant matching optimal transport maps.
Private optimization faster on interpolation problems with quadratic growth.
problem Private optimization in interpolation problems.
method Adaptive algorithm with improved sample complexity.
result Exponential improvement in private sample complexity for quadratic growth.
Interpolation with Laplace kernel fails in low dimensions but succeeds in high dimensions.
problem Consistency of interpolation with Laplace kernels in low-dimensional settings.
method Minimum-norm interpolation in Reproducing Kernel Hilbert Space (RKHS) with Laplace kernel.
result Consistency of interpolation is a high-dimensional phenomenon.