Proves a new law of robustness for interpolating arbitrary data distributions.
problem Understanding robust interpolation for arbitrary data distributions.
method Proves a Lipschitzness lower bound for robust interpolation.
result Demonstrates a two-fold law of robustness for interpolating functions.
Interpolation hurts robust generalization even without noise.
problem The challenge of robust generalization in the absence of noise.
method Avoiding interpolation through ridge regularization.
result Ridge regularization improves robust generalization.
Interpolating estimators in nonparametric regression become suboptimal under adversarial attacks.
problem Adversarial robustness of interpolating estimators in nonparametric regression.
method Investigation of adversarial robustness of interpolating estimators in a nonparametric regression framework.
result Interpolating estimators must be suboptimal even under a subtle future X-attack. Adversarial robustness has become a central goal in deep learning, both in the theory and the practice. However, successful methods to improve the adversarial robustness (such as adversarial training) greatly hurt generalization performance on the unperturbed data. This could have a major impact on how the adversarial …
Proposes robust model through Wasserstein geodesic interpolation of training data.
problem Improving model robustness through data augmentation.
method Augment data by finding worst-case Wasserstein barycenter on geodesic path.
result Improves robustness on CIFAR-10 up to 7.7% and on CIFAR-100 up to 16.8%.
Interpolation can prevent classifiers from having desired invariance properties.
problem Invariance properties in over-parameterized models are often ineffective.
method Theoretical analysis and algorithm design for non-interpolating classifiers.
result Interpolating classifiers cannot satisfy desired invariance properties.
Study on clustering in high dimensions with anisotropic Gaussian mixtures, showing interpolation can be optimal and robust.
problem Clustering in high-dimensional anisotropic Gaussian mixtures.
method Derive minimax bounds, analyze ℓ2-regularized classifiers, and investigate interpolation's robustness. result Interpolating solutions can be optimal and robust under certain conditions.
New method certifies generative models' robustness.
problem Certifying generative models' robustness is challenging due to non-convex sets.
method ApproxLine, a scalable certification method capturing infinite sets or distributions over them.
result ApproxLine provides sound deterministic and probabilistic guarantees.
The paper analyzes the robustness of a minimum ℓ2 interpolator in high-dimensional linear regression.
problem Analyzing the robustness of a minimum ℓ2 interpolator in high-dimensional linear regression. method The paper analyzes the interpolator with minimal ℓ2-norm in a general high-dimensional linear regression framework, proving bounds on prediction loss. result The paper shows that the prediction loss of the interpolator is bounded by (∥β∗∥22rcn(Σ)∨∥ξ∥2)/n with high probability, revealing a transition in rates. The study analyzes robustness of estimators in linear models with adversarial errors.
problem Analyzing robustness of estimators in linear models with adversarial errors.
method Develops a general theory for minimum norm interpolating estimators and RERM in linear models without conditions on errors.
result Quantitative bound for the prediction error relating it to Rademacher complexity, norm of minimum norm interpolator of errors, and subdifferential size.
We describe a robust calibration algorithm of a set of SSVI slices (i.e. a set of 3 SSVI parameters θ,ρ,φ attached to each option maturity available on the market), which grants that these slices are free of Butterfly and Calendar-Spread arbitrage. Given such a set of consistent SSVI parameters, we show that …
New law explains why deep learning models often have more parameters than needed.
problem Why deep learning models often have more parameters than classical theory suggests.
method Proved a universal law of robustness for smooth interpolation.
result Smooth interpolation requires d times more parameters than mere interpolation.
Recent SVD-free matrix factorization formulations have enabled rank minimization for systems with millions of rows and columns, paving the way for matrix completion in extremely large-scale applications, such as seismic data interpolation. In this paper, we consider matrix completion formulations designed to hit a targ…
Adversarial training improves linear regression solutions, revealing sparsity and abrupt interpolation.
problem Adversarial attacks on linear regression models.
method Formulated as a convex problem, adversarial training is used to find robust solutions that are sparse and interpolate data.
result Adversarial training with small disturbances gives the solution with the minimum-norm that interpolates the training data, revealing abrupt transition into interpolation.
Gaussian Processes improve data interpolation from diverse experiments.
problem Interpolation of sparse and inconsistent datasets from various experiments.
method Used Gaussian Processes (GP) for data interpolation, including uncertainty quantification.
result GPs successfully interpolate data and quantify uncertainties, demonstrating consistency across different sources.
New method certifies images against transformations like rotations and translations.
problem Certifying robustness of images against transformations like rotations and translations.
method Randomized smoothing with three different kinds of defenses.
result Individual certificates can be obtained via statistical error bounds or efficient online inverse computation.
Investigates numerical issues in GP interpolation parameter estimation.
problem Numerical issues in maximum likelihood parameter estimation for Gaussian process interpolation.
method Investigates and proposes strategies to improve open-source software implementations.
result Improves reliability and reproducibility of studies relying on GP implementations.
Many modern machine learning models are trained to achieve zero or near-zero training error in order to obtain near-optimal (but non-zero) test error. This phenomenon of strong generalization performance for "overfitted" / interpolated classifiers appears to be ubiquitous in high-dimensional data, having been observed …
Geometric approach improves probabilistic robustness in neural networks.
problem Widespread lack of robustness in deep neural networks to adversarial examples.
method Geometric view on Probabilistically Robust Learning (PRL) and introduction of new perimeters.
result Existence of solutions and properties of modified PRL models.
Improving the accuracy and robustness of deep neural nets (DNNs) and adapting them to small training data are primary tasks in deep learning research. In this paper, we replace the output activation function of DNNs, typically the data-agnostic softmax function, with a graph Laplacian-based high dimensional interpolati…
This paper explores two classes of model adaptation methods for Web search ranking: Model Interpolation and error-driven learning approaches based on a boosting algorithm. The results show that model interpolation, though simple, achieves the best results on all the open test sets where the test data is very different …
Study improves sugarcane plot prediction using data interpolation.
problem Predicting adventive plants in sugarcane plots with limited data.
method Interpolation techniques (Gaussian processes, kriging) for geo-referenced data augmentation.
result GP-COMB outperforms other methods with less additional data.
This study reveals fundamental trade-offs between memorization and robustness in neural networks.
problem Understanding the balance between memorization and robustness in neural networks.
method Analyzes two-layer neural networks in various high-dimensional linearized regimes, focusing on Sobolev-seminorm.
result Establishes fundamental trade-offs between memorization and robustness, with lower bounds on Sobolev-seminorm.
Unified framework for robust, stable, and efficient density ratio estimation.
problem Density-chasm and support-chasm problems in density ratio estimation.
method Dequantified diffusion-Schrödinger bridge (D3RE) framework with DDBI and DSBI.
result Offers uniform approximation and bounded time scores in theory and empirical performance.
Unified ODE model explains residual and non-residual networks.
problem Unclear relationship between residual and non-residual networks.
method Introducing a damping term in an ODE model to interpolate between ResNet and CNN architectures.
result Unified framework for understanding residual and non-residual networks.
This work proposes a novel method for interpolating ROMs without solving FEM models.
problem Interpolating ROMs for unseen parameter values without solving FEM models.
method Non-intrusive Space-Time POD interpolation on compact Stiefel manifolds.
result Robust ROMs derived for unseen parameter values with strong correlations to high-fidelity simulations.
Study reconstructs Faber-Schauder coefficients from antiderivative observations.
problem Reconstructing Faber-Schauder coefficients from discrete antiderivative observations.
method Piecewise quadratic spline interpolation and closed-form solution.
result Final-generation coefficients are unstable; others are robust.
NTK neural networks are robust to adversarial attacks in nonparametric regression.
problem Adversarial robustness of neural networks in nonparametric regression.
method Gradient flow with early stopping for NTK neural networks, proving robustness in Sobolev spaces.
result NTK neural networks achieve optimal adversarial robustness rates in Sobolev spaces.
C-Mixup improves generalization in regression tasks by adjusting label similarity.
problem Improving generalization in regression tasks with limited data.
method C-Mixup adjusts the probability of mixing examples based on label similarity.
result C-Mixup achieves better generalization and robustness compared to vanilla mixup.
Develops a robust learning method for unknown context distributions.
problem Learning from data in different, unknown contexts.
method Focuses on excess risks, constructs distribution sets with statistical coverage.
result Shows robustness in worst-case scenarios without sacrificing nominal performance.
A common approach for defining a reward function for Multi-objective Reinforcement Learning (MORL) problems is the weighted sum of the multiple objectives. The weights are then treated as design parameters dependent on the expertise (and preference) of the person performing the learning, with the typical result that a …
Deep neural networks excel at learning the training data, but often provide incorrect and confident predictions when evaluated on slightly different test examples. This includes distribution shifts, outliers, and adversarial examples. To address these issues, we propose Manifold Mixup, a simple regularizer that encoura…
Recent works have shown that stochastic gradient descent (SGD) achieves the fast convergence rates of full-batch gradient descent for over-parameterized models satisfying certain interpolation conditions. However, the step-size used in these works depends on unknown quantities and SGD's practical performance heavily re…
A new method for sampling from complex distributions using Langevin samplers.
problem Sampling from unnormalized Boltzmann densities.
method Probability flow ODE derived from linear stochastic interpolants, employing Langevin samplers.
result Efficient simulation of the flow with non-asymptotic convergence rate.
Synthesizes robust estimators for domain adaptation.
problem Improving prediction accuracy in target domain with limited data.
method Synthesizes a family of robust least squares estimators using convex optimization.
result Robust strategies can outperform non-robust interpolations.
Adaptive gradient methods converge faster with over-parameterization and line-search.
problem Training over-parameterized models using adaptive gradient methods.
method Simplified setting of smooth, convex losses with over-parameterized models, proving convergence rates and demonstrating improvements with line-search techniques.
result Adaptive gradient methods, particularly AMSGrad, converge faster with line-search techniques.
We improve the robustness of Deep Neural Net (DNN) to adversarial attacks by using an interpolating function as the output activation. This data-dependent activation remarkably improves both the generalization and robustness of DNN. In the CIFAR10 benchmark, we raise the robust accuracy of the adversarially trained Res…
Techniques known as Nonlinear Set Membership prediction, Kinky Inference or Lipschitz Interpolation are fast and numerically robust approaches to nonparametric machine learning that have been proposed to be utilised in the context of system identification and learning-based control. They utilise presupposed Lipschitz p…
Adversarial training improves linear regression solutions, offering robustness against small perturbations.
problem Vulnerability of linear models to adversarial perturbations.
method Formulated as a min-max problem, adversarial training minimizes the best solution under worst-case attacks.
result Adversarial training yields the minimum-norm interpolating solution in overparameterized models, equivalent to parameter shrinking methods in underparameterized models.
AdaBoost improves binary classification in robust one-bit compressed sensing with adversarial errors.
problem Binary classification in robust one-bit compressed sensing with adversarial errors.
method AdaBoost and max-ℓ1-margin-classifier approach, with convergence rates improved under certain feature conditions. result Improved convergence rates and explanation for harmless interpolating adversarial noise.
Enhances neural network robustness with Mixup and TLAT.
problem Neural networks are sensitive to various perturbations and adversarial examples.
method Combines Mixup augmentation with Targeted Labeling Adversarial Training (TLAT).
result M-TLAT increases robustness against 19 corruptions and 5 adversarial attacks without reducing clean sample accuracy.
New priors improve robustness and interpretability in penalized regression.
problem Improper priors in penalized regression lead to suboptimal solutions.
method Developed non-zero priors inspired by human decision heuristics.
result Robust priors yield excellent worst-case performance across various tasks.
This paper shows how forward rate interpolations are equivalent to discount factor interpolations in yield curve construction.
problem The challenge of choosing between different interpolation methods for yield curve construction.
method Demonstrates the equivalence between forward rate interpolations and discount factor interpolations.
result Some popular interpolation methods on forward rates are equivalent to classical interpolation methods on discount factors.
Improved forecasting for irregularly-sampled time series using kernel flows.
problem Forecasting dynamical systems from irregularly-sampled time series data.
method Directly approximating the vector field using time differences in data-adapted kernels.
result Significant improvement in forecasting accuracy compared to classical methods.
This paper provides a practical method to extract caplet volatilities from quoted data.
problem Extracting caplet volatilities from quoted data is complex and not straightforward.
method The paper presents a constructive algorithm based on criteria and robust outlier detection. It includes direct interpolation, bootstrap methods, and global search methods.
result The paper introduces methods to extract caplet volatilities that are arbitrage-free and consistent with quoted data.
New algorithms improve SGD convergence and reduce variance for over-parameterized models.
problem Slower convergence in non-interpolation settings for SGD variants.
method Proposed AdaSPS and AdaSLS with variance reduction for robust convergence.
result Achieves faster convergence rates and robustness in non-interpolation settings.
Algorithm solves robust linear regression with block Lewis weights.
problem Group distributionally robust least squares problem.
method Algorithm based on geometric construction and block Lewis weights, using accelerated proximal methods.
result Improves over known methods for moderate accuracy regimes and matches state-of-the-art guarantees.
The paper proposes a method for generating uniform interpolations on data manifolds.
problem Generating high-quality interpolations between data samples on complex manifolds.
method Autoencoder network with interpolation network, regularized by a Riemannian metric.
result The method generates interpolations that remain within the manifold's distribution.