Self-distillation improves model performance by increasing teacher diversity and smoothing predictions.
problem Improving model generalization and performance through self-distillation.
method Interpreting self-distillation as MAP estimation and proposing instance-specific label smoothing.
result Self-distillation enhances model performance by increasing teacher diversity and smoothing predictions.
Enhances random forests by smoothing predictions for better performance.
problem Suboptimal performance due to piecewise constant predictions in random forests.
method Kernel-based smoothing mechanism to introduce local regularity.
result Smoothed random forest model consistently improves predictive performance.
New method improves certified robustness for classifier confidence.
problem Certifying confidence in classifier predictions.
method Randomized smoothing with modified Neyman-Pearson lemma.
result Certified radii for prediction confidence improved.
Proposes SCD-split for CP to balance interpretability and efficiency.
problem Difficult interpretation of disconnected subintervals in CP prediction sets.
method Incorporates smoothing operations into CP framework.
result SCD-split balances interval length and subinterval number, theoretically provable.
Post-estimation smoothing improves prediction accuracy with structural indices.
problem Using natural structural indices in machine learning without losing robustness.
method A post-estimation smoothing operator that separates from the original predictor.
result Post-estimation smoothing improves accuracy over original predictors under simple conditions.
New metric improves latent dynamics inference from neural data.
problem Limitations of co-smoothing in predicting latent dynamics.
method Few-shot co-smoothing to assess latent dynamics.
result High co-smoothing models often have extraneous dynamics, which few-shot co-smoothing detects.
SWEEN improves certified robustness via weighted ensembling of smoothed classifiers.
problem Finding optimal base classifiers for randomized smoothing.
method Smoothed WEighted ENsembling (SWEEN) scheme.
result SWEEN achieves optimal certified robustness under mild assumptions.
Conditional COT-GAN predicts sequences using past data and kernel smoothing.
problem Predicting sequences given past data.
method Conditional COT-GAN with kernel smoothing.
result Improved convergence results for sequence prediction.
We present a framework to train a structured prediction model by performing smoothing on the inference algorithm it builds upon. Smoothing overcomes the non-smoothness inherent to the maximum margin structured prediction objective, and paves the way for the use of fast primal gradient-based optimization algorithms. We …
New algorithm speeds up RNN time series prediction by filtering noise.
problem Predicting smooth trajectories from noisy time series data.
method Analyzed RNN dynamics to propose an efficient noise filtering algorithm.
result Significant speedup in predictive process without accuracy loss.
Smoothing graphons improve link prediction in Bayesian SBM without increasing computational complexity.
problem Accurate modeling of exchangeable relational data with flexible and computationally efficient graphons.
method Introducing smoothing procedures to piecewise-constant graphons to create smoothing graphons, which allow continuous intensity values for relations.
result Smoothing graphons improve AUC and precision for link prediction in real-world data sets.
Study of filtering and smoothing in submanifolds of Euclidean space.
problem Filtering and smoothing in continuous-discrete time on submanifolds.
method Formal expressions and projection approach for prediction and smoothing.
result Agreement with classical results for prediction, differences for smoothing.
Stochastic gradient methods are dominant in nonconvex optimization especially for deep models but have low asymptotical convergence due to the fixed smoothness. To address this problem, we propose a simple yet effective method for improving stochastic gradient methods named predictive local smoothness (PLS). First, we …
Linear-Core Surrogates combine fast optimization and statistical efficiency in classification and structured prediction.
problem The trade-off between smoothness and margin-based losses in classification and structured prediction.
method Linear-Core (LC) Surrogates, a family of convex loss functions that stitch a linear core to a smooth tail.
result LC Surrogates achieve fast linear consistency rates while maintaining differentiability and strict H-consistency bounds. It is well-known that classifiers are vulnerable to adversarial perturbations. To defend against adversarial perturbations, various certified robustness results have been derived. However, existing certified robustnesses are limited to top-1 predictions. In many real-world applications, top-k predictions are more rel…
GPCDL uses Gaussian Processes to learn smooth templates from data.
problem Lack of smoothness in learned templates leads to overfitting and poor predictive performance.
method GPCDL incorporates Gaussian Process priors to enforce smoothness in the learned templates.
result GPCDL outperforms unregularized CDL in accuracy and predictive performance across various SNRs and applications.
Label smoothing improves generalization by controlling generalization loss.
problem Lack of mathematical understanding of label smoothing's effectiveness.
method Proposed a theoretical framework to show how label smoothing controls generalization loss in the label noise setting.
result Predicted an optimal label smoothing point that minimizes generalization loss.
Improved online classification with accurate predictions.
problem Online classification challenges with limited data.
method Designing an online learner that uses predictions to reduce regret.
result Expected regret is better than worst-case analysis, especially with accurate predictions.
This thesis predicts the distribution of smoothed zeros of random sections on line bundles.
problem Predicting the distribution of smoothed zeros of random sections on line bundles.
method Developing smoothing operators on discrete surfaces and computing the expected sum of indices on each face.
result Predictions on the distribution of smoothed section's signed zeros with multiplicity.
Regularization improves robustness of smoothed classifiers.
problem Certifying robustness of smoothed classifiers.
method Regularizing prediction consistency over Gaussian noise.
result Significantly improved certified robustness with less training costs.
Fusion of robustness and uncertainty techniques improves adversarial defense.
problem Adversarial attacks on deep neural networks.
method Integrating uncertainty quantification into randomized smoothing for robustness guarantees.
result Improved robustness guarantees for uncertainty aware classifiers.
Prediction of dynamical time series with additive noise using support vector machines or kernel based regression has been proved to be consistent for certain classes of discrete dynamical systems. Consistency implies that these methods are effective at computing the expected value of a point at a future time given the …
In this work we provide a theoretical framework for structured prediction that generalizes the existing theory of surrogate methods for binary and multiclass classification based on estimating conditional probabilities with smooth convex surrogates (e.g. logistic regression). The theory relies on a natural characteriza…
Smooth embedding space improves NAS performance.
problem Efficiently predicting good neural architectures.
method Two-sided variational graph autoencoder.
result Smooth embedding space facilitates extrapolation to unseen architectures.
The estimation of probabilities of network edges from the observed adjacency matrix has important applications to predicting missing links and network denoising. It has usually been addressed by estimating the graphon, a function that determines the matrix of edge probabilities, but this is ill-defined without strong a…
A new method for embedding temporal relationships in graphs.
problem Limited performance of existing time-aware graph embedding methods.
method Integrates temporal smoothness and task-oriented negative sampling.
result Improves performance in various tasks, especially entity/relationship/temporal scoping prediction.
Paper explains how tree ensembles improve predictions by smoothing and regulating smoothness.
problem Understanding why tree ensembles perform well despite their complexity.
method Interpreting tree ensembles as adaptive and self-regularizing smoothers.
result Ensemble trees make more smooth predictions than individual trees and adjust smoothness based on input dissimilarity.
Bagging is a device intended for reducing the prediction error of learning algorithms. In its simplest form, bagging draws bootstrap samples from the training sample, applies the learning algorithm to each bootstrap sample, and then averages the resulting prediction rules. We extend the definition of bagging from stati…
The use of machine-learning in neuroimaging offers new perspectives in early diagnosis and prognosis of brain diseases. Although such multivariate methods can capture complex relationships in the data, traditional approaches provide irregular (l2 penalty) or scattered (l1 penalty) predictive pattern with a very limited…
New framework certifies robustness for regression models.
problem Certifying robustness for regression models is challenging.
method Derives a prediction-centered certificate that exploits local geometry.
result Gradient information yields tighter robustness certificates.
Dynamic programming (DP) solves a variety of structured combinatorial problems by iteratively breaking them down into smaller subproblems. In spite of their versatility, DP algorithms are usually non-differentiable, which hampers their use as a layer in neural networks trained by backpropagation. To address this issue,…
Spatial smoothing improves BNNs' accuracy, uncertainty, and robustness without increasing computational cost.
problem Large ensembles in BNNs increase computational cost and reduce performance.
method Spatial smoothing adds blur layers to convolutional neural networks to ensemble neighboring feature map points.
result Spatial smoothing improves BNNs' performance with fewer ensembles and enhances robustness.
The paper provides a uniform convergence bound for smooth calibration error and its relationship with functional gradient.
problem Limited theoretical understanding of learning algorithms achieving high accuracy and good calibration.
method Focuses on smooth calibration error, providing a uniform convergence bound and proving the relationship with functional gradient.
result Derives conditions for simultaneous classification and calibration guarantees in gradient boosting trees, kernel boosting, and neural networks.
In modeling multivariate time series, it is important to allow time-varying smoothness in the mean and covariance process. In particular, there may be certain time intervals exhibiting rapid changes and others in which changes are slow. If such time-varying smoothness is not accounted for, one can obtain misleading inf…
Paper develops algorithms for PWA systems with polynomial regret.
problem Learning in piecewise affine systems due to discontinuities.
method Smoothed online learning framework applied to PWA systems.
result First algorithms with polynomial regret in PWA systems.
Predictive models can be used on high-dimensional brain images for diagnosis of a clinical condition. Spatial regularization through structured sparsity offers new perspectives in this context and reduces the risk of overfitting the model while providing interpretable neuroimaging signatures by forcing the solution to …
Adaptive algorithm minimizes online prediction errors for irregular data.
problem Online adversarial regression with highly irregular prediction rules.
method Adaptive wavelet-based algorithm for Besov space regression.
result Minimax-optimal regret bounds in adversarial settings.
New method improves Kalman filtering and smoothing for large state spaces.
problem High computational cost and uncertainty in large-scale Kalman filtering.
method Probabilistic numerical method leveraging GPU acceleration and tunable trade-off.
result Mitigates scaling issues and provides more accurate uncertainty estimates.
Label smoothing improves model performance even with noisy labels.
problem Mitigating label noise in deep learning models.
method Examined label smoothing as a technique to cope with label noise and compared it to loss-correction methods.
result Label smoothing is competitive with loss-correction techniques under label noise and beneficial for distillation from noisy data.
Study on optimizing model updates in performative prediction.
problem Optimizing model updates influenced by model predictions.
method Stochastic optimization with greedy and lazy deploy approaches.
result Rates of convergence for both greedy and lazy deploy methods.
Backdoor attacks make models predict a specific class near triggers, smoothing their decision function.
problem Understanding and mitigating backdoor attacks on deep neural networks.
method Defined a measure to quantify backdoor smoothing and detected other smoothing patterns.
result Backdoor attacks induce a smoother decision function around triggered samples.
New calibration measure SSCE ensures truthful prediction, unlike existing measures.
problem Ensuring truthful calibration measures in sequential prediction.
method Introduced a new calibration measure, Subsampled Smooth Calibration Error (SSCE).
result SSCE ensures truthful prediction, while existing measures are far from truthful.
Bayesian GAMs improve predictive performance for high-dimensional data.
problem Sparse regularization in GAMs leads to excess shrinkage and difficulty in selecting nonlinear effects.
method Developed a novel spike-and-slab LASSO prior and scalable EM-Coordinate Descent algorithm.
result Improved predictive and computational performance compared to existing models.
DMIDAS improves long-term forecasting accuracy in healthcare and electricity data.
problem Challenging long-term forecasting accuracy and computational complexity.
method Smoothness regularization and mixed data sampling techniques integrated into NBEATS architecture.
result Improves prediction accuracy by 5% on long forecasting horizons (1000 timestamps) compared to state-of-the-art models.
Midicoth compresses online probability estimates by correcting prior smoothing biases.
problem Compression inefficiency due to prior smoothing in adaptive models.
method Micro-diffusion denoising applied in a bitwise tree hierarchy.
result Significant compression improvement with reliable calibration.
TATD predicts missing entries in time-evolving tensors by exploiting temporal dependency and sparsity.
problem Predict missing entries in time-evolving tensors with temporal dependency and sparsity issues.
method TATD (Time-Aware Tensor Decomposition) integrates temporal dependency and time-varying sparsity through a smoothing regularization with Gaussian kernel and alternating optimization.
result TATD achieves state-of-the-art accuracy for decomposing temporal tensors.
New method calibrates multi-class predictions efficiently without sacrificing accuracy.
problem Efficiently calibrating multi-class predictions without sacrificing accuracy.
method Formulated robust projected smooth calibration and new recalibration algorithms.
result Achieves strong guarantees for binary classification tasks with polynomial complexity.
Graphs can be fooled by small edge changes, but this work protects them.
problem Adversaries can manipulate graph data to mislead graph classification models.
method We introduce a smoothed graph classification model with a robustness guarantee.
result The smoothed model maintains consistent predictions under small adversarial perturbations.