Improved k-NN active learning with local smoothness assumption.
problem Active learning convergence rates under smoothness assumptions.
method Designing an active learning algorithm with better convergence rate using local smoothness assumption for k-NN.
result Better convergence rate than in passive learning.
GPCDL uses Gaussian Processes to learn smooth templates from data.
problem Lack of smoothness in learned templates leads to overfitting and poor predictive performance.
method GPCDL incorporates Gaussian Process priors to enforce smoothness in the learned templates.
result GPCDL outperforms unregularized CDL in accuracy and predictive performance across various SNRs and applications.
New neural network smoothness constraints improve model performance.
problem Improving model sensitivity to input changes for better generalization and robustness.
method Exploring current smoothness constraints and proposing new flexible definitions.
result Current smoothness constraints lack flexibility and understanding of data, tasks, and learning.
New method improves RL in continuous spaces with kernel smoothing.
problem Sample efficiency and structural assumptions in classical RL.
method Kernel smoothing model-based approach with Bernstein-style exploration bonus.
result Achieves improved regret bound in finite-horizon settings.
New algorithms optimize non-smooth, non-convex objectives with improved complexity.
problem Optimizing non-smooth, non-convex stochastic objectives.
method Reduction to online learning, applying optimistic online learning techniques.
result Improved complexity for finding ( δ , ε ) (δ,ε) ( δ , ε ) -stationary points. Improved learning rates with new smoothness measure.
problem Learning with noisy data and unknown function class.
method Generalized Hölder smoothness to average smoothness, proving upper and lower bounds.
result Achieved nearly optimal learning rates in realizable and agnostic settings.
AsylADMM improves gossip-based learning for non-smooth objectives.
problem Efficient and robust decentralized learning on edge devices.
method Asynchronous gossip algorithm for non-smooth optimization.
result AsylADMM converges faster on non-smooth problems.
New bounds for agnostic learning with average smoothness.
problem Distribution-free nonparametric regression with average smoothness.
method Distribution-free uniform convergence bounds and agnostic learning algorithm.
result Distribution-free uniform convergence bounds for average-smoothness classes in the agnostic setting.
SATL adapts to varying smoothness in hypothesis transfer learning.
problem Fixed kernel regularization fails in varying smoothness settings.
method Proposes SATL, a two-phase KRR algorithm with adaptive Gaussian kernels.
result SATL achieves minimax optimality with matching upper and lower bounds.
This study improves scalability of randomized smoothing for certifying classifier robustness.
problem Certifying machine learning classifiers against adversarial attacks is challenging and scalable solutions are needed.
method The study reviews and explores randomized smoothing and its derivatives, focusing on scalability.
result The study provides theoretical guarantees and discusses scalability challenges of randomized smoothing.
This paper shows how many samples are needed for smooth functions in high dimensions.
problem The challenge of obtaining meaningful estimates of high-order derivatives in machine learning with limited data.
method Deriving new lower bounds on the generalization error.
result Formalizes the intuition that smoothness requires enough samples close to each other.
New algorithm solves federated minimax optimization problems.
problem Federated minimax optimization challenges.
method Federated Stochastic Smoothed Gradient Descent Ascent (FESS-GDA).
result FESS-GDA uniformly solves federated minimax problems.
New framework improves sample efficiency and robustness in RL with smooth policies.
problem Sample inefficiency and lack of robustness in deep reinforcement learning.
method SR^2L framework, smoothness-inducing regularization.
result Improved sample efficiency and robustness in both on-policy and off-policy RL algorithms.
New algorithm learns halfspaces over hypercube with random bit flips.
problem Agnostic learning of Boolean halfspaces over discrete domains is computationally hard.
method Smoothed analysis with random bit flips for discrete inputs.
result First efficient algorithm for smoothed agnostic learning of halfspaces over Boolean hypercube.
New algorithm tackles smooth online learning with optimal regret.
problem Smoothed online learning with adversarial distributions.
method Oracle-efficient algorithms for nonparametric function classes.
result Oracle-efficient algorithms achieve optimal regret bounds.
Post-estimation smoothing improves prediction accuracy with structural indices.
problem Using natural structural indices in machine learning without losing robustness.
method A post-estimation smoothing operator that separates from the original predictor.
result Post-estimation smoothing improves accuracy over original predictors under simple conditions.
Unified smoothing for robust classification improves accuracy.
problem Improving robustness of classifiers against adversarial attacks.
method Learned smoothed densities and randomized smoothing.
result Provable robust accuracies higher than state-of-the-art defenses.
Label smoothing improves model performance even with noisy labels.
problem Mitigating label noise in deep learning models.
method Examined label smoothing as a technique to cope with label noise and compared it to loss-correction methods.
result Label smoothing is competitive with loss-correction techniques under label noise and beneficial for distillation from noisy data.
Deep learning has exhibited superior performance for various tasks, especially for high-dimensional datasets, such as images. To understand this property, we investigate the approximation and estimation ability of deep learning on anisotropic Besov spaces. The anisotropic Besov space is characterized by direction-depen…
ConquerNet smooths quantile regression for deep learning with minimax guarantees.
problem Optimization challenges in quantile regression for deep models.
method ConquerNet uses convolution-smoothed quantile ReLU neural networks.
result ConquerNet provides minimax guarantees and outperforms standard quantile neural networks.
Deep ReLU networks can approximate and learn smooth functions efficiently.
problem Efficiently approximating and learning smooth functions using deep ReLU neural networks.
method Extending recent results to anisotropic and mixed smooth function classes, establishing approximation rates.
result Deep ReLU networks achieve minimax optimal rates up to logarithmic factors for various smooth function classes.
Hypergraph is a general way of representing high-order relations on a set of objects. It is a generalization of graph, in which only pairwise relations can be represented. It finds applications in various domains where relationships of more than two objects are observed. On a hypergraph, as a generalization of graph, o…
New algorithm tackles nonconvex machine learning problems with adaptive normalization and independent sampling.
problem Nonconvex machine learning problems with generalized-smoothness.
method Adaptive gradient normalization, independent sampling, and gradient clipping.
result Achieves an O(ε^(-4)) sample complexity for fast convergence.
Enhances privacy in federated learning with Laplacian smoothing.
problem Protecting data privacy in federated learning while maintaining model accuracy.
method Laplacian smoothing for differentially private federated learning (DP-Fed-LS).
result Improves model accuracy with differential privacy guarantee and membership privacy.
Smooth activations enable optimal error rates in neural networks for Sobolev function classes.
problem Achieving optimal approximation and estimation error rates for neural networks in Sobolev function classes.
method Study of neural networks with smooth activations, proving optimal rates via approximation and statistical properties.
result Constant-depth networks with smooth activations achieve optimal rates of approximation and estimation, demonstrating smoothness adaptivity.
SIXO improves inference by learning smoothing distributions from all observations.
problem Inference limitations due to ignoring future observations in filtering distributions.
method Density ratio estimation to warp filtering distributions into smoothing distributions, then use SMC with learned targets.
result Proves tighter log marginal lower bounds and more accurate inferences and estimates.
New SQ lower bound shows complexity nearly matches known upper bound for smoothed agnostic learning.
problem Smoothed agnostic learning of halfspaces under subgaussian distributions.
method Statistical Query (SQ) lower bound using moment-matching hard distribution and linear programming duality.
result First non-trivial lower bound on complexity nearly matches known upper bound.
The paper analyzes the trade-off between smoothness and sparsity in GCN using lp-regularized learning.
problem Quantifying the trade-off between smoothness and sparsity in GCN.
method Proposes a novel SGD proximal algorithm for GCNs with an inexact operator to analyze the stability of the ℓ p \ell_p ℓ p -regularized stochastic learning. result Establishes an explicit theoretical understanding of GCN with ℓ p \ell_p ℓ p -regularized stochastic learning. DPlis improves privacy in deep learning models by smoothing loss functions.
problem Privacy leakage in deep learning models trained on private data and low model performance.
method DPlis constructs a smooth loss function to favor noise-resilient models.
result DPlis effectively boosts model quality and training stability under privacy constraints.
Recent works have shown the effectiveness of randomized smoothing as a scalable technique for building neural network-based classifiers that are provably robust to ℓ 2 \ell_2 ℓ 2 -norm adversarial perturbations. In this paper, we employ adversarial training to improve the performance of randomized smoothing. We design an adap…
Randomized smoothing reduces accuracy in ML models, especially at higher noise levels.
problem Adversarial attacks on ML models, especially randomized smoothing's accuracy drop.
method Theoretical and empirical analysis of randomized smoothing's effect on feasible hypotheses space.
result For some noise levels, randomized smoothing shrinks the set of feasible hypotheses, leading to accuracy drops.
New method improves counterfactual distribution learning for high-dimensional outcomes.
problem Counterfactual distribution learning for high-dimensional outcomes with concentrated structure.
method Geometry-adaptive diffusion-guided smoothing estimators combining causal nuisance adjustment and local outcome geometry.
result Geometry-adaptive methods show steeper error decay in semi-synthetic experiments.
Paper learns hypergraph structures from signals with smoothness priors.
problem Learning hypergraph structures from signals with high-order relationships.
method Proposes HGSL framework with dual smoothness prior to map signals to hypergraph structure.
result HGSL efficiently infers meaningful hypergraph topologies from signals.
There is a large body of work on convergence rates either in passive or active learning. Here we outline some of the results that have been obtained, more specifically in a nonparametric setting under assumptions about the smoothness and the margin noise. We also discuss the relative merits of these underlying assumpti…
Hidden cost: Smoothing shrinks decision boundaries, affecting class-wise accuracy.
problem The fragility of machine learning models and the need for robustness verification.
method Randomized smoothing approach to achieve statistical robustness.
result Smoothed classifiers' decision boundaries shrink, leading to class-wise accuracy disparity.
Robust learning method combines kernel smoothing and robust optimization.
problem Certifying robustness against distribution shifts in machine learning models.
method Adapting integral operator using supremal convolution for robustness, leveraging optimal transport.
result The method provides theoretical guarantees for certified robustness and competitive performance.
New findings show learning deeper neural networks is hard even with Gaussian inputs and non-degenerate weights.
problem The computational complexity of learning neural networks, especially deeper ones.
method Smoothed analysis framework and local pseudorandom generators.
result Learning depth-3 ReLU networks under Gaussian input distribution is hard even if weight matrices are non-degenerate.
MARINA-P improves non-smooth federated optimization with adaptive stepsizes.
problem Non-smooth federated optimization in machine learning applications.
method Extends EF21-P and MARINA-P to non-smooth convex setting, proving optimal convergence rate and communication complexity bounds.
result MARINA-P achieves O ( 1 / T ) O(1/\sqrt{T}) O ( 1/ T ) convergence rate and communication complexity matching classical subgradient methods. Smooth calibration improves forecast reliability even with leaked information.
problem Improving forecast reliability with leaked information.
method Combining nearby forecasts to ensure smooth calibration, which can be guaranteed by deterministic procedures.
result Smooth calibration can be guaranteed by deterministic procedures even with leaked forecasts, and it yields uncoupled finite-memory dynamics in games.
NAS-X improves inference and model learning for SLVMs.
problem Challenges in analytic inference and model learning for flexible SLVMs.
method NAS-X combines reweighted wake-sleep and smoothing sequential Monte Carlo.
result NAS-X provides low-bias and low-variance gradient estimates.
New methods improve convergence in non-convex non-smooth learning problems.
problem Sparse learning from high-dimensional data with non-convex, non-smooth regularizers.
method Stochastic proximal gradient methods with arbitrary sampling.
result Independent sampling improves performance over uniform sampling.
Decision tree learning heuristics fail even in smoothed analysis for complex targets.
problem Greedy decision tree learning heuristics fail for complex target functions in the smoothed analysis model.
method Construct counterexamples and analyze the behavior of heuristics in the smoothed setting and agnostic setting.
result Greedy decision tree learning heuristics can build trees of exponential depth before achieving high accuracy for certain complex target functions.
Improved learning algorithms with privacy using smoothed analysis.
problem Designing robust and private learning algorithms.
method Smoothed analysis of adversarial and differentially private learning.
result Stronger regret and privacy error guarantees with smoothed adversaries.
Optimizes graph spectral density learning for large networks.
problem Ad-hoc kernel function and bandwidth selection in graph spectral techniques.
method Maximum Entropy approach to learn a smooth graph spectral density.
result Outperforms comparable iterative spectral approaches on synthetic and real graphs.
Safe-EF improves federated learning for non-smooth, constrained optimization.
problem Federated learning's communication bottlenecks with high-dimensional model updates.
method Error feedback (EF) for non-smooth convex optimization with safety constraints.
result Safe-EF matches lower complexity bounds and ensures safety constraints.
The paper tackles fair representation learning by smoothing feature mappings.
problem Legal liability for discriminatory use of data by organizations.
method Mapping features to a fair representation space, certifying fairness through chi-squared mutual information.
result Smoothing representation distribution provides generalization guarantees of fairness and maintains accuracy for downstream tasks.
Proposes a neural network autoencoder for smoothing and representation learning of functional data.
problem Lack of sufficient nonlinear representations in existing methods for functional data analysis.
method Develops a neural network autoencoder architecture to process functional data directly, learning both smoothing and representation.
result Outperforms traditional methods in prediction, classification, and computational efficiency.
New method smooths optimization for sparse regularization.
problem Non-smooth, non-convex optimization problems for sparsity.
method Overparameterization and smooth surrogate penalties.
result Surrogate objective has identical global and local minima.