arXiv research

Improved natural gradient boosting with leaf number clipping for faster and better performance.

Paper proves convergence for private FL on non-Lipschitz convex objectives using normalization instead of clipping.

U-Clip clips gradients to reduce bias without discarding them.

Algorithm removes leaves to find root in uniform trees.

Improved differentially private deep learning with group-wise clipping techniques.

Study on gradient clipping in SGD for high-dimensional problems.

New method improves fairness in DP learning by preventing excessive gradient suppression.

This paper analyzes convergence of DP-SGD with adaptive quantile clipping.

A new method improves training speed and robustness for deep learning models.

CLIP models robustness to spurious features is re-evaluated using a new dataset.

CLIP learns joint image-text representations for zero-shot learning.

Paper introduces Latent-CLIP for efficient text-image comparison in latent space.

Gradient clipping helps private SGD converge despite potential bias.

Improves GBDT accuracy with differential privacy.

This paper improves analysis of clipping algorithms for non-convex optimization.

Enhances CLIP's similarity computation using PMI's linear structure.

A new algorithm reduces communication in deep learning training.

AutoClip automatically adjusts gradient clipping for better audio separation.

Dp-CLIP preserves privacy in multimodal AI training.

Classifies foliations with a hypersurface as the singular leaf and open leaves.

This paper tackles minimizing clipped convex functions with heuristics and mixed-integer convex programming.

This paper improves convergence guarantees for gradient clipping in deep learning.

Unified CLIP space manipulations improve GAN adaptation with a single target image.

Existing approaches for training neural networks with user-level differential privacy (e.g., DP Federated Averaging) in federated learning (FL) settings involve bounding the contribution of each user's model update by clipping it to some constant value. However there is no good a priori setting of the clipping norm acr…

Clip21 improves convergence of gradient-clipped methods in DP settings.

New method for estimating covariance with robustness to outliers.

Deep neural networks reveal a low-dimensional manifold structure in data.

INF-clip optimizes heavy-tailed MAB problems with improved performance.

Paper explores whether gradient normalization can replace clipping for SGD in heavy-tailed noise.

Many continuous control tasks have bounded action spaces. When policy gradient methods are applied to such tasks, out-of-bound actions need to be clipped before execution, while policies are usually optimized as if the actions are not clipped. We propose a policy gradient estimator that exploits the knowledge of action…

Privacy-preserving SGD with heavy-tailed noise achieves differential privacy guarantees.

In importance sampling (IS)-based reinforcement learning algorithms such as Proximal Policy Optimization (PPO), IS weights are typically clipped to avoid large variance in learning. However, policy update from clipped statistics induces large bias in tasks with high action dimensions, and bias from clipping makes it di…

TROLL improves RL for LLMs by replacing clipping with a trust region projection.

New analysis shows how to balance privacy and accuracy in deep learning.

Symile learns joint representations across multiple modalities, outperforming pairwise CLIP.

Robust CLIP improves vision models' resistance to attacks.

Theory for RLHF generalization under reward shift and clipped KL.

Enhanced metrics improve generative model evaluation reliability.

In this paper, we investigate the mean curvature flows starting from all non-minimal leaves of the isoparametric foliation given by a certain kind of solvable group action on a symmetric space of non-compact type. We prove that the mean curvature flow starting from each non-minimal leaf of the foliation exists in infin…

Improved Clipped-SGD achieves near-optimal heavy-tailed statistical estimation in streaming settings.

New optimization method combines gradient clipping and non-Euclidean smoothness.

We consider the problem of recovering a low-rank matrix from its clipped observations. Clipping is conceivable in many scientific areas that obstructs statistical analyses. On the other hand, matrix completion (MC) methods can recover a low-rank matrix from various information deficits by using the principle of low-ran…

New method improves convergence of SGD for heavy-tailed noise.

We prove that the boundary of an orbit space or more generally a leaf space of a singular Riemannian foliation is an Alexandrov space in its intrinsic metric, and that its lower curvature bound is that of the leaf space. A rigidity theorem for positively curved leaf spaces with maximal boundary volume is also establish…

Study controls bifurcations in Eulerian flows with multiple Hopf singularities.

We study the geometry of the leaf closure space of regular and singular Riemannian foliations. We give conditions which assure that this leaf space is a singular symplectic or Kähler space.

Riemannian manifolds can be realized as leaf spaces of matchbox manifolds.

Paper compares RL models for finance, finding Reward Clipping best.