arXiv research

Mitigates faults in DNNs by clipping activation values, improving their resilience.

Paper proposes an active learning method for surgical workflow recognition using long-range temporal dependency.

In training speech recognition systems, labeling audio clips can be expensive, and not all data is equally valuable. Active learning aims to label only the most informative samples to reduce cost. For speech recognition, confidence scores and other likelihood-based active learning methods have been shown to be effectiv…

U-Clip clips gradients to reduce bias without discarding them.

Bayesian active learning method improved for censored regression data.

Improved differentially private deep learning with group-wise clipping techniques.

Study on gradient clipping in SGD for high-dimensional problems.

New method improves fairness in DP learning by preventing excessive gradient suppression.

This paper analyzes convergence of DP-SGD with adaptive quantile clipping.

A new method improves training speed and robustness for deep learning models.

CLIP models robustness to spurious features is re-evaluated using a new dataset.

CLIP learns joint image-text representations for zero-shot learning.

Paper introduces Latent-CLIP for efficient text-image comparison in latent space.

Gradient clipping helps private SGD converge despite potential bias.

This paper improves analysis of clipping algorithms for non-convex optimization.

Enhances CLIP's similarity computation using PMI's linear structure.

Quantization can improve the execution latency and energy efficiency of neural networks on both commodity GPUs and specialized accelerators. The majority of existing literature focuses on training quantized DNNs, while this work examines the less-studied topic of quantizing a floating-point model without (re)training. …

AutoClip automatically adjusts gradient clipping for better audio separation.

Dp-CLIP preserves privacy in multimodal AI training.

This paper improves convergence guarantees for gradient clipping in deep learning.

Unified CLIP space manipulations improve GAN adaptation with a single target image.

Existing approaches for training neural networks with user-level differential privacy (e.g., DP Federated Averaging) in federated learning (FL) settings involve bounding the contribution of each user's model update by clipping it to some constant value. However there is no good a priori setting of the clipping norm acr…

Clip21 improves convergence of gradient-clipped methods in DP settings.

New method for estimating covariance with robustness to outliers.

Paper proves convergence for private FL on non-Lipschitz convex objectives using normalization instead of clipping.

INF-clip optimizes heavy-tailed MAB problems with improved performance.

Paper explores whether gradient normalization can replace clipping for SGD in heavy-tailed noise.

S2D selectively decays large singular values to improve quantization of neural activations.

Many continuous control tasks have bounded action spaces. When policy gradient methods are applied to such tasks, out-of-bound actions need to be clipped before execution, while policies are usually optimized as if the actions are not clipped. We propose a policy gradient estimator that exploits the knowledge of action…

We propose Additive Powers-of-Two~(APoT) quantization, an efficient non-uniform quantization scheme for the bell-shaped and long-tailed distribution of weights and activations in neural networks. By constraining all quantization levels as the sum of Powers-of-Two terms, APoT quantization enjoys high computational effic…

In importance sampling (IS)-based reinforcement learning algorithms such as Proximal Policy Optimization (PPO), IS weights are typically clipped to avoid large variance in learning. However, policy update from clipped statistics induces large bias in tasks with high action dimensions, and bias from clipping makes it di…

TROLL improves RL for LLMs by replacing clipping with a trust region projection.

New analysis shows how to balance privacy and accuracy in deep learning.

In this paper, we consider the problem of recovering a sparse signal based on penalized least squares formulations. We develop a novel algorithm of primal-dual active set type for a class of nonconvex sparsity-promoting penalties, including $\ell^0$, bridge, smoothly clipped absolute deviation, capped $\ell^1$ and mini…

We consider the problem of minimizing a sum of clipped convex functions; applications include clipped empirical risk minimization and clipped control. While the problem of minimizing the sum of clipped convex functions is NP-hard, we present some heuristics for approximately solving instances of these problems. These h…

Robust CLIP improves vision models' resistance to attacks.

Theory for RLHF generalization under reward shift and clipped KL.

Enhanced metrics improve generative model evaluation reliability.

New algorithm balances personalization and statistical validity in MRTs.

Improved Clipped-SGD achieves near-optimal heavy-tailed statistical estimation in streaming settings.

New optimization method combines gradient clipping and non-Euclidean smoothness.

We consider the problem of recovering a low-rank matrix from its clipped observations. Clipping is conceivable in many scientific areas that obstructs statistical analyses. On the other hand, matrix completion (MC) methods can recover a low-rank matrix from various information deficits by using the principle of low-ran…

New method improves convergence of SGD for heavy-tailed noise.

Paper compares RL models for finance, finding Reward Clipping best.

Fast algorithm for rescaling vectors with clipping, improving training efficiency.

PPOS improves PPO by smoothing the surrogate objective function.

Study finds CLIP's caption-based learning outperforms image-only methods under certain conditions.

New algorithm robustly optimizes data streams with heavy-tailed or infinite variance samples.