A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
CLIP dataset helps extract action items from hospital discharge notes.
problem Lost action items in long discharge notes hinder information sharing.
method Created CLIP dataset of annotated discharge notes, used multi-aspect extractive summarization, trained models on pre-trained language models and context.
result Best models improved by incorporating context and pre-trained language models.
Paper proves convergence for private FL on non-Lipschitz convex objectives using normalization instead of clipping.
problem Lack of convergence results for differentially private federated learning with non-Lipschitz objectives.
method Developed a convergence result for private FL on smooth convex objectives without assuming Lipschitzness, using normalization instead of clipping.
result Normalization-based private FL algorithm converges better than clipping-based counterpart on smooth convex functions.
Symile learns joint representations across multiple modalities, outperforming pairwise CLIP.
problem Pairwise contrastive learning fails to capture joint information between multiple modalities.
method Symile uses a flexible, architecture-agnostic objective to learn modality-specific representations by deriving a lower bound on total correlation.
result Symile outperforms pairwise CLIP on cross-modal classification and retrieval across various datasets.
Natural gradient has been recently introduced to the field of boosting to enable the generic probabilistic predication capability. Natural gradient boosting shows promising performance improvements on small datasets due to better training dynamics, but it suffers from slow training speed overhead especially for large d…
This paper improves analysis of clipping algorithms for non-convex optimization.
problem Improving convergence analysis of clipping algorithms for non-convex optimization.
method Develops a general framework to study clipping algorithms, including momentum methods, and provides convergence analysis in deterministic and stochastic settings.
result Demonstrates that clipping methods can maintain efficiency even in highly non-smooth regions.
Existing approaches for training neural networks with user-level differential privacy (e.g., DP Federated Averaging) in federated learning (FL) settings involve bounding the contribution of each user's model update by clipping it to some constant value. However there is no good a priori setting of the clipping norm acr…
Clip21 improves convergence of gradient-clipped methods in DP settings.
problem Gradient clipping introduces bias in distributed training, causing convergence issues.
method Clip21 designs an error feedback mechanism to mitigate bias in gradient-clipped methods.
result Clip21 converges at the same rate as distributed gradient descent, improving from previous $\mathcal{O}\left(\frac{1}{\sqrt{K}}
ight)$ to $\mathcal{O}\left(\frac{1}{K}
ight)$.
In this paper, we show that a simple coloring scheme can improve, both theoretically and empirically, the expressive power of Message Passing Neural Networks(MPNNs). More specifically, we introduce a graph neural network called Colored Local Iterative Procedure (CLIP) that uses colors to disambiguate identical node att…
Many continuous control tasks have bounded action spaces. When policy gradient methods are applied to such tasks, out-of-bound actions need to be clipped before execution, while policies are usually optimized as if the actions are not clipped. We propose a policy gradient estimator that exploits the knowledge of action…
In importance sampling (IS)-based reinforcement learning algorithms such as Proximal Policy Optimization (PPO), IS weights are typically clipped to avoid large variance in learning. However, policy update from clipped statistics induces large bias in tasks with high action dimensions, and bias from clipping makes it di…
We consider the problem of minimizing a sum of clipped convex functions; applications include clipped empirical risk minimization and clipped control. While the problem of minimizing the sum of clipped convex functions is NP-hard, we present some heuristics for approximately solving instances of these problems. These h…
We consider the problem of recovering a low-rank matrix from its clipped observations. Clipping is conceivable in many scientific areas that obstructs statistical analyses. On the other hand, matrix completion (MC) methods can recover a low-rank matrix from various information deficits by using the principle of low-ran…