arXiv research

Warm-up improves training by adapting learning rate based on curvature.

Paper eliminates warm-up phase for PO in linear MDPs, achieving optimal regret.

Layer normalization placement affects training stability and warm-up stage necessity.

DP-SGD can update fewer coordinates while maintaining privacy.

Click-through rate (CTR) prediction has been one of the most central problems in computational advertising. Lately, embedding techniques that produce low-dimensional representations of ad IDs drastically improve CTR prediction accuracies. However, such learning techniques are data demanding and work poorly on new ads w…

Paper optimizes MVE network convergence and regularization.

Active learning tackles cold start and imbalanced data issues.

The paper shows how warming up the learning rate improves deep learning performance.

AdaScale SGD adapts learning rates for large-batch training efficiently.

Optimal learning rate schedules derived for various tasks.

Despite the advances in the representational capacity of approximate distributions for variational inference, the optimization process can still limit the density that is ultimately learned. We demonstrate the drawbacks of biasing the true posterior to be unimodal, and introduce Annealed Variational Objectives (AVO) in…

We provide a differential cocycle model for elliptic cohomology with complex coefficients and use analytic methods to construct a cocycle representative for the Witten class in this language. Our motivation stems from the conjectural connection between 2-dimensional field theories and elliptic cohomology originally due…

The paper provides examples of geometric transitions in low dimensions.

We prove that in CAT(0) spaces a quasi-geodesic is Morse if and only if it is contracting. Specifically, in our main theorem we prove that for $γ$ a quasi-geodesic in a CAT(0) space X, the following four statements are equivalent: (i) $γ$ is Morse, (ii) $γ$ is (b,c)--contracting, (iii), $γ$ is strongly contracting, and…

This work analyzes the statistical properties of adaptive gradient methods.

Variational Autoencoders are powerful models for unsupervised learning. However deep models with several layers of dependent stochastic variables are difficult to train which limits the improvements obtained using these highly expressive models. We propose a new inference model, the Ladder Variational Autoencoder, that…

SALSA automatically adjusts learning rates in stochastic gradient methods.

A new dynamic learning-rate scheme for optimization.

The aim of multi-task reinforcement learning is two-fold: (1) efficiently learn by training against multiple tasks and (2) quickly adapt, using limited samples, to a variety of new tasks. In this work, the tasks correspond to reward functions for environments with the same (or similar) dynamical models. We propose to l…

Although exploration in reinforcement learning is well understood from a theoretical point of view, provably correct methods remain impractical. In this paper we study the interplay between exploration and approximation, what we call approximate exploration. Our main goal is to further our theoretical understanding of …

The paper extends gradient flow and relaxation studies to non-flat Riemannian manifolds.

New algorithm finds critical points in non-convex optimization with heavy-tailed gradients.

Large-scale distributed training requires significant communication bandwidth for gradient exchange that limits the scalability of multi-node training, and requires expensive high-bandwidth network infrastructure. The situation gets even worse with distributed training on mobile devices (federated learning), which suff…

VCoTTA uses variational Bayesian methods to adapt models under continuous domain shifts.

Optimizes neural network training by dynamically updating Tucker decomposition ranks.

Paper proposes EEIPU, a memoization-aware BO algorithm to reduce hyperparameter tuning costs.

New algorithm speeds up Lasso computation by proving faster convergence.

Proposes efficient stochastic algorithms for optimizing NDCG with provable convergence guarantees.

SA-PEF improves federated learning efficiency by correcting gradient mismatches.

Improves neural network performance by dynamically adjusting model weights based on source reliability.

The large $k$ asymptotics (perturbation series) for integrals of the form $\int_{\cal F}μe^{i k S}$, where $μ$ is a smooth top form and $S$ is a smooth function on a manifold ${\cal F}$, both of which are invariant under the action of a symmetry group ${\cal G}$, may be computed using the stationary phase approximation…

Dual model combines HMM and neural networks for energy trading during volatile periods.

Proposes efficient training method for deep thin networks.

Introduces gradient decay in Softmax for better generalization.

We address the cold start problem in recommendation systems assuming no contextual information is available neither about users, nor items. We consider the case in which we only have access to a set of ratings of items by users. Most of the existing works consider a batch setting, and use cross-validation to tune param…

A new TS-SA method alleviates non-stationarity in TS algorithms for bandits.

VBS improves sampling efficiency in cosmological data analysis.

Study decision boundaries using heat diffusion and probabilistic techniques.

More and more companies have deployed machine learning (ML) clusters, where deep learning (DL) models are trained for providing various AI-driven services. Efficient resource scheduling is essential for maximal utilization of expensive DL clusters. Existing cluster schedulers either are agnostic to ML workload characte…

A novel framework IMBoost improves outlier detection by leveraging the inlier memorization effect.

Study on multi-head softmax attention dynamics for in-context learning.

Study of repeated principal-agent bandit game with self-interested and exploratory learning agents.

Survival analysis of 832,941 Solana token launches shows a significant decline in graduation rate.

A new framework for selecting base classes in multi-class classification boosts accuracy.

New method tightens federated probe-logit distillation rates under varying bandwidths.