arXiv research

Paper eliminates warm-up phase for PO in linear MDPs, achieving optimal regret.

DP-SGD can update fewer coordinates while maintaining privacy.

Click-through rate (CTR) prediction has been one of the most central problems in computational advertising. Lately, embedding techniques that produce low-dimensional representations of ad IDs drastically improve CTR prediction accuracies. However, such learning techniques are data demanding and work poorly on new ads w…

Warm-up improves training by adapting learning rate based on curvature.

The paper shows how warming up the learning rate improves deep learning performance.

The Transformer is widely used in natural language processing tasks. To train a Transformer however, one usually needs a carefully designed learning rate warm-up stage, which is shown to be crucial to the final performance but will slow down the optimization and bring more hyper-parameter tunings. In this paper, we fir…

Optimizes neural network training by dynamically updating Tucker decomposition ranks.

Study on multi-head softmax attention dynamics for in-context learning.

A new dynamic learning-rate scheme for optimization.

A novel framework IMBoost improves outlier detection by leveraging the inlier memorization effect.

VBS improves sampling efficiency in cosmological data analysis.

Paper optimizes MVE network convergence and regularization.

Active learning tackles cold start and imbalanced data issues.

SA-PEF improves federated learning efficiency by correcting gradient mismatches.

The large $k$ asymptotics (perturbation series) for integrals of the form $\int_{\cal F}μe^{i k S}$, where $μ$ is a smooth top form and $S$ is a smooth function on a manifold ${\cal F}$, both of which are invariant under the action of a symmetry group ${\cal G}$, may be computed using the stationary phase approximation…

AdaScale SGD adapts learning rates for large-batch training efficiently.

Optimal learning rate schedules derived for various tasks.

Despite the advances in the representational capacity of approximate distributions for variational inference, the optimization process can still limit the density that is ultimately learned. We demonstrate the drawbacks of biasing the true posterior to be unimodal, and introduce Annealed Variational Objectives (AVO) in…

More and more companies have deployed machine learning (ML) clusters, where deep learning (DL) models are trained for providing various AI-driven services. Efficient resource scheduling is essential for maximal utilization of expensive DL clusters. Existing cluster schedulers either are agnostic to ML workload characte…

We provide a differential cocycle model for elliptic cohomology with complex coefficients and use analytic methods to construct a cocycle representative for the Witten class in this language. Our motivation stems from the conjectural connection between 2-dimensional field theories and elliptic cohomology originally due…

The paper provides examples of geometric transitions in low dimensions.

We prove that in CAT(0) spaces a quasi-geodesic is Morse if and only if it is contracting. Specifically, in our main theorem we prove that for $γ$ a quasi-geodesic in a CAT(0) space X, the following four statements are equivalent: (i) $γ$ is Morse, (ii) $γ$ is (b,c)--contracting, (iii), $γ$ is strongly contracting, and…

This work analyzes the statistical properties of adaptive gradient methods.

We propose a statistical adaptive procedure called SALSA for automatically scheduling the learning rate (step size) in stochastic gradient methods. SALSA first uses a smoothed stochastic line-search procedure to gradually increase the learning rate, then automatically switches to a statistical method to decrease the le…

Variational Autoencoders are powerful models for unsupervised learning. However deep models with several layers of dependent stochastic variables are difficult to train which limits the improvements obtained using these highly expressive models. We propose a new inference model, the Ladder Variational Autoencoder, that…

Solves Dirichlet problem for Lagrangian phase equation with critical and supercritical phase.

New method tightens federated probe-logit distillation rates under varying bandwidths.

A major issue in harmonic analysis is to capture the phase dependence of frequency representations, which carries important signal properties. It seems that convolutional neural networks have found a way. Over time-series and images, convolutional networks often learn a first layer of filters which are well localized i…

New algorithms handle phase retrieval with rank d measurements, revealing phase transitions.

New estimators improve efficiency in two-phase designs with coarsened data.

Paper develops estimates for Lagrangian phase changes in 2D.

Machine learning predicts phase behavior in active matter suspensions.

In this paper, we perform statistical segmentation and clustering analysis of the Dow Jones Industrial Average time series between January 1997 and August 2008. Modeling the index movements and log-index movements as stationary Gaussian processes, we find a total of 116 and 119 statistically stationary segments respect…

Machine learning approximates phase transitions using Fisher information.

UPR hybrid model improves phase retrieval performance.

If a given behavior of a multi-agent system restricts the phase variable to a invariant manifold, then we define a phase transition as change of physical characteristics such as speed, coordination, and structure. We define such a phase transition as splitting an underlying manifold into two sub-manifolds with distinct…

The aim of multi-task reinforcement learning is two-fold: (1) efficiently learn by training against multiple tasks and (2) quickly adapt, using limited samples, to a variety of new tasks. In this work, the tasks correspond to reward functions for environments with the same (or similar) dynamical models. We propose to l…

The big phase space, the geometric setting for the study of quantum cohomology with gravitational descendents, is a complex manifold and consists of an infinite number of copies of the small phase space. The aim of this paper is to define a Hermitian geometry on the big phase space. Using the approach of Dijkgraaf and …

Diffusion maps help learn complex quantum phase transitions from data.

The paper models market crashes as phase transitions, finding dynamic transitions offer better predictions.

Unsupervised learning is a discipline of machine learning which aims at discovering patterns in big data sets or classifying the data into several categories without being trained explicitly. We show that unsupervised learning techniques can be readily used to identify phases and phases transitions of many body systems…

Common models for two-phase lipid bilayer membranes are based on an energy that consists of an elastic term for each lipid phase and a line energy at interfaces. Although such an energy controls only the length of interfaces, the membrane surface is usually assumed to be at least $C^1$ across phase boundaries. We consi…

Novel M-theory approach classifies topological phases of matter.

Consumers with low demand, like households, are generally supplied single-phase power by connecting their service mains to one of the phases of a distribution transformer. The distribution companies face the problem of keeping a record of consumer connectivity to a phase due to uninformed changes that happen. The exact…

Study oscillatory integrals with degenerate singular points in multivariable phase functions.

The stability of money value is an important requisite for a functioning economy, yet it critically depends on the actions of participants in the market themselves. Here we model the value of money as a dynamical variable that results from trading between agents. The basic trading scenario can be recast into an Ising t…

The classification of phase transitions is a central and challenging task in condensed matter physics. Typically, it relies on the identification of order parameters and the analysis of singularities in the free energy and its derivatives. Here, we propose an alternative framework to identify quantum phase transitions,…

The Allen-Cahn system on manifolds yields multiple phase distributions.