arXiv research

Paper eliminates warm-up phase for PO in linear MDPs, achieving optimal regret.

DP-SGD can update fewer coordinates while maintaining privacy.

The paper shows how warming up the learning rate improves deep learning performance.

Click-through rate (CTR) prediction has been one of the most central problems in computational advertising. Lately, embedding techniques that produce low-dimensional representations of ad IDs drastically improve CTR prediction accuracies. However, such learning techniques are data demanding and work poorly on new ads w…

Warm-up improves training by adapting learning rate based on curvature.

Layer normalization placement affects training stability and warm-up stage necessity.

Robust algorithm optimizes corrupted Gaussian process bandits.

Optimizes neural network training by dynamically updating Tucker decomposition ranks.

Study on multi-head softmax attention dynamics for in-context learning.

This paper explains a mechanism called phase collapse that improves image classification accuracy.

How can we control for latent discrimination in predictive models? How can we provably remove it? Such questions are at the heart of algorithmic fairness and its impacts on society. In this paper, we define a new operational fairness criteria, inspired by the well-understood notion of omitted variable-bias in statistic…

A new dynamic learning-rate scheme for optimization.

A novel framework IMBoost improves outlier detection by leveraging the inlier memorization effect.

Paper tackles non-stationary kernelized bandits with near-optimal algorithm.

Study optimizes best-arm identification with minimax and Bayes strategies.

The training phases of Deep neural network~(DNN) consumes enormous processing time and energy. Compression techniques utilizing the sparsity of DNNs can effectively accelerate the inference phase of DNNs. However, it can be hardly used in the training phase because the training phase involves dense matrix-multiplicatio…

Study of repeated principal-agent bandit game with self-interested and exploratory learning agents.

VBS improves sampling efficiency in cosmological data analysis.

We propose an algorithm for incremental learning of classifiers. The proposed method enables an ensemble of classifiers to learn incrementally by accommodating new training data. We use an effective mechanism to overcome the stability-plasticity dilemma. In incremental learning, the general convention is to use only th…

Paper optimizes MVE network convergence and regularization.

Discontinuous Finite Element Methods (DFEM) have been widely used for solving $S_n$ radiation transport problems in participative and non-participative media. In the DFEM $S_n$ methodology, the transport equation is discretized into a set of algebraic equations that have to be solved for each spatial cell and angular d…

Recent state-of-the-art language models utilize a two-phase training procedure comprised of (i) unsupervised pre-training on unlabeled text, and (ii) fine-tuning for a specific supervised task. More recently, many studies have been focused on trying to improve these models by enhancing the pre-training phase, either vi…

The paper tackles fair sequential decision making with biased linear bandit feedback.

Deep heteroskedastic models overfit, showing a phase transition with regularization strength.

We describe a bottom-up framework, based on the identification of appropriate order parameters and determination of phase diagrams, for understanding progressively refined agent-based models and simulations of financial markets. We illustrate this framework by starting with a deterministic toy model, whereby $N$ indepe…

RW-based learning is vulnerable to the Pac-Man attack, which eliminates active RWs.

Automates phased release strategy to balance risk and speed.

Active learning tackles cold start and imbalanced data issues.

Algorithm identifies the best arm in linear bandits with high probability.

SA-PEF improves federated learning efficiency by correcting gradient mismatches.

A new feature selection method using random forest and Kolmogorov filter.

This paper proposes a new AED framework for multi-metric experiments with fixed budget.

The large $k$ asymptotics (perturbation series) for integrals of the form $\int_{\cal F}μe^{i k S}$, where $μ$ is a smooth top form and $S$ is a smooth function on a manifold ${\cal F}$, both of which are invariant under the action of a symmetry group ${\cal G}$, may be computed using the stationary phase approximation…

AdaScale SGD adapts learning rates for large-batch training efficiently.

Optimal learning rate schedules derived for various tasks.

Despite the advances in the representational capacity of approximate distributions for variational inference, the optimization process can still limit the density that is ultimately learned. We demonstrate the drawbacks of biasing the true posterior to be unimodal, and introduce Annealed Variational Objectives (AVO) in…

More and more companies have deployed machine learning (ML) clusters, where deep learning (DL) models are trained for providing various AI-driven services. Efficient resource scheduling is essential for maximal utilization of expensive DL clusters. Existing cluster schedulers either are agnostic to ML workload characte…

PROBE optimizes best-arm identification with cheap proxies, improving sample complexity.

New algorithm for collaborative bandit learning reduces sample complexity and regret.

Safe-DRFS selects features robust to covariate shifts for reliable performance.

Two efficient algorithms improve online item recommendation for large user-item matrices.

Optimization-based pruning eliminates backpropagation for large language models.

We provide a differential cocycle model for elliptic cohomology with complex coefficients and use analytic methods to construct a cocycle representative for the Witten class in this language. Our motivation stems from the conjectural connection between 2-dimensional field theories and elliptic cohomology originally due…

Proposes a flexible tournament design combining knockout and round-robin.

Optimal learning rates decay to zero in easy tasks and maintain a warmup phase in hard tasks.

UCoS avoids forward model evaluations in sampling for large-scale linear inverse problems.

Multi-agent reinforcement learning (MARL) has recently received considerable attention due to its applicability to a wide range of real-world applications. However, achieving efficient communication among agents has always been an overarching problem in MARL. In this work, we propose Variance Based Control (VBC), a sim…

Improved elimination strategies for adaptive bandit identification reduce sample complexity and computational burden.