arXiv research

A simple modification improves GAN performance by discarding bad samples.

We present online boosting algorithms for multilabel ranking with top-k feedback, where the learner only receives information about the top k items from the ranking it provides. We propose a novel surrogate loss function and unbiased estimator, allowing weak learners to update themselves with limited information. Using…

A new energy-efficient pruning method for federated learning.

Variance reduction methods such as SVRG and SpiderBoost use a mixture of large and small batch gradients to reduce the variance of stochastic gradients. Compared to SGD, these methods require at least double the number of operations per update to model parameters. To reduce the computational cost of these methods, we i…

Paper analyzes trade-offs in top-k classification accuracies and proposes a new loss function.

Class ambiguity is typical in image classification problems with a large number of classes. When classes are difficult to discriminate, it makes sense to allow k guesses and evaluate classifiers based on the top-k error instead of the standard zero-one loss. We propose top-k multiclass SVM as a direct method to optimiz…

The top-$k$ error is often employed to evaluate performance for challenging classification tasks in computer vision as it is designed to compensate for ambiguity in ground truth labels. This practical success motivates our theoretical analysis of consistent top-$k$ classification. Surprisingly, it is not rigorously und…

Paper introduces a new loss function for deep imbalanced classification.

Smoothed top-k operator improves model training efficiency.

Paper introduces efficient top-k selection with differential privacy.

Paper tackles ranking items with a semi-random comparison graph and a monotone adversary.

Motivated by applications in recommender systems, web search, social choice and crowdsourcing, we consider the problem of identifying the set of top $K$ items from noisy pairwise comparisons. In our setting, we are non-actively given $r$ pairwise comparisons between each pair of $n$ items, where each comparison has noi…

In order to push the performance on realistic computer vision tasks, the number of classes in modern benchmark datasets has significantly increased in recent years. This increase in the number of classes comes along with increased ambiguity between the class labels, raising the question if top-1 error is the right perf…

Paper characterizes minimax regret rates for online ranking with top-k feedback.

Unified model for prediction and deferral selects top-k entities efficiently.

Top-k error is currently a popular performance measure on large scale image classification benchmarks such as ImageNet and Places. Despite its wide acceptance, our understanding of this metric is limited as most of the previous research is focused on its special case, the top-1 error. In this work, we explore two direc…

Paper introduces MPES for top-k ranking BO with preferential observations.

Study improves top-k set prediction with low cardinality.

Paper extends top-k Mallows model for better user preference analysis.

Proposes differentiable and sparse top-k operators for neural networks.

This paper compares average-K and top-K classification methods under ambiguity.

Study on top-$k$ classification with new loss functions and algorithms.

This paper explores the preference-based top-$K$ rank aggregation problem. Suppose that a collection of items is repeatedly compared in pairs, and one wishes to recover a consistent ordering that emphasizes the top-$K$ ranked items, based on partially revealed preferences. We focus on the Bradley-Terry-Luce (BTL) model…

The paper proposes methods to identify and sample from mixtures of Mallows models for top-k rankings.

CPATTA uses conformal prediction for efficient test-time adaptation.

Unified framework for deferring queries to top-k experts, improving accuracy-cost trade-offs.

We propose the Limited Multi-Label (LML) projection layer as a new primitive operation for end-to-end learning systems. The LML layer provides a probabilistic way of modeling multi-label predictions limited to having exactly k labels. We derive efficient forward and backward passes for this layer and show how the layer…

Analyze SGD with biased gradients, improving convergence rates and accuracy.

New method makes CNN interpretations robust to adversarial attacks.

Introduces top-$k$ regularization for better feature selection in machine learning.

New insights into the top-K sparse softmax gating function for deep learning.

Optimizes communication in federated learning using rate-distortion theory.

We study the top-$K$ ranking problem where the goal is to recover the set of top-$K$ ranked items out of a large collection of items based on partially revealed preferences. We consider an adversarial crowdsourced setting where there are two population sets, and pairwise comparison samples drawn from one of the populat…

Optimizes ranking of top-k players from partial comparison data.

Optimizes identifying top-k items from comparisons with minimal comparisons.

Distributed stochastic gradient descent (SGD) algorithms are widely deployed in training large-scale deep learning models, while the communication overhead among workers becomes the new system bottleneck. Recently proposed gradient sparsification techniques, especially Top-$k$ sparsification with error compensation (To…

It is well-known that classifiers are vulnerable to adversarial perturbations. To defend against adversarial perturbations, various certified robustness results have been derived. However, existing certified robustnesses are limited to top-1 predictions. In many real-world applications, top-$k$ predictions are more rel…

FedSel uses local differential privacy to protect data privacy in federated SGD.

The paper addresses calibration in label ranking, a structured prediction task.

We explore the top-$K$ rank aggregation problem. Suppose a collection of items is compared in pairs repeatedly, and we aim to recover a consistent ordering that focuses on the top-$K$ ranked items based on partially revealed preference information. We investigate the Bradley-Terry-Luce model in which one ranks items ac…

RAMPART ranks top-k features more accurately than existing methods.

Proposes a method to infer ranking properties and top-K rankings with uncertainty quantification.

As traditional neural network consumes a significant amount of computing resources during back propagation, \citet{Sun2017mePropSB} propose a simple yet effective technique to alleviate this problem. In this technique, only a small subset of the full gradients are computed to update the model parameters. In this paper …

We propose a simple yet effective technique to simplify the training and the resulting model of neural networks. In back propagation, only a small subset of the full gradient is computed to update the model parameters. The gradient vectors are sparsified in such a way that only the top-k elements (in terms of magnitude…

We study the active learning problem of top-$k$ ranking from multi-wise comparisons under the popular multinomial logit model. Our goal is to identify the top-$k$ items with high probability by adaptively querying sets for comparisons and observing the noisy output of the most preferred item from each comparison. To ac…

This paper explores the adaptive (active) PAC (probably approximately correct) top-$k$ ranking (i.e., top-$k$ item selection) and total ranking problems from $l$-wise ($l\geq 2$) comparisons under the multinomial logit (MNL) model. By adaptively choosing sets to query and observing the noisy output of the most favored …

Efficiently addresses federated learning challenges with reduced communication and sample complexity.

Top-k Combinatorial Bandits generalize multi-armed bandits, where at each round any subset of $k$ out of $n$ arms may be chosen and the sum of the rewards is gained. We address the full-bandit feedback, in which the agent observes only the sum of rewards, in contrast to the semi-bandit feedback, in which the agent obse…