We propose Top-N-Rank, a novel family of list-wise Learning-to-Rank models for reliably recommending the N top-ranked items. The proposed models optimize a variant of the widely used discounted cumulative gain (DCG) objective function which differs from DCG in two important aspects: (i) It limits the evaluation of DCG …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “top ranking”
Paper characterizes minimax regret rates for online ranking with top-k feedback.
problem Analyzing online ranking with partial feedback.
method Developed techniques from partial monitoring to characterize minimax regret rates.
result Full characterization of minimax regret rates for Precision@n.
The paper addresses calibration in label ranking, a structured prediction task.
problem Calibration in label ranking is not well understood and often poorly calibrated.
method Formalized calibration for label ranking, developed a hierarchy of notions, and empirically evaluated models.
result Popular label ranking models are often poorly calibrated, with differences between sub-ranking and top-k metrics.
The paper proposes methods to identify and sample from mixtures of Mallows models for top-k rankings.
problem Identifying and sampling from mixtures of Mallows models for top-k rankings in a heterogeneous population.
method Efficient sampling algorithms and identifiability proofs for both components of the mixture.
result The identifiability and learnability of the Mallows components' parameters in the mixture.
Paper introduces MPES for top-k ranking BO with preferential observations.
problem Handling top-k ranking and tie/indifference observations in Bayesian optimization.
method Designs a surrogate model and introduces MPES acquisition function.
result MPES outperforms existing acquisition functions in handling preferential observations.
We present online boosting algorithms for multilabel ranking with top-k feedback, where the learner only receives information about the top k items from the ranking it provides. We propose a novel surrogate loss function and unbiased estimator, allowing weak learners to update themselves with limited information. Using…
Top-Rank-Focused Adaptive Vote Collection for the Evaluation of Domain-Specific Semantic Modelscs.CL
Protocol for constructing tailored evaluation datasets for semantic models.
problem Evaluation of domain-specific semantic models, focusing on top ranks.
method Adaptive pairwise comparisons, relatedness-based evaluation dataset, metrics, stochastic transitivity model.
result Effectiveness of the proposed dataset construction protocol confirmed.
RAMPART ranks top-k features more accurately than existing methods.
problem Accurate ranking of important features in machine learning.
method Adaptive sequential halving strategy combined with ensembling techniques.
result RAMPART achieves the correct top-k ranking with high probability.
This paper explores the preference-based top- rank aggregation problem. Suppose that a collection of items is repeatedly compared in pairs, and one wishes to recover a consistent ordering that emphasizes the top- ranked items, based on partially revealed preferences. We focus on the Bradley-Terry-Luce (BTL) model…
Proposes a method to infer ranking properties and top-K rankings with uncertainty quantification.
problem General uncertainty quantification in ranking problems.
method Combinatorial inference framework for the Bradley-Terry-Luce model, generalized to multiple testing.
result Minimax optimal method for inferring top-K rankings with FDR control.
The importance of accurate recommender systems has been widely recognized by academia and industry. However, the recommendation quality is still rather low. Recently, a linear sparse and low-rank representation of the user-item matrix has been applied to produce Top-N recommendations. This approach uses the nuclear nor…
Dropping a tiny fraction of preferences can significantly alter the rankings of top LLMs.
problem Robustness of LLM ranking systems to small changes in preference data.
method A computational method based on the Bradley-Terry model to evaluate robustness.
result Top LLM rankings can be highly sensitive to the removal of a small fraction of preferences.
Optimizes ranking of top-k players from partial comparison data.
problem Identifying the top-k players from incomplete pairwise comparisons.
method Maximum Likelihood Estimator (MLE) and Spectral Method.
result MLE achieves optimal partial and exact recovery, while Spectral Method is sub-optimal.
Extends linear classification framework to nonlinear SVM-based ranking problems.
problem Maximizing performance on relevant samples in ranking problems.
method Dualization, kernel addition, componentwise dual ascent method.
result General framework for nonlinear classifiers in ranking problems.
We explore the top- rank aggregation problem. Suppose a collection of items is compared in pairs repeatedly, and we aim to recover a consistent ordering that focuses on the top- ranked items based on partially revealed preference information. We investigate the Bradley-Terry-Luce model in which one ranks items ac…
New methods provide stable ranking without assumptions on data distributions.
problem Stability issues in ranking problems with noisy data.
method Developed a stability framework and two ranking operators.
result Guaranteed stability without assumptions on data distributions.
The paper ranks items based on top choices in multiway comparisons.
problem Ranking items based on top choices in multiway comparisons.
method Uniform sampling scheme, statistical rates of convergence, asymptotic normality, maximum likelihood estimator, Gaussian multiplier bootstrap.
result Proposed inference framework for ranking items through maximum pairwise difference statistic.
Top-N recommender systems have been investigated widely both in industry and academia. However, the recommendation quality is far from satisfactory. In this paper, we propose a simple yet promising algorithm. We fill the user-item matrix based on a low-rank assumption and simultaneously keep the original information. T…
We study the active learning problem of top- ranking from multi-wise comparisons under the popular multinomial logit model. Our goal is to identify the top- items with high probability by adaptively querying sets for comparisons and observing the noisy output of the most preferred item from each comparison. To ac…
This paper explores the adaptive (active) PAC (probably approximately correct) top- ranking (i.e., top- item selection) and total ranking problems from -wise () comparisons under the multinomial logit (MNL) model. By adaptively choosing sets to query and observing the noisy output of the most favored …
Mixtures of ranking models have been widely used for heterogeneous preferences. However, learning a mixture model is highly nontrivial, especially when the dataset consists of partial orders. In such cases, the parameter of the model may not be even identifiable. In this paper, we focus on three popular structures of p…
We study the top- ranking problem where the goal is to recover the set of top- ranked items out of a large collection of items based on partially revealed preferences. We consider an adversarial crowdsourced setting where there are two population sets, and pairwise comparison samples drawn from one of the populat…
This paper uncovers the low-rank structure of neural network Hessians.
problem Understanding the structure of Hessians in neural networks.
method Proposes a decoupling conjecture to decompose layer-wise Hessians into Kronecker products of smaller matrices.
result Proves the structure of top eigenspaces in 2-layer networks and shows high overlap in top eigenvectors across different models.
In designing personalized ranking algorithms, it is desirable to encourage a high precision at the top of the ranked list. Existing methods either seek a smooth convex surrogate for a non-smooth ranking metric or directly modify updating procedures to encourage top accuracy. In this work we point out that these methods…
We implement a method for re-ranking top-10 results of a state-of-the-art question answering (QA) system. The goal of our re-ranking approach is to improve the answer selection given the user question and the top-10 candidates. We focus on improving deployed QA systems that do not allow re-training or re-training comes…
Given an incomplete ratings data over a set of users and items, the preference completion problem aims to estimate a personalized total preference order over a subset of the items. In practical settings, a ranked list of top- items from the estimated preference order is recommended to the end user in the decreasing …
A framework for binary classification on top samples.
problem Binary classification problems above/below a threshold.
method General framework for ranking problems, hypothesis testing.
result Theoretical and numerical analysis of methods.
Low Rank for Rank: Uncertainty-Aware Task-Specific LLM Ranking under Sparse Pairwise Comparisonsstat.ME
Low-rank framework for task-specific LLM ranking from sparse comparisons.
problem Challenges in reliable task-specific ranking of LLMs under sparse, imbalanced comparisons.
method Low-rank modeling of task-by-model ability matrix, max-norm accurate estimator, task-wise top-K recovery guarantees, uncertainty quantification framework.
result Improves sample efficiency and produces tighter, better-calibrated ranking certificates.
Investment challenge study finds luck and strategy equally important.
problem The role of luck and strategic considerations in M6 investment challenge performance.
method Introduced a stylized model to derive and analyze a portfolio strategy.
result Improving chances of winning without attaining abnormal returns possible.
Study on signal recovery from low-rank matrix with sparse noise.
problem Inference of a rank-one signal in the presence of sparse noise.
method Replica method from statistical physics, recursive distributional equations, population dynamics algorithm.
result Critical signal strength for recovery via top eigenvector identified.
Paper extends top-k Mallows model for better user preference analysis.
problem Capturing real-world user preferences focusing on a limited set of items.
method Generalized top-k Mallows model, novel sampling scheme, efficient algorithm, active learning.
result New tools for analysis and prediction in decision-making scenarios.
Efficiently calculates PL model likelihood for partitioned preference data.
problem Computational infeasibility of calculating PL model likelihood for partitioned preference data.
method Random utility model formulation and efficient numerical integration approach.
result Proposed method outperforms existing LTR baselines and scales to real-world tasks.
Identification of Atlas modelsq-fin.MF
Atlas models are systems of Ito processes with parameters that depend on rank. We show that the parameters of a simple Atlas model can be identified by measuring the variance of the top-ranked process for different sampling intervals.
Many latent (factorized) models have been proposed for recommendation tasks like collaborative filtering and for ranking tasks like document or image retrieval and annotation. Common to all those methods is that during inference the items are scored independently by their similarity to the query in the latent embedding…
Regression Enrichment Surfaces: a Simple Analysis Technique for Virtual Drug Screening Modelsq-bio.QM
A new method for virtual drug screening detects top treatments.
problem Understanding model performance in virtual drug screening tasks.
method Regression Enrichment Surfaces (RES) method.
result RES detects more top-performing treatments than existing methods.
We propose a rank- variant of the classical Frank-Wolfe algorithm to solve convex optimization over a trace-norm ball. Our algorithm replaces the top singular-vector computation (-SVD) in Frank-Wolfe with a top- singular-vector computation (-SVD), which can be done by repeatedly applying -SVD times. …
A new method for efficiently updating large-scale matrices in real-time.
problem Updating large-scale matrices with evolving data in real-time.
method Incremental SVD approach that handles row/column appends, rank-1 updates, and refresh strategies.
result Incremental SVD achieves accuracy close to full SVD with a fraction of the computational cost.
Origin of Crashes in 3 US stock markets: Shocks and Bubblescond-mat.stat-mech
This paper presents an exclusive classification of the largest crashes in Dow Jones Industrial Average (DJIA), SP500 and NASDAQ in the past century. Crashes are objectively defined as the top-rank filtered drawdowns (loss from the last local maximum to the next local minimum disregarding noise fluctuations), where the …
Paper introduces efficient top-k selection with differential privacy.
problem Efficiently selecting top-k elements with differential privacy.
method Oneshot Laplace mechanism, generalizing Report Noisy Max.
result Noise level of O(sqrt(k)/eps) for approximate differential privacy.
New method for summarizing ranking distributions using consensus ranking distributions.
problem Summarizing ranking distributions efficiently and accurately.
method Introducing consensus ranking distributions and a top-down tree-structured statistical algorithm.
result Optimal distortion can be expressed as a function of pairwise probabilities, enabling efficient learning methods.
New method reduces variance in estimating PL model expectations.
problem High variance in Monte Carlo estimates of PL model expectations.
method Combining Gumbel top-k trick with quasi-Monte Carlo sampling.
result More sample-efficient estimators of PL model expectations.
This paper is concerned with the problem of top- ranking from pairwise comparisons. Given a collection of items and a few pairwise comparisons across them, one wishes to identify the set of items that receive the highest ranks. To tackle this problem, we adopt the logistic parametric model --- the Bradley-Te…
Efficiently allocate budgets for LLM-assisted virtual screening to reduce costs.
problem Reducing the cost of evaluating alternatives in large-scale screening tasks.
method Propose a top- greedy evaluation mechanism and the EFG- algorithm for efficient budget allocation.
result Prove that EFG- is both sample-optimal and consistent in large-scale virtual screening.
Motivated by applications in recommender systems, web search, social choice and crowdsourcing, we consider the problem of identifying the set of top items from noisy pairwise comparisons. In our setting, we are non-actively given pairwise comparisons between each pair of items, where each comparison has noi…
We propose a novel ranking model that combines the Bradley-Terry-Luce probability model with a nonnegative matrix factorization framework to model and uncover the presence of latent variables that influence the performance of top tennis players. We derive an efficient, provably convergent, and numerically stable majori…
In the IEEE Investment ranking challenge 2018, participants were asked to build a model which would identify the best performing stocks based on their returns over a forward six months window. Anonymized financial predictors and semi-annual returns were provided for a group of anonymized stocks from 1996 to 2017, which…
Paper tackles ranking items with a semi-random comparison graph and a monotone adversary.
problem Ranking items based on pairwise comparisons from a semi-random comparison graph with a monotone adversary.
method Developed a weighted maximum likelihood estimator (MLE) and an SDP-based approach to reweight the semi-random graph.
result Achieves near-optimal sample complexity, up to a log^2(n) factor, for identifying the top-K preferred items.
Learning-to-rank techniques have proven to be extremely useful for prioritization problems, where we rank items in order of their estimated probabilities, and dedicate our limited resources to the top-ranked items. This work exposes a serious problem with the state of learning-to-rank algorithms, which is that they are…