arXiv research

MIRA scores assess conditional distribution accuracy using joint samples.

We performed an empirical comparison of ICA and PCA algorithms by applying them on two simulated noisy time series with varying distribution parameters and level of noise. In general, ICA shows better results than PCA because it takes into account higher moments of data distribution. On the other hand, PCA remains quit…

The paper designs tests for comparing ranked preference data and finds significant differences.

The main object of Bayesian statistical inference is the determination of posterior distributions. Sometimes these laws are given for quantities devoid of empirical value. This serious drawback vanishes when one confines oneself to considering a finite horizon framework. However, assuming infinite exchangeability gives…

Real-world problems typically require the simultaneous optimization of several, often conflicting objectives. Many of these multi-objective optimization problems are characterized by wide ranges of uncertainties in their decision variables or objective functions, which further increases the complexity of optimization. …

We consider the problem of search through comparisons, where a user is presented with two candidate objects and reveals which is closer to her intended target. We study adaptive strategies for finding the target, that require knowledge of rank relationships but not actual distances between objects. We propose a new str…

Develops a statistical framework to measure uncertainty in model rankings based on human preferences.

Divergence estimators based on direct approximation of density-ratios without going through separate approximation of numerator and denominator densities have been successfully applied to machine learning tasks that involve distribution comparison such as outlier detection, transfer learning, and two-sample homogeneity…

A new metric HCP distance for comparing distributions.

Efron et al. (2001) proposed empirical Bayes formulation of the frequentist Benjamini and Hochbergs False Discovery Rate method (Benjamini and Hochberg,1995). This article attempts to unify the `two cultures' using concepts of comparison density and distribution function. We have also shown how almost all of the existi…

A new notion of stochastic ordering is introduced to compare multivariate stochastic risk models with respect to extreme portfolio losses. In the framework of multivariate regular variation comparison criteria are derived in terms of ordering conditions on the spectral measures, which allows for analytical or numerical…

Study learns linear utility functions from comparisons, showing learnability gaps between passive and active learning.

Enhances Bayesian model comparison with a probabilistic framework for meta-uncertainty.

DEOT method compares distributions across agents with privacy and efficiency.

We consider the problem of learning the qualities of a collection of items by performing noisy comparisons among them. Following the standard paradigm, we assume there is a fixed "comparison graph" and every neighboring pair of items in this graph is compared $k$ times according to the Bradley-Terry-Luce model (where t…

Binary feedback outperforms ordinal comparisons in ranking recovery.

In the absence of prior knowledge, ordinal embedding methods obtain new representation for items in a low-dimensional Euclidean space via a set of quadruple-wise comparisons. These ordinal comparisons often come from human annotators, and sufficient comparisons induce the success of classical approaches. However, colle…

We consider the problem of stochastic comparison of general Garch-like processes, for different parameters and different distributions of the innovations. We identify several stochastic orders that are propagated from the innovations to the Garch process itself, and discuss their interpretations. We focus on the convex…

Modern excavations yielded a distribution of the house areas in the ancient Egyptian city Akhetaten, which was populated for a short period during the 14th century BC. Assuming that the house area is a measure of the wealth of its inhabitants allows us to make a comparison of the wealth distributions in ancient and mod…

High-dimensional k-sample comparison is a common applied problem. We construct a class of easy-to-implement nonparametric distribution-free tests based on new tools and unexplored connections with spectral graph theory. The test is shown to possess various desirable properties along with a characteristic exploratory fl…

Active sampling algorithm improves accuracy of inferred scores from pairwise comparisons.

GOTEX synthesizes textures by optimizing feature distributions using optimal transport.

We study the problem of learning Markov decision processes with finite state and action spaces when the transition probability distributions and loss functions are chosen adversarially and are allowed to change with time. We introduce an algorithm whose regret with respect to any policy in a comparison class grows as t…

This paper provides a block coordinate descent algorithm to solve unconstrained optimization problems. In our algorithm, computation of function values or gradients is not required. Instead, pairwise comparison of function values is used. Our algorithm consists of two steps; one is the direction estimate step and the o…

There is increasing interest in learning algorithms that involve interaction between human and machine. Comparison-based queries are among the most natural ways to get feedback from humans. A challenge in designing comparison-based interactive learning algorithms is coping with noisy answers. The most common fix is to …

SC improves robustness in model comparison for misspecified models.

A new metric CKCE improves model calibration comparison.

This paper studies binary classification problem associated with a family of loss functions called large-margin unified machines (LUM), which offers a natural bridge between distribution-based likelihood approaches and margin-based approaches. It also can overcome the so-called data piling issue of support vector machi…

Estimates density ratio for two-sample comparison using tree models.

Expectile bears some interesting properties in comparison to the industry wide expected shortfall in terms of assessment of tail risk. We study the relationship between expectile and expected shortfall using duality results and the link to optimized certainty equivalent. Lower and upper bounds of expectile are derived …

Kernel testing compares cell states in single-cell data.

Study compares federated learning and coreset approaches for privacy in distributed machine learning.

Suppose that we wish to estimate a vector $\mathbf{x}$ from a set of binary paired comparisons of the form "$\mathbf{x}$ is closer to $\mathbf{p}$ than to $\mathbf{q}$" for various choices of vectors $\mathbf{p}$ and $\mathbf{q}$. The problem of estimating $\mathbf{x}$ from this type of observation arises in a variety …

Self-consistency improves the accuracy of model comparison methods.

Logistic regression for brain imaging without p-values.

We review a simple model of closed economy, where the economic agents make money transactions and a saving criterion is present. We observe the Gibbs distribution for zero saving propensity, and non-Gibbs distributions otherwise. While the exact solution in the case of zero saving propensity is already known to be give…

The misclassification error distance and the adjusted Rand index are two of the most commonly used criteria to evaluate the performance of clustering algorithms. This paper provides an in-depth comparison of the two criteria, aimed to better understand exactly what they measure, their properties and their differences. …

The paper ranks items based on top choices in multiway comparisons.

Paper proposes a new framework to compare trading strategies by accounting for market conditions.

Nested sampling improved for arbitrary priors.

Uncoupled regression is the problem to learn a model from unlabeled data and the set of target values while the correspondence between them is unknown. Such a situation arises in predicting anonymized targets that involve sensitive information, e.g., one's annual income. Since existing methods for uncoupled regression …

New framework estimates treatment effects based on preferences.

Approximate Bayesian Computation (ABC) are likelihood-free Monte Carlo methods. ABC methods use a comparison between simulated data, using different parameters drew from a prior distribution, and observed data. This comparison process is based on computing a distance between the summary statistics from the simulated da…

Inspired by applications in sports where the skill of players or teams competing against each other varies over time, we propose a probabilistic model of pairwise-comparison outcomes that can capture a wide range of time dynamics. We achieve this by replacing the static parameters of a class of popular pairwise-compari…

Evaluation of Bayesian deep learning (BDL) methods is challenging. We often seek to evaluate the methods' robustness and scalability, assessing whether new tools give `better' uncertainty estimates than old ones. These evaluations are paramount for practitioners when choosing BDL tools on-top of which they build their …

Quantum probability metrics improve distribution comparison in high dimensions.

In this short paper, we overview and extend the results of our papers cond-mat/0001432, cond-mat/0008305, and cond-mat/0103544, where we use an analogy with statistical physics to describe probability distributions of money, income, and wealth in society. By making a detailed quantitative comparison with the available …

New metrics improve regression evaluation across different data distributions.