Defines a similarity measure for classification distributions.
problem Measuring similarity between classification distributions.
method Proposes task similarity, a novel measure quantifying performance of source distributions on target distributions.
result Empirical task similarity correlates with transfer efficiency and semantic similarity of source distributions.
New findings clarify the link between distributional closeness and representational similarity.
problem When and why do different neural network representations become similar?
method Identifiability theory, focusing on model families including autoregressive language models.
result Small Kullback-Leibler divergence does not guarantee similar representations.
We introduce a new discrepancy score between two distributions that gives an indication on their similarity. While much research has been done to determine if two samples come from exactly the same distribution, much less research considered the problem of determining if two finite samples come from similar distributio…
Probabilistic graphical models compactly represent joint distributions by decomposing them into factors over subsets of random variables. In Bayesian networks, the factors are conditional probability distributions. For many problems, common information exists among those factors. Adding similarity restrictions can be v…
The problem of unsupervised learning node embeddings in graphs is one of the important directions in modern network science. In this work we propose a novel framework, which is aimed to find embeddings by \textit{discriminating distributions of similarities (DDoS)} between nodes in the graph. The general idea is implem…
Deconfounds neural network representation similarity metrics to improve consistency and accuracy.
problem Confounding by population structure in similarity metrics like RSA and CKA.
method Covariate adjustment regression to adjust for confounders.
result Improves detection of semantically similar neural networks and consistency in transfer learning.
Linear trends in classifier accuracy observed under distribution shift.
problem Understanding why classifier accuracies show linear trends under distribution shift.
method Assumed model similarity and verified empirically.
result Linear trend in classifier accuracy occurs unless distribution shift is large.
C-Mixup improves generalization in regression tasks by adjusting label similarity.
problem Improving generalization in regression tasks with limited data.
method C-Mixup adjusts the probability of mixing examples based on label similarity.
result C-Mixup achieves better generalization and robustness compared to vanilla mixup.
Geospatial analysis lacks methods like the word vector representations and pre-trained networks that significantly boost performance across a wide range of natural language and computer vision tasks. To fill this gap, we introduce Tile2Vec, an unsupervised representation learning algorithm that extends the distribution…
We introduce a conceptually simple and effective method to quantify the similarity between relations in knowledge bases. Specifically, our approach is based on the divergence between the conditional probability distributions over entity pairs. In this paper, these distributions are parameterized by a very simple neural…
This thesis studies domain adaptation under minimal distribution similarity assumptions using moments.
problem Learning from samples with distributions different from training samples.
method Uses minimal similarity assumptions modeled by moments.
result Establishes learning bounds and algorithms for domain adaptation.
Paper introduces a geometric approach to model similar probability distributions.
problem Incorporating similar probability distributions into graphical models.
method Information geometric approach to model similar distributions.
result Allows reinterpretation of existing models.
Logit distance bounds representational similarity of models.
problem Approximating linear similarity when distributions are close.
method Defined a logit distance and proved its relationship to representational dissimilarity.
result Logit distance bounds representational similarity, providing nontrivial control in practice.
New compression schemes save communication in distributed mean estimation.
problem Efficiently compressing vectors for mean estimation in a limited communication setting.
method Collaborative compression schemes that exploit vector similarities without requiring known correlations.
result Analysis shows varying error types with vector similarity.
Beta-SOD detects and corrects noisy object re-identification using cosine similarity and Beta mixtures.
problem Noisy object re-identification in image datasets.
method Reframed Re-ID as a similarity task, using Siamese networks and Beta mixture models.
result Superior performance in noisy conditions compared to state-of-the-art methods.
Paper proposes a new regularization method to prevent model degradation under distribution shifts.
problem Model performance degrades under distribution shifts.
method Supervised contrastive learning with heterogeneous similarity.
result The proposed method outperforms existing regularization methods on benchmark datasets.
Approaches KL divergence for learning multi-sense word distributions.
problem Capturing the polysemy and uncertainty of words in word embeddings.
method Modeling words as multi-sense Gaussian mixtures and using KL divergence for learning.
result The proposed approach effectively captures word entailment and distribution similarity.
Similarity measure for Gaussian process predictive distributions.
problem Comparing predictive distributions of Gaussian processes for correlated functions.
method Developed a similarity metric to compare predictive distributions of Gaussian processes.
result Gaussian process predictive distributions can be compared and one is enough to model two correlated functions.
In this paper we show similarities between turbulence and financial systems. Motivated by similarities between the two systems, we construct a multiscale model for hierarchical financial structures that exhibits a constant cascade of wealth from large financial entities to small financial entities. According to our mod…
The paper analyzes covariate shift in nonparametric regression with Markovian data.
problem Covariate shift in regression problems with Markovian data.
method Extension of nonparametric convergence rates to Markovian dependence structures, using Hölder smoothness assumptions and similarity measures.
result Precise convergence rates for Nadaraya-Watson kernel estimators under specific Markovian conditions.
The paper examines how to test if two learning algorithms produce similar outcomes.
problem Testing if two learning algorithms produce similar outcomes when trained on different data sets.
method Using Total Variation (TV) distance to measure similarity of posterior distributions.
result TV indistinguishable learning rules are equivalent to existing stability notions and can be statistically amplified.
Introduces CHL, a new loss function for continuous similarity learning.
problem Binary similarity learning limitations.
method CHL is a novel loss function that generalizes histogram loss to continuous similarities.
result CHL solves a wider range of tasks including similarity learning, representation learning, and data visualization.
This paper introduces individual fairness in clustering using f-divergence.
problem Ensuring fair clustering by treating similar individuals similarly.
method Uses f-divergence to measure statistical similarity and assigns individuals to probability distributions over cluster centers. result Provides an algorithm with provable approximation guarantee for clustering with individual fairness constraints.
We propose a novel model for generating graphs similar to a given example graph. Unlike standard approaches that compute features of graphs in Euclidean space, our approach obtains features on a surface of a hypersphere. We then utilize a von Mises-Fisher distribution, an exponential family distribution on the surface …
We propose weighted inner product similarity (WIPS) for neural network-based graph embedding. In addition to the parameters of neural networks, we optimize the weights of the inner product by allowing positive and negative values. Despite its simplicity, WIPS can approximate arbitrary general similarities in…
LSH methods extend to function spaces for efficient similarity search.
problem Efficient similarity search in function spaces.
method Locality-sensitive hashing (LSH) extended to Lp spaces using function approximation or Monte Carlo techniques. result An LSH family for Wasserstein distance over continuous probability distributions.
The study reveals the hierarchical structure of the international FOREX market using currency fluctuation distribution similarities.
problem Understanding the hierarchical structure of the international FOREX market.
method Using Jensen-Shannon divergence to quantify the similarity between normalized logarithmic return distributions of currencies.
result Clusters of currencies are consistent with the nature of underlying economies but diverge during crises.
We describe financial systems as condensates, similar to Bose-Einstein condensates, and calculate statistical distributions following from the model. The calculated distributions of investments into speculated financial assets are found equivalent to a Pareto distribution, and the calculated distributions of the price …
This paper investigates the rank distribution, cumulative probability, and probability density of price returns for the stocks traded in the KSE and the KOSDAQ market. This research demonstrates that the rank distribution is consistent approximately with the Zipf's law with exponent α=−1.00 (KSE) and -1.31 (KOSDAQ),…
Paper tackles multi-label learning by improving SVR for positive semidefinite metrics.
problem Learning positive semidefinite metrics for multi-label and label distribution learning.
method Proposes two methods to overcome SVR's limitation in learning positive semidefinite metrics.
result Demonstrates new methods achieve favorable performance in multi-label and label distribution learning.
Two new algorithms improve federated optimization under second-order similarity.
problem Federated learning under communication constraints and second-order similarity.
method SVRP and Catalyzed SVRP algorithms combining proximal point evaluations, client sampling, and variance reduction.
result Achieves superior performance and uniformly improves upon existing algorithms for federated optimization under second-order similarity and strong convexity.
In optimization, the natural gradient method is well-known for likelihood maximization. The method uses the Kullback-Leibler divergence, corresponding infinitesimally to the Fisher-Rao metric, which is pulled back to the parameter space of a family of probability distributions. This way, gradients with respect to the p…
The paper proposes a new method to evaluate LLM agent responses using ECDF clustering.
problem The standard evaluation of LLM agent responses via majority voting obscures response quality and distribution.
method The paper introduces a novel evaluation framework based on ECDF of cosine similarities and clustering of ECDFs using distances and k-medoids algorithm. result ECDF clustering reveals interpretable group structures in LLM responses, offering insights into agent settings.
This work improves transferability by considering conditional distributions in feature representations.
problem Improving transferability across multiple domains by considering conditional distributions.
method Introducing von Neumann conditional divergence to quantify the functional dependence between features and desired response.
result Favorable performance in terms of smaller generalization error and less catastrophic forgetting.
Improves confidence calibration in neural networks by smoothing labels based on class similarity.
problem Improving confidence calibration in deep neural networks for safety-critical applications.
method Proposes a novel label smoothing technique where label values are based on similarities with the reference class, using different similarity measurements.
result Consistently outperforms state-of-the-art calibration techniques on various datasets and network architectures.
Investigation of the market graph attracts a growing attention in market network analysis. One of the important problem connected with market graph is to identify it from observations. Traditional way for the market graph identification is to use a simple procedure based on statistical estimations of Pearson correlatio…
The study reveals flaws in pruning criteria and proposes a new assumption for better filter selection.
problem Flaws in existing pruning criteria for CNNs.
method Empirical experiments and Convolutional Weight Distribution Assumption.
result The Convolutional Weight Distribution Assumption improves filter selection in pruning.
The ability to measure similarity between documents enables intelligent summarization and analysis of large corpora. Past distances between documents suffer from either an inability to incorporate semantic similarities between words or from scalability issues. As an alternative, we introduce hierarchical optimal transp…
We study the fundamental limits to communication-efficient distributed methods for convex learning and optimization, under different assumptions on the information available to individual machines, and the types of functions considered. We identify cases where existing algorithms are already worst-case optimal, as well…
A new method speeds up SoftMax normalization for embedding learning.
problem Efficiently learning distributed representations with SoftMax normalization.
method Proposes a linear-time heuristic approximation for mSoftMax(XYT), optimizing cross entropy. result Achieves higher or comparable accuracy to existing methods with lower computational time.
CLS measures dataset similarity through decision rule performance.
problem Measuring dataset similarity in machine learning, especially for transfer learning and domain adaptation.
method Cross-Learning Score (CLS) measures similarity through bidirectional generalization performance of decision rules, linking to cosine similarity under canonical linear models.
result CLS effectively measures dataset similarity and transferability, validated on synthetic and real-world datasets.
New framework to test neural network representation similarity measures.
problem Disagreements among dissimilarity measures in neural networks.
method Statistical testing framework to evaluate measures based on functional behavior.
result Current metrics have different weaknesses; a classical baseline performs surprisingly well.
MixMOOD improves SSDL by selecting unlabelled data based on deep feature similarity.
problem Class distribution mismatch in semi-supervised learning.
method MixMOOD uses deep dataset dissimilarity measures to select unlabelled data.
result MixMOOD selects unlabelled data based on strong correlation with MixMatch accuracy.
Similarity algebra extends algebraic structures with quantitative bounds.
problem Exact algebraic structures with strict axioms.
method Framework for approximate algebraic and Lie structures with ε-estimates. result Similarity structures converge to classical algebraic objects as εightarrow0. kdiff measures distances for time series and structured data.
problem Estimating distances between time series and structured data.
method kdiff uses non-linear kernel distances based on matching overlapping distributions.
result kdiff is more robust to noise and partial occlusions.
We develop a general framework to analyze the distribution functions of wealth and income. Within this framework we study wealth distribution in a society by using a model which turns on two-party trading for poor people while for rich people interaction with wealthy entities (huge reservoir) is relevant. At equilibriu…
In data science, determining proximity between observations is critical to many downstream analyses such as clustering, information retrieval and classification. However, when the underlying structure of the data probability space is unclear, the function used to compute similarity between data points is often arbitrar…
Algorithm aggregates rewards from multiple players to learn related tasks in online bandit learning.
problem Learning related but slightly different tasks in an online setting with heterogeneous feedback.
method RobustAgg(ε) algorithm that aggregates rewards from different players. result Achieves instance-dependent regret guarantees and nearly matching lower bounds.