EKM addresses imbalanced data clustering by repelling centroids in large clusters.
problem Imbalanced data leads to biased clustering of large clusters.
method EKM introduces a novel centroid repulsion mechanism based on the Boltzmann operator.
result EKM outperforms benchmark algorithms on imbalanced data.
Study shows global invertibility in nonlinear elasticity with vanishing self-repulsion term.
problem Global invertibility in nonlinear elasticity with a vanishing nonlocal self-repulsion term.
method Proves global invertibility in the Γ-limit of elastic energy with a vanishing nonlocal self-repulsion term. result Global invertibility can be obtained in the Γ-limit of the elastic energy with a vanishing nonlocal self-repulsion term. Repulsive ensembles improve uncertainty estimates in PINNs for differential equations.
problem Improving uncertainty estimates in PINNs for differential equations.
method Employing repulsive ensembles (RE-PINN) with a repulsive term in the loss function.
result Repulsive ensembles produce more accurate uncertainty estimates and higher sample diversity.
Neighbor embeddings balance attraction and repulsion to visualize data.
problem Visualizing high-dimensional datasets with trade-offs between continuous and discrete structures.
method Neighbor embeddings combine attractive and repulsive forces to visualize data.
result Changing the exaggeration parameter in t-SNE yields a spectrum of embeddings with a trade-off between continuous and discrete structures.
ARS visualization improves t-SNE dynamics with tunable attraction and repulsion.
problem Improve data visualization techniques for complex data sets.
method ARS framework based on t-SNE dynamics with normalized interactions and tunable kernels.
result ARS visualization provides better control over cluster tightness and spacing.
We propose a new Stein self-repulsive dynamics for obtaining diversified samples from intractable un-normalized distributions. Our idea is to introduce Stein variational gradient as a repulsive force to push the samples of Langevin dynamics away from the past trajectories. This simple idea allows us to significantly de…
Repulsive deep ensembles improve diversity and Bayesian inference.
problem Challenges in maintaining diversity among ensemble members trained independently.
method Introducing a repulsive term in the update rule of deep ensembles.
result Training dynamics of repulsive ensembles follow a Wasserstein gradient flow of KL divergence with the true posterior.
Computer experiments reveal complex knots that don't simplify.
problem Understanding the dynamics of complex knots under self-repulsion.
method Computer simulations of knot theory, focusing on rational knots and tangles.
result Discovered hard unknots and complexified knots that do not reduce to simpler forms under self-repulsion.
CDF uses centroids to split features for high-dimensional classification.
problem High-dimensional classification problems with complex class structures.
method CDF introduces a centroid-driven splitting strategy in decision trees.
result CDF outperforms conventional methods in high-dimensional classification.
The study characterizes quadrics among affine hyperspheres based on centroid collinearity of sections.
problem Characterizing quadrics among affine hyperspheres based on section centroid collinearity.
method Extending Meyer and Reisner's theorem to unbounded convex sets and identifying additional assumptions.
result Ellipsoids, paraboloids, and one sheet of a two-sheeted hyperboloid are the only quadrics satisfying the centroid collinearity condition.
FoRDE uses input gradients to improve neural network ensembles.
problem Improving neural network ensembles for robustness and accuracy.
method Proposes FoRDE, an ensemble learning method based on ParVI, which repels function space by input gradients.
result FoRDE significantly outperforms DEs and other ensemble methods in accuracy and calibration.
Study identifies stable configurations of intertwined threads with repulsive interactions.
problem Stable configurations of entangled systems with repulsive interactions.
method Analysis of steepest descent flow of an energy functional.
result Existence and uniqueness of stable configuration of two layers drifting apart at t1/3 rate. A new classification method using disjoint centroids and normalized distance.
problem Improving classification accuracy and feature selection.
method Nearest disjoint centroid classifier with normalized distance.
result Our method outperforms other classifiers in terms of misclassification rates and feature usage.
This work shows dimension regularization can replace skip-gram negative sampling for graph embeddings, improving efficiency and performance.
problem Efficiently enforcing dissimilarity among node embeddings in graph learning.
method Dimension regularization as an alternative to skip-gram negative sampling.
result Dimension regularization is a more efficient approach to enforcing dissimilarity in graph embeddings.
Ball k-means reduces point-centroid distance computations for faster k-means clustering.
problem Efficiently finding k-means clusters in large datasets.
method Uses a ball to describe clusters, dividing them into stable and active areas, and adjusting points within annulus areas.
result Significantly reduces point-centroid distance computations, making k-means faster and more efficient.
We define a new method to estimate centroid for text classification based on the symmetric KL-divergence between the distribution of words in training documents and their class centroids. Experiments on several standard data sets indicate that the new method achieves substantial improvements over the traditional classi…
Centroid Transformers reduce memory and computation by summarizing inputs into centroids.
problem Efficiently summarize inputs with reduced memory and computation.
method Generalizes self-attention to map N inputs to M centroids (M ≤ N), reducing complexity.
result Centroid Transformers reduce memory and computation while preserving key information.
Centroid-Encoder reduces high-dimensional data for better visualization.
problem Visualizing high-dimensional data efficiently and accurately.
method Centroid-Encoder integrates label information to keep similar objects close in reduced space.
result Centroid-Encoder outperforms other techniques in visualizing high-dimensional data.
Empty core found in max-loss non-centroid clustering.
problem Core stability in non-centroid clustering under max-loss objective.
method Proof for all k≥3 and n≥9 agents, computer-aided proof for 2D Euclidean points.
result Core can be empty in non-centroid clustering under max-loss objective.
Shielded LMC samples from non-convex spaces with repulsive drift.
problem Sampling from non-convex spaces with convex holes.
method Combining adaptive temperature and repulsive drift.
result Advantages over unconstrained sampling in constrained spaces.
Due to the success of the bag-of-word modeling paradigm, clustering histograms has become an important ingredient of modern information processing. Clustering histograms can be performed using the celebrated k-means centroid-based algorithm. From the viewpoint of applications, it is usually required to deal with symm…
New approach improves multi-head attention by making heads less similar.
problem Multi-head attention can lead to similar features, reducing model expressiveness.
method Proposes a non-parametric approach using Bayesian techniques to make heads repel each other.
result Improves feature diversity, leading to better representations and performance.
Paper improves particle variational inference by optimizing generalization error bound.
problem Improving the diversity of models in particle variational inference to enhance generalization.
method Develops a new second-order Jensen inequality with a repulsion term based on the loss function, leading to a tighter generalization error bound.
result The proposed PVI optimizes the generalization error bound directly, improving performance compared to existing methods.
The paper proves a theorem linking convex body centroids and category theory.
problem Understanding centroids of sections of convex bodies.
method Lusternik-Schnirelmann category theory.
result At least n hyperplanes exist such that the center of mass of their intersection with a convex body lies on the boundary of the convex body.
Research examines correlations of complex logarithms of lattice points, showing level repulsion and Poissonian behavior.
problem Analyzing correlations of complex logarithms of lattice points.
method Proving existence of pair correlation functions and examining behavior at various scalings.
result Level repulsion observed at linear scaling, Poissonian behavior at sublinear scalings.
Generative adversarial nets (GANs) are widely used to learn the data sampling process and their performance may heavily depend on the loss functions, given a limited computational budget. This study revisits MMD-GAN that uses the maximum mean discrepancy (MMD) as the loss function for GAN and makes two contributions. F…
Centroids Matching tackles catastrophic forgetting by matching feature vectors to class centroids.
problem Catastrophic forgetting in neural networks when learning new tasks.
method Centroids Matching operates in the embedding space of neural network features, matching these vectors to class centroids.
result Centroids Matching achieves high accuracy on all tasks without using external memory, even in realistic scenarios.
Optimizes a small set of centroid points to approximate bootstrap distribution.
problem Computational inefficiency of standard bootstrap methods in large-scale machine learning.
method Explicitly optimizes a small set of high quality centroid points to approximate the ideal bootstrap distribution.
result Accurately estimates uncertainty with a small number of bootstrap centroids, outperforming i.i.d. sampling.
This paper proposes the use of an optimization algorithm, namely PSO to decide the initial centroids in K-means, to eventually get better accuracy. The vectorized notation of the optimal centroids can be thought of as entities in an optimization space, where the accuracy of K-means over a random subset of the data coul…
Text clustering method replaces centroids with summaries for interpretability and scalability.
problem Efficiently clustering text data while maintaining interpretability and scalability.
method k-NLPmeans and k-LLMmeans, which periodically replace numeric centroids with textual summaries.
result Consistently outperforms classical baselines and recent LLM-based clustering methods.
We formally prove the connection between k-means clustering and the predictions of neural networks based on the softmax activation layer. In existing work, this connection has been analyzed empirically, but it has never before been mathematically derived. The softmax function partitions the transformed input space into…
The nearest-centroid classifier is a simple linear-time classifier based on computing the centroids of the data classes in the training phase, and then assigning a new datum to the class corresponding to its nearest centroid. Thanks to its very low computational cost, the nearest-centroid classifier is still widely use…
The paper generalizes the second Pappus-Guldin theorem for calculating volumes of bodies.
problem Calculating the volume of a body cut into perpendicular slices.
method Using a generalized formula and properties of centroids and floating bodies.
result A curve with centroid property exists for convex bodies, leading to simpler volume calculations.
Sharp Lp affine isoperimetric inequalities are established for the entire class of Lp projection bodies and the entire class of Lp centroid bodies. These new inequalities strengthen the Lp Petty projection and the Lp Busemann--Petty centroid inequality.
A conceptually simple way to classify images is to directly compare test-set data and training-set data. The accuracy of this approach is limited by the method of comparison used, and by the extent to which the training-set data cover configuration space. Here we show that this coverage can be substantially increased u…
In addition to finding meaningful clusters, centroid-based clustering algorithms such as K-means or mean-shift should ideally find centroids that are valid patterns in the input space, representative of data in their cluster. This is challenging with data having a nonconvex or manifold structure, as with images or text…
RePULSe improves language model alignment by reducing undesired outputs without sacrificing overall performance.
problem Aligning language models with human preferences while minimizing undesired outputs.
method Integrates probabilistic inference into RL training to reduce undesired outputs.
result RePULSe achieves a better balance between expected reward and undesired output probability.
The convergence speed of stochastic gradient descent (SGD) can be improved by actively selecting mini-batches. We explore sampling schemes where similar data points are less likely to be selected in the same mini-batch. In particular, we prove that such repulsive sampling schemes lowers the variance of the gradient est…
CCC clusters with controlled spread, outperforming standard methods.
problem Compact and structured clustering with spread control.
method Constrained Centroid Clustering using Lagrangian formulation.
result CCC achieves more compact clusters with reduced radial spread.
We study a notion of a Lipschitz, permutation-invariant "centroid" for triples of points in mapping class groups MCG(S), which satisfies a certain polynomial growth bound. A consequence (via work of Drutu-Sapir or Chatterji-Ruane) is the Rapid Decay Property for MCG(S).
This paper introduces repulsive Monte Carlo methods for computing the sliced Wasserstein distance.
problem Computing the integral of a function on the unit sphere using Monte Carlo methods.
method The approach involves using determinantal point processes and repelled point processes to create quadratures for the sliced Wasserstein distance.
result The UnifOrtho estimator is recommended for the computation of the sliced Wasserstein distance in large dimensions.
New algorithm detects anomalies by forcing samples to displace mass in low-density regions.
problem Detecting anomalies in datasets.
method Mass Repulsing Optimal Transport (MROT) approach.
result Our algorithm improves anomaly detection over existing methods.
Method counters noisy labels by discounting distant samples.
problem Training models with noisy labels in medical and autonomous domains.
method Discounting distant samples from class centroids in latent space.
result Significant improvements in classification accuracy.
New meta-learning method improves domain generalization by balancing parameters closer to domain centroids.
problem Improving domain generalization by reducing overfitting to specific domains.
method Arithmetic meta-learning with arithmetic-weighted gradients to balance parameters closer to domain centroids.
result Experimental validation of improved domain generalization performance.
New clustering method reduces data redundancy for better summaries.
problem Redundancies in data summaries limit their effectiveness in large datasets.
method Khatri-Rao clustering extends centroid-based clustering to produce more succinct summaries.
result Khatri-Rao k-Means and deep clustering frameworks produce more succinct summaries with similar accuracy.
We consider the problem of diversity enhancing clustering, i.e, developing clustering methods which produce clusters that favour diversity with respect to a set of protected attributes such as race, sex, age, etc. In the context of fair clustering, diversity plays a major role when fairness is understood as demographic…
SIVF k-means algorithm speeds up sparse data clustering.
problem Efficiently clustering large-scale high-dimensional sparse data.
method Inverted-file structure for centroids, filter-based similarity reduction.
result SIVF achieves higher speed and lower memory consumption.
We introduce a new volume definition on normed vector spaces. We show that the induced k-area functionals are convex for all k. In the particular case k=2, our theorem implies that Busemann's 2-volume density is convex, which was recently shown by Burago-Ivanov. We also show how the new volume definition is relat…