arXiv research

K-means fails catastrophically in high dimensions, Hartigan's avoids it.

Hierarchical clustering is a popular method for analyzing data which associates a tree to a dataset. Hartigan consistency has been used extensively as a framework to analyze such clustering algorithms from a statistical point of view. Still, as we show in the paper, a tree which is Hartigan consistent with a given dens…

The paper connects two clustering methods by showing gradient ascent flow can move up the cluster tree.

Defines hierarchical clustering axioms for various densities.

Finding the optimal $k$-means clustering is NP-hard in general and many heuristics have been designed for minimizing monotonically the $k$-means objective. We first show how to extend Lloyd's batched relocation heuristic and Hartigan's single-point relocation heuristic to take into account empty-cluster and single-poin…

Selective clustering annotated using modes of projections (SCAMP) is a new clustering algorithm for data in $\mathbb{R}^p$. SCAMP is motivated from the point of view of non-parametric mixture modeling. Rather than maximizing a classification likelihood to determine cluster assignments, SCAMP casts clustering as a searc…

In this paper we propose a mixture model, SparseMix, for clustering of sparse high dimensional binary data, which connects model-based with centroid-based clustering. Every group is described by a representative and a probability distribution modeling dispersion from this representative. In contrast to classical mixtur…

Following Hartigan, a cluster is defined as a connected component of the t-level set of the underlying density, i.e., the set of points for which the density is greater than t. A clustering algorithm which combines a density estimate with spectral clustering techniques is proposed. Our algorithm is composed of two step…

Energy statistics was proposed by Sz\' ekely in the 80's inspired by Newton's gravitational potential in classical mechanics and it provides a model-free hypothesis test for equality of distributions. In its original form, energy statistics was formulated in Euclidean spaces. More recently, it was generalized to metric…

A new fuzzy clustering method using hyperbolic smoothing for large datasets.

In this paper we introduce three methods for re-scaling data sets aiming at improving the likelihood of clustering validity indexes to return the true number of spherical Gaussian clusters with additional noise features. Our method obtains feature re-scaling factors taking into account the structure of a given data set…

We describe $k$-MLE, a fast and efficient local search algorithm for learning finite statistical mixtures of exponential families such as Gaussian mixture models. Mixture models are traditionally learned using the expectation-maximization (EM) soft clustering technique that monotonically increases the incomplete (expec…

The level set tree approach of Hartigan (1975) provides a probabilistically based and highly interpretable encoding of the clustering behavior of a dataset. By representing the hierarchy of data modes as a dendrogram of the level sets of a density estimator, this approach offers many advantages for exploratory analysis…

Biclustering structures in data matrices were first formalized in a seminal paper by John Hartigan (1972) where one seeks to cluster cases and variables simultaneously. Such structures are also prevalent in block modeling of networks. In this paper, we develop a unified theory for the estimation and completion of matri…

Examines algorithmic modeling across three cultures.

Playing repeated matrix games (RMG) while maximizing the cumulative returns is a basic method to evaluate multi-agent learning (MAL) algorithms. Previous work has shown that $UCB$, $M3$, $S$ or $Exp3$ algorithms have good behaviours on average in RMG. Besides, hedging algorithms have been shown to be effective on predi…

Meta-algorithm selection aims to choose the best algorithm selector for a given problem instance.

Proposes CLRS benchmark to evaluate algorithmic reasoning.

Combines multiple bandit algorithms to create a nearly optimal single algorithm.

We propose accelerated randomized coordinate descent algorithms for stochastic optimization and online learning. Our algorithms have significantly less per-iteration complexity than the known accelerated gradient algorithms. The proposed algorithms for online learning have better regret performance than the known rando…

The exchange algorithm is studied for its convergence and asymptotic variance.

Bayesian networks (BN) are used in a big range of applications but they have one issue concerning parameter learning. In real application, training data are always incomplete or some nodes are hidden. To deal with this problem many learning parameter algorithms are suggested foreground EM, Gibbs sampling and RBE algori…

No algorithm outperforms uniform sampling in A/B testing.

Improves algorithm selection for thousands of candidates using dyadic features.

New ELM algorithms reduce computation time and complexity.

This review article surveys data augmentation MCMC algorithms.

Bayesian learning rule unifies and generalizes various machine learning algorithms.

Algorithm design is a laborious process and often requires many iterations of ideation and validation. In this paper, we explore automating algorithm design and present a method to learn an optimization algorithm, which we believe to be the first method that can automatically discover a better algorithm. We approach th…

This review summarizes five Lasso optimization algorithms.

Neural networks mimic algorithms to solve complex problems.

Paper proposes a reinforcement learning framework for efficient hyper-parameter tuning of stochastic optimization algorithms.

Approximate probabilistic inference algorithms are central to many fields. Examples include sequential Monte Carlo inference in robotics, variational inference in machine learning, and Markov chain Monte Carlo inference in statistics. A key problem faced by practitioners is measuring the accuracy of an approximate infe…

In this paper, we propose a convergent parallel best-response algorithm with the exact line search for the nondifferentiable nonconvex sparsity-regularized rank minimization problem. On the one hand, it exhibits a faster convergence than subgradient algorithms and block coordinate descent algorithms. On the other hand,…

New algorithms reduce bilevel optimization complexity to ε^(-1.5).

Researchers analyze how algorithmic and implementation choices affect RL performance.

Study on selecting between base algorithms in stochastic bandit problems.

New bounds derived for KG algorithm's performance in finite time.

Paper proves linear convergence of SCMS algorithm for directional data.

MLE and CVE are equivalent under exponential families, leading to faster and more stable EM algorithms.

We resolve the fundamental problem of online decoding with general $n^{th}$ order ergodic Markov chain models. Specifically, we provide deterministic and randomized algorithms whose performance is close to that of the optimal offline algorithm even when latency is small. Our algorithms admit efficient implementation vi…

New algorithm speeds up learning of graphical models.

The goal of data-driven algorithm design is to obtain high-performing algorithms for specific application domains using machine learning and data. Across many fields in AI, science, and engineering, practitioners will often fix a family of parameterized algorithms and then optimize those parameters to obtain good perfo…

The paper examines how algorithmic classification affects behavior and proposes democratizing stakes to mitigate predatory practices.

Run2Survive uses survival analysis for algorithm selection, outperforming traditional methods.

IRT improves algorithm evaluation across datasets.

New algorithms improve machine learning performance with explicit regret bounds.

The Kaczmarz algorithm is popular for iteratively solving an overdetermined system of linear equations. The traditional Kaczmarz algorithm can approximate the solution in few sweeps through the equations but a randomized version of the Kaczmarz algorithm was shown to converge exponentially and independent of number of …

The book explores alternatives to worst-case analysis for algorithm performance.