arXiv research

Evidential Softmax preserves multimodality in sparse probability distributions for generative models.

Unified framework for constructing nonconvex sparse recovery methods.

SJS model predicts label shifts in multinomial datasets.

This note provides an elementary proof of the folklore fact that draws from a Dirichlet distribution (with parameters less than 1) are typically sparse (most coordinates are small).

Efficiently estimates sparse mean from heavy-tailed data.

We present an algorithm to identify sparse dependence structure in continuous and non-Gaussian probability distributions, given a corresponding set of data. The conditional independence structure of an arbitrary distribution can be represented as an undirected graph (or Markov random field), but most algorithms for lea…

The study optimizes distribution estimation from samples with relative entropy error, adapting to sparse distributions.

We review and illustrate how the volatility smile translates into a probability distribution, the market-implied probability distribution representing believes priced in. The effects of changes in the smile are examined. Special attention is given to the effects of slope, which might appear at first counter-intuitive. …

Improves sparse recovery with non-linear Fourier features.

We consider the following conditional linear regression problem: the task is to identify both (i) a $k$-DNF condition $c$ and (ii) a linear rule $f$ such that the probability of $c$ is (approximately) at least some given bound $μ$, and $f$ minimizes the $\ell_p$ loss of predicting the target $z$ in the distribution of …

In multi-instance (MI) learning, each object (bag) consists of multiple feature vectors (instances), and is most commonly regarded as a set of points in a multidimensional space. A different viewpoint is that the instances are realisations of random vectors with corresponding probability distribution, and that a bag is…

Sparse representations have proven their efficiency in solving a wide class of inverse problems encountered in signal and image processing. Conversely, enforcing the information to be spread uniformly over representation coefficients exhibits relevant properties in various applications such as digital communications. A…

A novel method to propagate uncertainty through the soft-thresholding nonlinearity is proposed in this paper. At every layer the current distribution of the target vector is represented as a spike and slab distribution, which represents the probabilities of each variable being zero, or Gaussian-distributed. Using the p…

New lower bounds show sparse recovery is hard even with multiple preconditioners.

This paper investigates the problem of sparse signal recovery in the presence of additive impulsive noise. The heavytailed impulsive noise is well modelled with stable distributions. Since there is no explicit formulation for the probability density function of $SαS$ distribution, alternative approximations like Genera…

This work improves distribution recovery from sparse data using Random Forest implicit regularization.

Paper connects probability density cuts to graph theory eigenfunctions.

Method introduces topological regularization using information filtering networks.

This paper improves topic model estimation for sparse distributions and applies it to Wasserstein distances.

Proposes MSS to identify causal structure from heterogeneous environments.

In this note we discuss a common misconception, namely that embeddings are always used to reduce the dimensionality of the item space. We show that when we measure dimensionality in terms of information entropy then the embedding of sparse probability distributions, that can be used to represent sparse features or data…

Paper extends sparse alternatives to softmax for continuous domains, enabling efficient attention mechanisms.

Proposes novel wSVMs for sparse learning and accurate probability estimation.

This paper studies Fenchel-Young losses, a generic way to construct convex loss functions from a regularization function. We analyze their properties in depth, showing that they unify many well-known loss functions and allow to create useful new ones easily. Fenchel-Young losses constructed from a generalized entropy, …

The incredible variety of galaxy shapes cannot be summarized by human defined discrete classes of shapes without causing a possibly large loss of information. Dictionary learning and sparse coding allow us to reduce the high dimensional space of shapes into a manageable low dimensional continuous vector space. Statisti…

New algorithms reduce communication for sparse mean estimation in noisy distributed systems.

The paper extends and applies a new shrinkage prior in Bayesian factor analysis.

In this paper, we propose a novel maximum causal Tsallis entropy (MCTE) framework for imitation learning which can efficiently learn a sparse multi-modal policy distribution from demonstrations. We provide the full mathematical analysis of the proposed framework. First, the optimal solution of an MCTE problem is shown …

Study on signal recovery from low-rank matrix with sparse noise.

New model allows sparse graphs with many triangles to be represented.

In this paper we propose a mixture model, SparseMix, for clustering of sparse high dimensional binary data, which connects model-based with centroid-based clustering. Every group is described by a representative and a probability distribution modeling dispersion from this representative. In contrast to classical mixtur…

We study the problem of inferring a sparse vector from random linear combinations of its components. We propose the Accelerated Orthogonal Least-Squares (AOLS) algorithm that improves performance of the well-known Orthogonal Least-Squares (OLS) algorithm while requiring significantly lower computational costs. While OL…

Bayesian inference for topics in documents with many potential causes.

We address two shortcomings in online travel time estimation methods for congested urban traffic. The first shortcoming is related to the determination of the number of mixture modes, which can change dynamically, within day and from day to day. The second shortcoming is the wide-spread use of Gaussian probability dens…

The paper proposes a new auto-regressive model for multivariate distributional time series.

Paper analyzes sparse aggregation in GLMs with Kullback-Leibler risk bounds.

We consider the problem of selecting non-zero entries of a matrix $A$ in order to produce a sparse sketch of it, $B$, that minimizes $\|A-B\|_2$. For large $m \times n$ matrices, such that $n \gg m$ (for example, representing $n$ observations over $m$ attributes) we give sampling distributions that exhibit four importa…

Compact Gaussian model approximates deep ensemble predictions.

We consider a non-projective class of inhomogeneous random graph models with interpretable parameters and a number of interesting asymptotic properties. Using the results of Bollobás et al. [2007], we show that i) the class of models is sparse and ii) depending on the choice of the parameters, the model is either scale…

The paper shows how instrumental variables can help identify sparse causal effects in linear models.

In this paper, we present a Bayesian channel estimation algorithm for multicarrier receivers based on pilot symbol observations. The inherent sparse nature of wireless multipath channels is exploited by modeling the prior distribution of multipath components' gains with a hierarchical representation of the Bessel K pro…

Kernel means are frequently used to represent probability distributions in machine learning problems. In particular, the well known kernel density estimator and the kernel mean embedding both have the form of a kernel mean. Unfortunately, kernel means are faced with scalability issues. A single point evaluation of the …

When training data is sparse, more domain knowledge must be incorporated into the learning algorithm in order to reduce the effective size of the hypothesis space. This paper builds on previous work in which knowledge about qualitative monotonicities was formally represented and incorporated into learning algorithms (e…

Study financial contagion and risk in sparse networks with directed edges.

We introduce a non-parametric method to recover physical probability distributions of asset returns based on their European option prices and some other sparse parametric information. Thus the main problem is similar to the one considered foir instance in the Recovery Theorem by Ross (2015), except that here we conside…

We consider the Orthogonal Least-Squares (OLS) algorithm for the recovery of a $m$-dimensional $k$-sparse signal from a low number of noisy linear measurements. The Exact Recovery Condition (ERC) in bounded noisy scenario is established for OLS under certain condition on nonzero elements of the signal. The new result a…

A variety of machine learning tasks---e.g., matrix factorization, topic modelling, and feature allocation---can be viewed as learning the parameters of a probability distribution over bipartite graphs. Recently, a new class of models for networks, the sparse exchangeable graphs, have been introduced to resolve some imp…

NGRs merge sparse graph recovery with PGMs for efficient probabilistic inference.