arXiv research

Improves GAN-based semi-supervised learning with consistency regularization.

Proposes a method for multi-view clustering that integrates consistent and complementary graph regularizers.

Consistency regularization improves robustness to noisy labels.

Regularized M-estimators are used in diverse areas of science and engineering to fit high-dimensional models with some low-dimensional structure. Usually the low-dimensional structure is encoded by the presence of the (unknown) parameters in some low-dimensional model subspace. In such settings, it is desirable for est…

New theorem for generalized group sparsity improves consistency and convergence rates.

ε-Consistent Mixup improves semi-supervised classification accuracy.

Paper connects risk consistency to L_p consistency for broader loss functions.

Generative Adversarial Networks (GANs) are known to be difficult to train, despite considerable research effort. Several regularization techniques for stabilizing training have been proposed, but they introduce non-trivial computational overheads and interact poorly with existing techniques like spectral normalization.…

SCR improves GNN training with consistency regularization.

Deep neural networks without regularization can achieve consistent estimates with good convergence rates.

Contrastive regularization improves semi-supervised learning by better propagating confident pseudo-labels.

Regularization improves stability and consistency of sparse autoencoders.

We theoretically investigate the convergence rate and support consistency (i.e., correctly identifying the subset of non-zero coefficients in the large sample limit) of multiple kernel learning (MKL). We focus on MKL with block-l1 regularization (inducing sparse kernel combination), block-l2 regularization (inducing un…

MaxMatch improves SSL with worst-case consistency for better generalization.

We propose and analyze a regularization approach for structured prediction problems. We characterize a large class of loss functions that allows to naturally embed structured outputs in a linear space. We exploit this fact to design learning algorithms using a surrogate loss approach and regularization techniques. We p…

The paper proves consistency of neural networks with regularization.

Improved VAEs learn consistent posterior distributions from missing data.

This paper studies least-square regression penalized with partly smooth convex regularizers. This class of functions is very large and versatile allowing to promote solutions conforming to some notion of low-complexity. Indeed, they force solutions of variational problems to belong to a low-dimensional manifold (the so…

Recent work has increased the performance of Generative Adversarial Networks (GANs) by enforcing a consistency cost on the discriminator. We improve on this technique in several ways. We first show that consistency regularization can introduce artifacts into the GAN samples and explain how to fix this issue. We then pr…

A new method for PU learning improves classification error on CIFAR-10.

Modelling statistical relationships beyond the conditional mean is crucial in many settings. Conditional density estimation (CDE) aims to learn the full conditional probability density from data. Though highly expressive, neural network based CDE models can suffer from severe over-fitting when trained with the maximum …

ARGEN method improves variable selection and regularization in high-dimensional sparse models.

Lasso proves consistent model selection for high-dimensional Ising models.

We propose a vector-valued regression problem whose solution is equivalent to the reproducing kernel Hilbert space (RKHS) embedding of the Bayesian posterior distribution. This equivalence provides a new understanding of kernel Bayesian inference. Moreover, the optimization problem induces a new regularization for the …

The problem of learning from label proportions (LLP) involves training classifiers with weak labels on bags of instances, rather than strong labels on individual instances. The weak labels only contain the label proportion of each bag. The LLP problem is important for many practical applications that only allow label p…

RegMixMatch optimizes Mixup for semi-supervised learning by integrating high- and low-confidence samples.

Convex optimization with sparsity-promoting convex regularization is a standard approach for estimating sparse signals in noise. In order to promote sparsity more strongly than convex regularization, it is also standard practice to employ non-convex optimization. In this paper, we take a third approach. We utilize a no…

Study evaluates various regularization methods for electricity price forecasting.

Consistent partial identification of causal effects proved for neural models.

Recent advances in semi-supervised learning have shown tremendous potential in overcoming a major barrier to the success of modern machine learning algorithms: access to vast amounts of human-labeled training data. Previous algorithms based on consistency regularization can harness the abundance of unlabeled data to pr…

A new method for community detection in networks is presented.

Sparse model selection is ubiquitous from linear regression to graphical models where regularization paths, as a family of estimators upon the regularization parameter varying, are computed when the regularization parameter is unknown or decided data-adaptively. Traditional computational methods rely on solving a set o…

Paper proposes a new sparsity scheme for high-dimensional VAR models.

In 1976, Dodziuk and Patodi employed Whitney forms to define a combinatorial codifferential operator on cochains, and they raised the question whether it is consistent in the sense that for a smooth enough differential form the combinatorial codifferential of the associated cochain converges to the exterior codifferent…

Dropout and similar stochastic neural network regularization methods are often interpreted as implicitly averaging over a large ensemble of models. We propose STE (stochastically trained ensemble) layers, which enhance the averaging properties of such methods by training an ensemble of weight matrices with stochastic r…

Recent GAN-based architectures have been able to deliver impressive performance on the general task of image-to-image translation. In particular, it was shown that a wide variety of image translation operators may be learned from two image sets, containing images from two different domains, without establishing an expl…

Regularization improves robustness of smoothed classifiers.

Classifies tilings of hyperbolic plane by regular polygons.

Theoretical analysis of MCR for improving imputation quality in partially observed data.

We consider the least-square linear regression problem with regularization by the $\ell^1$-norm, a problem usually referred to as the Lasso. In this paper, we first present a detailed asymptotic analysis of model consistency of the Lasso in low-dimensional settings. For various decays of the regularization parameter, w…

We demonstrate that the primal-dual witness proof method may be used to establish variable selection consistency and $\ell_\infty$-bounds for sparse regression problems, even when the loss function and/or regularizer are nonconvex. Using this method, we derive two theorems concerning support recovery and $\ell_\infty$-…

Generative adversarial network for probabilistic forecasting of random systems.

Presently the most successful approaches to semi-supervised learning are based on consistency regularization, whereby a model is trained to be robust to small perturbations of its inputs and parameters. To understand consistency regularization, we conceptually explore how loss geometry interacts with training procedure…

This work investigates image augmentations for GAN training, improving image quality.

TAMD prevents degeneracy in finite mixtures, offering strong guarantees but modest practical improvements.

Lasso and other regularization procedures are attractive methods for variable selection, subject to a proper choice of shrinkage parameter. Given a set of potential subsets produced by a regularization algorithm, a consistent model selection criterion is proposed to select the best one among this preselected set. The a…

Deep neural networks (DNNs) often require good regularizers to generalize well. Currently, state-of-the-art DNN regularization techniques consist in randomly dropping units and/or connections on each iteration of the training algorithm. Dropout and DropConnect are characteristic examples of such regularizers, that are …

Reducing ICD-10 code granularity improves cost model accuracy and stability.