arXiv research

In the high-dimensional regression model a response variable is linearly related to $p$ covariates, but the sample size $n$ is smaller than $p$. We assume that only a small subset of covariates is `active' (i.e., the corresponding coefficients are non-zero), and consider the model-selection problem of identifying the a…

PROD method improves high-dimensional regression by handling strong correlations.

Unified framework for pattern recovery in penalized and thresholded estimation.

Forward regression is a statistical model selection and estimation procedure which inductively selects covariates that add predictive power into a working statistical regression model. Once a model is selected, unknown regression parameters are estimated by least squares. This paper analyzes forward regression in high-…

We generalize to tree graphs obtained by connecting path graphs an oracle result obtained for the Fused Lasso over the path graph. Moreover we show that it is possible to substitute in the oracle inequality the minimum of the distances between jumps by their harmonic mean. In doing so we prove a lower bound on the comp…

We present a sparse analogue to stochastic gradient descent that is guaranteed to perform well under similar conditions to the lasso. In the linear regression setup with irrepresentable noise features, our algorithm recovers the support set of the optimal parameter vector with high probability, and achieves a statistic…

This paper studies least-square regression penalized with partly smooth convex regularizers. This class of functions is very large and versatile allowing to promote solutions conforming to some notion of low-complexity. Indeed, they force solutions of variational problems to belong to a low-dimensional manifold (the so…

Variable screening is a fast dimension reduction technique for assisting high dimensional feature selection. As a preselection method, it selects a moderate size subset of candidate variables for further refining via feature selection to produce the final model. The performance of variable screening depends on both com…

In this paper, we address the problem of learning the structure of a pairwise graphical model from samples in a high-dimensional setting. Our first main result studies the sparsistency, or consistency in sparsity pattern recovery, properties of a forward-backward greedy algorithm as applied to general statistical model…

Boosting as gradient descent algorithms is one popular method in machine learning. In this paper a novel Boosting-type algorithm is proposed based on restricted gradient descent with structural sparsity control whose underlying dynamics are governed by differential inclusions. In particular, we present an iterative reg…

New method recovers PDEs from noisy data, even when conditions are violated.

Unified analysis for graph learning from multi-attribute Gaussian time series.

Sparse model selection is ubiquitous from linear regression to graphical models where regularization paths, as a family of estimators upon the regularization parameter varying, are computed when the regularization parameter is unknown or decided data-adaptively. Traditional computational methods rely on solving a set o…

Unified analysis of multi-attribute graph learning with non-convex penalties.

We study the property of the Fused Lasso Signal Approximator (FLSA) for estimating a blocky signal sequence with additive noise. We transform the FLSA to an ordinary Lasso problem. By studying the property of the design matrix in the transformed Lasso problem, we find that the irrepresentable condition might not hold, …

High throughput genetic sequencing arrays with thousands of measurements per sample and a great amount of related censored clinical data have increased demanding need for better measurement specific model selection. In this paper we establish strong oracle properties of nonconcave penalized methods for nonpolynomial (N…

Proposes a method to improve Lasso stability via variable decorrelation.

Paper identifies sparse structures and communities in heterogeneous graphical models.

Regularized M-estimators are used in diverse areas of science and engineering to fit high-dimensional models with some low-dimensional structure. Usually the low-dimensional structure is encoded by the presence of the (unknown) parameters in some low-dimensional model subspace. In such settings, it is desirable for est…

It is known that for a certain class of single index models (SIMs) $Y = f(\boldsymbol{X}_{p \times 1}^\intercal\boldsymbolβ_0, \varepsilon)$, support recovery is impossible when $\boldsymbol{X} \sim \mathcal{N}(0, \mathbb{I}_{p \times p})$ and a model complexity adjusted sample size is below a critical threshold. Recen…

In this paper, we introduce Adaptive Cluster Lasso(ACL) method for variable selection in high dimensional sparse regression models with strongly correlated variables. To handle correlated variables, the concept of clustering or grouping variables and then pursuing model fitting is widely accepted. When the dimension is…

We introduce a very general method for sparse and large-scale variable selection. The large-scale regression settings is such that both the number of parameters and the number of samples are extremely large. The proposed method is based on careful combination of penalized estimators, each applied to a random projection…

In this paper we consider the task of estimating the non-zero pattern of the sparse inverse covariance matrix of a zero-mean Gaussian random vector from a set of iid samples. Note that this is also equivalent to recovering the underlying graph structure of a sparse Gaussian Markov Random Field (GMRF). We present two no…

The Gaussian graphical model, a popular paradigm for studying relationship among variables in a wide range of applications, has attracted great attention in recent years. This paper considers a fundamental question: When is it possible to estimate low-dimensional parameters at parametric square-root rate in a large Gau…

Solar improves variable selection in high-dimensional data with complicated dependence structures.

Given i.i.d. observations of a random vector $X \in \mathbb{R}^p$, we study the problem of estimating both its covariance matrix $Σ^*$, and its inverse covariance or concentration matrix {$Θ^* = (Σ^*)^{-1}$.} We estimate $Θ^*$ by minimizing an $\ell_1$-penalized log-determinant Bregman divergence; in the multivariate G…

Generates samples conditioned on labels using optimal transport.

The paper classifies Finsler surfaces satisfying the T-condition or σT-condition.

The paper develops a new approach to conditional risk measures using modular convex analysis.

Paper constructs solutions to Bogomolny equations with specific boundary and asymptotic conditions.

We extend probabilistic programming to handle conditioning on marginal distributions.

New tests for conditional copulas based on decision trees.

Paper finds necessary condition for logarithmic Minkowski problem in higher dimensions.

This paper introduces a neural operator for probabilistic conditioning.

CSI method learns conditional distributions by estimating flow equations.

New conditional risk measures called conditional generalized quantiles defined and characterized.

A new method for learning conditional distributions using ODEs and neural networks.

Sharp statistical theory for conditional diffusion models.

An analysis is made of reality conditions within the context of noncommutative geometry. We show that if a covariant derivative satisfies a given left Leibniz rule then a right Leibniz rule is equivalent to the reality condition. We show also that the matrix which determines the reality condition must satisfy the Yang-…

New conditions prevent gaps in optimal control problems.

We identify conditional parity as a general notion of non-discrimination in machine learning. In fact, several recently proposed notions of non-discrimination, including a few counterfactual notions, are instances of conditional parity. We show that conditional parity is amenable to statistical analysis by studying ran…

We consider families of strongly consistent multivariate conditional risk measures. We show that under strong consistency these families admit a decomposition into a conditional aggregation function and a univariate conditional risk measure as introduced Hoffmann et al. (2016). Further, in analogy to the univariate cas…

Proposes a new method for interpreting feature importance and effects in dependent feature models.

New boundary conditions solve Cauchy problem for Dirac operators on spacetimes.

We extend CS divergence to conditional distributions and show its advantages in time series data and sequential decision making.

The Samuelson condition is not satisfied by tangent lines of quadratic curves.

We describe a Groebner basis of relations among conditional probabilities in a discrete probability space, with any set of conditioned-upon events. They may be specialized to the partially-observed random variable case, the purely conditional case, and other special cases. We also investigate the connection to generali…

A new method tests conditional independence by transforming it into an unconditional problem using transport maps.