arXiv research

Paper proposes a new sparse group k-max regularization for sparsity constraints.

In this paper, we obtain several new intrinsic and extrinsic differential sphere theorems via Ricci flow. For intrinsic case, we show that a closed simply connected $n(\ge 4)$-dimensional Riemannian manifold $M$ is diffeomorphic to $S^n$ if one of the following conditions holds pointwisely: $$ (i)\ R_0>\left(1-\frac{24…

We study how well one can recover sparse principal components of a data matrix using a sketch formed from a few of its elements. We show that for a wide class of optimization problems, if the sketch is close (in the spectral norm) to the original data matrix, then one can recover a near optimal solution to the optimiza…

The famous pinching problem says that on a compact simply connected $n$-manifold if its sectional curvature satisfies $K_{min} > (1/4)K_{max} > 0$, then the manifold is homeomorphic to the sphere. In [8, problem 12], S. T. Yau proposed the following problem: If we replace $K_{max}$ by the scalar curvature, can we deduc…

This paper develops a new class of nonconvex regularizers for low-rank matrix recovery. Many regularizers are motivated as convex relaxations of the matrix rank function. Our new factor group-sparse regularizers are motivated as a relaxation of the number of nonzero columns in a factorization of the matrix. These nonco…

New theorem for generalized group sparsity improves consistency and convergence rates.

Unified analysis of multilabel Fisher discriminants with improved dimensionality and robustness.

A fast method for discrete OT with group-sparse regularization for class label preservation.

Unified analysis of multilabel Fisher discriminants with improved dimensionality and robustness.

New FGSPCA method captures grouping and sparse structures in PCA without prior info.

Since learning is typically very slow in Boltzmann machines, there is a need to restrict connections within hidden layers. However, the resulting states of hidden units exhibit statistical dependencies. Based on this observation, we propose using $l_1/l_2$ regularization upon the activation possibilities of hidden unit…

Convex optimization with sparsity-promoting convex regularization is a standard approach for estimating sparse signals in noise. In order to promote sparsity more strongly than convex regularization, it is also standard practice to employ non-convex optimization. In this paper, we take a third approach. We utilize a no…

Proposes an algorithm for infinite-dimensional sparse learning in system identification.

We study a family of sparse estimators defined as minimizers of some empirical Lipschitz loss function -- which include the hinge loss, the logistic loss and the quantile regression loss -- with a convex, sparse or group-sparse regularization. In particular, we consider the L1 norm on the coefficients, its sorted Slope…

We consider adaptive system identification problems with convex constraints and propose a family of regularized Least-Mean-Square (LMS) algorithms. We show that with a properly selected regularization parameter the regularized LMS provably dominates its conventional counterpart in terms of mean square deviations. We es…

The paper calculates bounds for unknotting rational tangles using knot Floer homology.

Gradient descent implicitly favors group sparsity in neural networks.

This paper develops a theory for group Lasso using a concept called strong group sparsity. Our result shows that group Lasso is superior to standard Lasso for strongly group-sparse signals. This provides a convincing theoretical justification for using group sparse regularization when the underlying group structure is …

In this paper we introduce a new optimization formulation for sparse regression and compressed sensing, called CLOT (Combined L-One and Two), wherein the regularizer is a convex combination of the $\ell_1$- and $\ell_2$-norms. This formulation differs from the Elastic Net (EN) formulation, in which the regularizer is a…

Let $(M,g)$ be a compact Ricci-flat 4-manifold. For $p \in M$ let $K_{max}(p)$ (respectively $K_{min}(p)$) denote the maximum (respectively the minimum) of sectional curvatures at $p$. We prove that if $K_{max} (p) \le \ -c K_{min}(p)$ for all $p \in M$, for some constant $c$ with $0 \leq c < \frac{2+\sqrt 6}{4}$, th…

Entropic regularization is quickly emerging as a new standard in optimal transport (OT). It enables to cast the OT computation as a differentiable and unconstrained convex optimization problem, which can be efficiently solved using the Sinkhorn algorithm. However, entropy keeps the transportation plan strictly positive…

The paper improves machine learning for heavy-tailed panel data.

We present reconstruction algorithms for smooth signals with block sparsity from their compressed measurements. We tackle the issue of varying group size via group-sparse least absolute shrinkage selection operator (LASSO) as well as via latent group LASSO regularizations. We achieve smoothness in the signal via fusion…

Novel approach reduces DNN redundancy and enhances computational efficiency.

sgboost reduces variable selection bias in boosting with balanced group selection.

Sparse mapping has been a key methodology in many high-dimensional scientific problems. When multiple tasks share the set of relevant features, learning them jointly in a group drastically improves the quality of relevant feature selection. However, in practice this technique is used limitedly since such grouping infor…

Framework for inferring latent structure from sparse, imperfectly detected bipartite networks.

A new estimator learns sparse linear models with context-dependent coefficients.

A new method for sparse regression models using graph structure.

Regularized regression problems are ubiquitous in statistical modeling, signal processing, and machine learning. Sparse regression in particular has been instrumental in scientific model discovery, including compressed sensing applications, variable selection, and high-dimensional analysis. We propose a broad framework…

New guarantees for Group LASSO in sparse convex optimization.

We consider the problem of matrix completion with side information (\textit{inductive matrix completion}). In real-world applications many side-channel features are typically non-informative making feature selection an important part of the problem. We incorporate feature selection into inductive matrix completion by p…

New estimators improve sparse semiparametric additive modeling.

We apply the OSCAR (octagonal selection and clustering algorithms for regression) in recovering group-sparse matrices (two-dimensional---2D---arrays) from compressive measurements. We propose a 2D version of OSCAR (2OSCAR) consisting of the $\ell_1$ norm and the pair-wise $\ell_{\infty}$ norm, which is convex but non-d…

In text classification, the problem of overfitting arises due to the high dimensionality, making regularization essential. Although classic regularizers provide sparsity, they fail to return highly accurate models. On the contrary, state-of-the-art group-lasso regularizers provide better results at the expense of low s…

Joint sparsity regularization in multi-task learning has attracted much attention in recent years. The traditional convex formulation employs the group Lasso relaxation to achieve joint sparsity across tasks. Although this approach leads to a simple convex formulation, it suffers from several issues due to the loosenes…

Given two data matrices $X$ and $Y$, sparse canonical correlation analysis (SCCA) is to seek two sparse canonical vectors $u$ and $v$ to maximize the correlation between $Xu$ and $Yv$. However, classical and sparse CCA models consider the contribution of all the samples of data matrices and thus cannot identify an unde…

SNAM improves NAM's accuracy and feature selection via group sparsity.

New GIC improves model selection for structured sparse models.

We introduce a recursive adaptive group lasso algorithm for real-time penalized least squares prediction that produces a time sequence of optimal sparse predictor coefficient vectors. At each time index the proposed algorithm computes an exact update of the optimal $\ell_{1,\infty}$-penalized recursive least squares (R…

In this paper, we study the stochastic combinatorial multi-armed bandit (CMAB) framework that allows a general nonlinear reward function, whose expected value may not depend only on the means of the input random variables but possibly on the entire distributions of these variables. Our framework enables a much larger c…

Modern technologies are generating ever-increasing amounts of data. Making use of these data requires methods that are both statistically sound and computationally efficient. Typically, the statistical and computational aspects are treated separately. In this paper, we propose an approach to entangle these two aspects …

Proposes a multivariate regression model for better analysis of multiple datasets.

A new method solves sparse regularization problems efficiently and robustly.

Proposes novel wSVMs for sparse learning and accurate probability estimation.

In this paper, a sparse Markov decision process (MDP) with novel causal sparse Tsallis entropy regularization is proposed.The proposed policy regularization induces a sparse and multi-modal optimal policy distribution of a sparse MDP. The full mathematical analysis of the proposed sparse MDP is provided.We first analyz…

The time-evolving precision matrix of a piecewise-constant Gaussian graphical model encodes the dynamic conditional dependency structure of a multivariate time-series. Traditionally, graphical models are estimated under the assumption that data is drawn identically from a generating distribution. Introducing sparsity a…

We consider the problem of learning a sparse rule model, a prediction model in the form of a sparse linear combination of rules, where a rule is an indicator function defined over a hyper-rectangle in the input space. Since the number of all possible such rules is extremely large, it has been computationally intractabl…