arXiv research

We adapt a Markov Random Field learning algorithm for continuous variables.

VAR-GPs solve continual learning by updating posteriors sequentially.

AEN-SAEs address feature starvation in sparse autoencoders by stabilizing the geometric alignment of sparse coding.

This study uses continuous-time analysis to understand how momentum affects the optimisation of diagonal linear networks.

Modeling inverse dynamics is crucial for accurate feedforward robot control. The model computes the necessary joint torques, to perform a desired movement. The highly non-linear inverse function of the dynamical system can be approximated using regression techniques. We propose as regression method a tensor decompositi…

c-lasso is a Python tool for robust and sparse regression with linear constraints.

In this paper we combine two important extensions of ordinary least squares regression: regularization and optimal scaling. Optimal scaling (sometimes also called optimal scoring) has originally been developed for categorical data, and the process finds quantifications for the categories that are optimal for the regres…

Evidential Softmax preserves multimodality in sparse probability distributions for generative models.

New test detects sparse alternatives in Gaussian random fields.

We consider inference about a scalar parameter under a non-parametric model based on a one-step estimator computed as a plug in estimator plus the empirical mean of an estimator of the parameter's influence function. We focus on a class of parameters that have influence function which depends on two infinite dimensiona…

Proposes FARM model combining latent factor and sparse regression.

Sparse additive modeling is a class of effective methods for performing high-dimensional nonparametric regression. In this work we show how shape constraints such as convexity/concavity and their extensions, can be integrated into additive models. The proposed sparse difference of convex additive models (SDCAM) can est…

Linear Mixed Models (LMMs) are important tools in statistical genetics. When used for feature selection, they allow to find a sparse set of genetic traits that best predict a continuous phenotype of interest, while simultaneously correcting for various confounding factors such as age, ethnicity and population structure…

In this paper, we study the information-theoretic limits of learning the structure of Bayesian networks (BNs), on discrete as well as continuous random variables, from a finite number of samples. We show that the minimum number of samples required by any procedure to recover the correct structure grows as $Ω(m)$ and $Ω…

Logistic regression models with $n$ observations and $q$ linearly-independent covariates are shown to have Fisher information volumes which are bounded below by $π^q$ and above by ${n \choose q} π^q$. This is proved with a novel generalization of the classical theorems of Pythagoras and de Gua, which is of independent …

Optimal sketching bounds for sparse linear regression under various loss functions are established.

Bayesian method predicts runtime metrics for fog manufacturing.

We consider the classical sparse regression problem of recovering a sparse signal $x_0$ given a measurement vector $y = Φx_0+w$. We propose a tree search algorithm driven by the deep neural network for sparse regression (TSN). TSN improves the signal reconstruction performance of the deep neural network designed for sp…

Bayesian approach improves uncertainty in deep learning models.

Picasso is a new library for sparse learning problems in R and Python.

Unified framework for sparse logistic regression with nonconvex regularization.

In this paper, we address the challenging problem of selecting tuning parameters for high-dimensional sparse regression. We propose a simple and computationally efficient method, called path thresholding (PaTh), that transforms any tuning parameter-dependent sparse regression algorithm into an asymptotically tuning-fre…

Novel approximation hierarchy for sparse quadratic programs.

We introduce multiscale invariant dictionaries to estimate quantum chemical energies of organic molecules, from training databases. Molecular energies are invariant to isometric atomic displacements, and are Lipschitz continuous to molecular deformations. Similarly to density functional theory (DFT), the molecule is re…

Adaptive sparseness enhances robust regression using MCC and ARD.

Efficiently estimates sparse linear regression with heavy-tailed data and outliers.

The generalized linear model (GLM) plays a key role in regression analyses. In high-dimensional data, the sparse GLM has been used but it is not robust against outliers. Recently, the robust methods have been proposed for the specific example of the sparse GLM. Among them, we focus on the robust and sparse linear regre…

Paper develops algorithms for sparse linear regression with generalized elastic net penalty.

OKRidge solves sparse ridge regression problems for nonlinear systems.

Efficiently selects predictors in sparse regression without approximations.

We consider the high-dimensional sparse linear regression problem of accurately estimating a sparse vector using a small number of linear measurements that are contaminated by noise. It is well known that the standard cadre of computationally tractable sparse regression algorithms---such as the Lasso, Orthogonal Matchi…

Paper develops an efficient method for conformal prediction in sparse linear models.

The l1-regularized logistic regression (or sparse logistic regression) is a widely used method for simultaneous classification and feature selection. Although many recent efforts have been devoted to its efficient implementation, its application to high dimensional data still poses significant challenges. In this paper…

Paper develops compact formulations for optimization problems with rank-one convex functions and indicator variables.

We provide a novel -- and to the best of our knowledge, the first -- algorithm for high dimensional sparse regression with constant fraction of corruptions in explanatory and/or response variables. Our algorithm recovers the true sparse parameters with sub-linear sample complexity, in the presence of a constant fractio…

Developed efficient distributed logistic regression for large datasets.

In this paper we discuss the variable selection method from \ell0-norm constrained regression, which is equivalent to the problem of finding the best subset of a fixed size. Our study focuses on two aspects, consistency and computation. We prove that the sparse estimator from such a method can retain all of the importa…

Multivariate regression model is a natural generalization of the classical univari- ate regression model for fitting multiple responses. In this paper, we propose a high- dimensional multivariate conditional regression model for constructing sparse estimates of the multivariate regression coefficient matrix that accoun…

Kernel regression is a popular non-parametric fitting technique. It aims at learning a function which estimates the targets for test inputs as precise as possible. Generally, the function value for a test input is estimated by a weighted average of the surrounding training examples. The weights are typically computed b…

New method generates continuous Pareto sets for multi-task learning.

Oracle inequality for sparse neural nets adapts to unknown structure.

Volterra and polynomial regression models play a major role in nonlinear system identification and inference tasks. Exciting applications ranging from neuroscience to genome-wide association analysis build on these models with the additional requirement of parsimony. This requirement has high interpretative value, but …

Efficiently scales continuous kernels with sparse Fourier domain learning.

Estimates signals from a continuous dictionary with sparse mixtures using optimization.

New algorithms optimize constrained problems faster, avoiding full set optimization.

Efficiently estimates sparse linear regression with heavy-tailed and outlier-contaminated data.

LDAO addresses imbalanced regression by learning local distribution structures.

Paper develops minimax rates for non-exact sparse models in high dimensions.