arXiv research

Randomized algorithm solves vector-valued regression problems with low-rank operators.

Gradient-based optimization improves variational empirical Bayes regression.

Support vector regression (SVR) is one of the most popular machine learning algorithms aiming to generate the optimal regression curve through maximizing the minimal margin of selected training samples, i.e., support vectors. Recent researchers reveal that maximizing the margin distribution of whole training dataset ra…

We give convergence guarantees for estimating the coefficients of a symmetric mixture of two linear regressions by expectation maximization (EM). In particular, we show that the empirical EM iterates converge to the target parameter vector at the parametric rate, provided the algorithm is initialized in an unbounded co…

Introduces a new quantile regression method for financial and wage data analysis.

A new framework for dimension reduction using ensemble of random projections.

We consider the problem of learning Relational Logistic Regression (RLR). Unlike standard logistic regression, the features of RLRs are first-order formulae with associated weight vectors instead of scalar weights. We turn the problem of learning RLR to learning these vector-weighted formulae and develop a learning alg…

Efficiently estimates private least squares with linear error growth.

Modified ReLU networks improve regression estimation rates.

The paper analyzes how adversarial attacks affect sparse regression models.

We consider the high-dimensional sparse linear regression problem of accurately estimating a sparse vector using a small number of linear measurements that are contaminated by noise. It is well known that the standard cadre of computationally tractable sparse regression algorithms---such as the Lasso, Orthogonal Matchi…

In this paper, we propose a novel asymmetric $ε$-insensitive pinball loss function for quantile estimation. There exists some pinball loss functions which attempt to incorporate the $ε$-insensitive zone approach in it but, they fail to extend the $ε$-insensitive approach for quantile estimation in true sense. The propo…

We show theoretical similarities between the Least Squares Support Vector Regression (LS-SVR) model with a Radial Basis Functions (RBF) kernel and maximum a posteriori (MAP) inference on Bayesian RBF networks with a specific Gaussian prior on the regression weights. Although previous works have pointed out similar expr…

This work improves multi-task regression performance using approximations of full-conformal prediction.

This paper proposes a novel '$ν$-support vector quantile regression' ($ν$-SVQR) model for the quantile estimation. It can facilitate the automatic control over accuracy by creating a suitable asymmetric $ε$-insensitive zone according to the variance present in data. The proposed $ν$-SVQR model uses the $ν$ fraction of …

Novel SVM approach for extreme quantile regression with heavy tailed inputs.

The paper explores MAE as a loss function for DNN vector-to-vector regression, proving its advantages over MSE.

We analyze coresets for regularized regression problems and propose a modified lasso that yields smaller coresets.

Study online linear regression with paid noise reduction.

Linear regression is a classical paradigm in statistics. A new look at it is provided via the lens of universal learning. In applying universal learning to linear regression the hypotheses class represents the label $y\in {\cal R}$ as a linear combination of the feature vector $x^Tθ$ where $x\in {\cal R}^M$, within a G…

Study improves error bounds for sparse regression with heavy-tailed covariates.

The paper bounds the mean absolute error in DNN vector-to-vector regression.

We investigate Monte Carlo based algorithms for solving stochastic control problems with probabilistic constraints. Our motivation comes from microgrid management, where the controller tries to optimally dispatch a diesel generator while maintaining low probability of blackouts. The key question we investigate are empi…

Optimal rates for vector-valued regression on various norms.

Mixed linear regression involves the recovery of two (or more) unknown vectors from unlabeled linear measurements; that is, where each sample comes from exactly one of the vectors, but we do not know which one. It is a classic problem, and the natural and empirically most popular approach to its solution has been the E…

A new method for support vector regression using a data-driven insensitive parameter.

This paper proposes a new algorithm for multiple sparse regression in high dimensions, where the task is to estimate the support and values of several (typically related) sparse vectors from a few noisy linear measurements. Our algorithm is a "forward-backward" greedy procedure that -- uniquely -- operates on two disti…

Study confirms learning rates for vector-valued spectral algorithms, proving consistency.

In this work, we design a machine learning based method, online adaptive primal support vector regression (SVR), to model the implied volatility surface (IVS). The algorithm proposed is the first derivation and implementation of an online primal kernel SVR. It features enhancements that allow efficient online adaptive …

Sparse linear regression -- finding an unknown vector from linear measurements -- is now known to be possible with fewer samples than variables, via methods like the LASSO. We consider the multiple sparse linear regression problem, where several related vectors -- with partially shared support sets -- have to be recove…

CPCR mitigates bias in PCR for overparameterized models.

This paper studies the addition of linear constraints to the Support Vector Regression (SVR) when the kernel is linear. Adding those constraints into the problem allows to add prior knowledge on the estimator obtained, such as finding probability vector or monotone data. We propose a generalization of the Sequential Mi…

Paper solves NP-hard sparse mixed linear regression problem with provable guarantees.

Paper proposes a statistical test for transfer learning in linear regression.

Uncertainty analysis in the form of probabilistic forecasting can provide significant improvements in decision-making processes in the smart power grid for better integrating renewable energies such as wind. Whereas point forecasting provides a single expected value, probabilistic forecasts provide more information in …

Recently, (Blanchet, Kang, and Murhy 2016, and Blanchet, and Kang 2017) showed that several machine learning algorithms, such as square-root Lasso, Support Vector Machines, and regularized logistic regression, among many others, can be represented exactly as distributionally robust optimization (DRO) problems. The dist…

A framework for transformer attention layers derived from SVR.

We propose an algorithm to separate simultaneously speaking persons from each other, the "cocktail party problem", using a single microphone. Our approach involves a deep recurrent neural networks regression to a vector space that is descriptive of independent speakers. Such a vector space can embed empirically determi…

FineMorphs models smooth transformations for multivariate regression.

A new algorithm reduces communication rounds for distributed convex optimization.

New ensemble SVM model reduces prediction error without choosing best kernel.

Model selection is a crucial issue in machine-learning and a wide variety of penalisation methods (with possibly data dependent complexity penalties) have recently been introduced for this purpose. However their empirical performance is generally not well documented in the literature. It is the goal of this paper to in…

Algorithm solves robust linear regression with block Lewis weights.

π-GNN learns soft permutations for graph representations, improving graph classification and regression.

Most high-dimensional estimation and prediction methods propose to minimize a cost function (empirical risk) that is written as a sum of losses associated to each data point. In this paper we focus on the case of non-convex losses, which is practically important but still poorly understood. Classical empirical process …

SVM and linear regression models coincide in high dimensions.

Flexible empirical Bayes for large-scale multiple linear regression.

This paper proposes robust matrix variate regression models with rank constraints and vector regularization.