arXiv research

PROD method improves high-dimensional regression by handling strong correlations.

In the multiple linear regression setting, we propose a general framework, termed weighted orthogonal components regression (WOCR), which encompasses many known methods as special cases, including ridge regression and principal components regression. WOCR makes use of the monotonicity inherent in orthogonal components …

Study shows RFRR's effectiveness with nearly orthogonal data in overparameterized settings.

OOMP selects features online for sparse linear regression.

The paper argues for using Neyman orthogonal score for balancing in debiased machine learning.

A l1-norm penalized orthogonal forward regression (l1-POFR) algorithm is proposed based on the concept of leaveone- out mean square error (LOOMSE). Firstly, a new l1-norm penalized cost function is defined in the constructed orthogonal space, and each orthogonal basis is associated with an individually tunable regulari…

We propose two nonlinear regression methods, named Adversarial Orthogonal Regression (AdOR) for additive noise models and Adversarial Orthogonal Structural Equation Model (AdOSE) for the general case of structural equation models. Both methods try to make the residual of regression independent from regressors while put…

Semi-parametric framework for nonlinear system identification

We propose a penalized orthogonal-components regression (POCRE) for large p small n data. Orthogonal components are sequentially constructed to maximize, upon standardization, their correlation to the response residuals. A new penalization framework, implemented via empirical Bayes thresholding, is presented to effecti…

New algorithm identifies best arm in semiparametric bandits with near optimal efficiency.

A new method selects features for better model performance.

Sparse linear regression, which entails finding a sparse solution to an underdetermined system of linear equations, can formally be expressed as an $l_0$-constrained least-squares problem. The Orthogonal Least-Squares (OLS) algorithm sequentially selects the features (i.e., columns of the coefficient matrix) to greedil…

We propose a novel application of the Simultaneous Orthogonal Matching Pursuit (S-OMP) procedure for sparsistant variable selection in ultra-high dimensional multi-task regression problems. Screening of variables, as introduced in \cite{fan08sis}, is an efficient and highly scalable way to remove many irrelevant variab…

SOFARI improves inference on multi-task learning latent factors.

KOOW method provides optimal covariate balance for continuous treatments.

Unified framework for debiased machine learning using Riesz representer and Bregman divergence.

The paper improves GP regression for sparse sensor data in structural mode shape reconstruction.

Distributed-OMP recovers sparse vectors with low communication costs.

VAR-GPs solve continual learning by updating posteriors sequentially.

A new classifier uses weighted orthogonal regression for robust classification with limited data.

Double machine learning provides $\sqrt{n}$-consistent estimates of parameters of interest even when high-dimensional or nonparametric nuisance parameters are estimated at an $n^{-1/4}$ rate. The key is to employ Neyman-orthogonal moment equations which are first-order insensitive to perturbations in the nuisance param…

Extends deep learning with interpretable additive models.

A key question in modern statistics is how to make fast and reliable inferences for complex, high-dimensional data. While there has been much interest in sparse techniques, current methods do not generalize well to data with nonlinear structure. In this work, we present an orthogonal series estimator for predictors tha…

Orthogonal matching pursuit (OMP) is a widely used algorithm for recovering sparse high dimensional vectors in linear regression models. The optimal performance of OMP requires \textit{a priori} knowledge of either the sparsity of regression vector or noise statistics. Both these statistics are rarely known \textit{a p…

ICCNLS models complex relationships as convex and concave components.

Develops a direct debiased machine learning framework using Bregman divergence.

Gradient descent dynamics in quadratic regression models are analyzed, revealing five phases: monotonic, catapult, periodic, chaotic, and divergent.

New scalable GP approximation using Fourier series decomposition.

Estimates treatment effects in randomized experiments with non-compliance.

New method for semiparametric bandits reduces regret to optimal levels.

A new method ODR-BINDy improves model discovery from noisy data.

A common method of generalizing binary to multi-class classification is the error correcting code (ECC). ECCs may be optimized in a number of ways, for instance by making them orthogonal. Here we test two types of orthogonal ECCs on seven different datasets using three types of binary classifier and compare them with t…

This paper provides mathematical foundations for regression methods used in forward initial margin approximation.

ParK efficiently solves kernel ridge regression for large datasets.

New method speeds up solving orthogonality constrained problems.

Many modern big data applications feature large scale in both numbers of responses and predictors. Better statistical efficiency and scientific insights can be enabled by understanding the large-scale response-predictor association network structures via layers of sparse latent factors ranked by importance. Yet sparsit…

Overparameterized models improve performance in sequential learning tasks.

We investigate the prediction capability of the orthogonal greedy algorithm (OGA) in high-dimensional regression models with dependent observations. The rates of convergence of the prediction error of OGA are obtained under a variety of sparsity conditions. To prevent OGA from overfitting, we introduce a high-dimension…

Discovering quasipotential equations from data using machine learning.

Two new algorithms reduce online kernel regression's computational cost while maintaining optimal regret bounds.

We propose the orthogonal random forest, an algorithm that combines Neyman-orthogonality to reduce sensitivity with respect to estimation error of nuisance parameters with generalized random forests (Athey et al., 2017)--a flexible non-parametric method for statistical estimation of conditional moment models using rand…

This paper compares two stock factor models in China's A-share market.

Gradient-enhanced GSA uses Poincaré chaos expansions for accurate sensitivity analysis.

New method optimizes model selection in high-dimensional regression models.

Proposes a new algorithm for Sparse Bayesian Learning connected to Stepwise Regression.

This paper provides estimation and inference methods for a conditional average treatment effects (CATE) characterized by a high-dimensional parameter in both homogeneous cross-sectional and unit-heterogeneous dynamic panel data settings. In our leading example, we model CATE by interacting the base treatment variable w…

Echo state network (ESN) is viewed as a temporal non-orthogonal expansion with pseudo-random parameters. Such expansions naturally give rise to regressors of various relevance to a teacher output. We illustrate that often only a certain amount of the generated echo-regressors effectively explain the variance of the tea…

We consider the high-dimensional sparse linear regression problem of accurately estimating a sparse vector using a small number of linear measurements that are contaminated by noise. It is well known that the standard cadre of computationally tractable sparse regression algorithms---such as the Lasso, Orthogonal Matchi…