arXiv research

Noise makes learning linear thresholds hard, but algorithms can still learn near-optimal thresholds.

Optimal algorithm for identifying best arm in stochastic linear bandits with fixed confidence.

LinearAPT optimizes decision-making under resource constraints for a linear threshold problem.

In this paper, we investigate a multivariate multi-response (MVMR) linear regression problem, which contains multiple linear regression models with differently distributed design matrices, and different regression and output vectors. The goal is to recover the support union of all regression vectors using $l_1/l_2$-reg…

Noise in linear networks minimizes sharpness and leads to shrinkage-thresholding.

Robust learning mixtures of linear regressions improve robustness.

Paper discusses new stochastic algorithms for sparse signal recovery.

Unified framework for shrinkage, thresholding, and regularization in normal mean estimation and linear regression.

This work interprets GELU and related activations via a first-order loss function.

Paper provides linear convergence guarantees for KZIHT and KZPT methods.

To estimate a sparse linear model from data with Gaussian noise, consilience from lasso and compressed sensing literatures is that thresholding estimators like lasso and the Dantzig selector have the ability in some situations to identify with high probability part of the significant covariates asymptotically, and are …

We consider the problem of learning a non-negative linear classifier with a $1$-norm of at most $k$, and a fixed threshold, under the hinge-loss. This problem generalizes the problem of learning a $k$-monotone disjunction. We prove that we can learn efficiently in this setting, at a rate which is linear in both $k$ and…

Iterative thresholding algorithms seek to optimize a differentiable objective function over a sparsity or rank constraint by alternating between gradient steps that reduce the objective, and thresholding steps that enforce the constraint. This work examines the choice of the thresholding operator, and asks whether it i…

We study the problem of robust linear regression with response variable corruptions. We consider the oblivious adversary model, where the adversary corrupts a fraction of the responses in complete ignorance of the data. We provide a nearly linear time estimator which consistently estimates the true regression vector, e…

Optimal algorithm for high-dimensional stochastic linear bandits with sparse parameters.

Thresholded Lasso bandit minimizes regret in sparse linear bandits.

Improved learning bounds for corrupted data using thresholded gradient descent.

We compute the log canonical thresholds of non-negatively curved singular hermitian metrics on ample linearized line bundles on bi-equivariant group compactifications of complex reductive groups. To this end, we associate to any such metric a convex function whose asymptotic behavior determines the log canonical thresh…

We consider the problem of the optimal trading strategy in the presence of linear costs, and with a strict cap on the allowed position in the market. Using Bellman's backward recursion method, we show that the optimal strategy is to switch between the maximum allowed long position and the maximum allowed short position…

Solves asset allocation for investors with utility functions and limits.

Paper analyzes robust matrix completion with efficient nonconvex method and leave-one-out analysis.

We consider the problem of online active learning to collect data for regression modeling. Specifically, we consider a decision maker with a limited experimentation budget who must efficiently learn an underlying linear population model. Our main contribution is a novel threshold-based algorithm for selection of most i…

Unified analysis of parameter norms in overparameterized linear models, revealing scaling laws and thresholds.

In recent years, unfolding iterative algorithms as neural networks has become an empirical success in solving sparse recovery problems. However, its theoretical understanding is still immature, which prevents us from fully utilizing the power of neural networks. In this work, we study unfolded ISTA (Iterative Shrinkage…

In this paper, we study the proximal gradient algorithm with extrapolation for minimizing the sum of a Lipschitz differentiable function and a proper closed convex function. Under the error bound condition used in [19] for analyzing the convergence of the proximal gradient algorithm, we show that there exists a thresho…

The thresholded feature has recently emerged as an extremely efficient, yet rough empirical approximation, of the time-consuming sparse coding inference process. Such an approximation has not yet been rigorously examined, and standard dictionaries often lead to non-optimal performance when used for computing thresholde…

This paper explains a mechanism called phase collapse that improves image classification accuracy.

New algorithms estimate function levels with near-optimal efficiency.

FILTER model uses fusion penalized logistic threshold regression for high-dimensional data with unknown cut points.

In this paper, we propose a communication- and computation-efficient algorithm to solve a convex consensus optimization problem defined over a decentralized network. A remarkable existing algorithm to solve this problem is the alternating direction method of multipliers (ADMM), in which at every iteration every node up…

New method identifies extreme risk propagation in financial networks.

In this paper, non-linear time series models are used to describe volatility in financial time series data. To describe volatility, two of the non-linear time series are combined into form TAR (Threshold Auto-Regressive Model) with AARCH (Asymmetric Auto-Regressive Conditional Heteroskedasticity) error term and its par…

We study confidence intervals based on hard-thresholding, soft-thresholding, and adaptive soft-thresholding in a linear regression model where the number of regressors $k$ may depend on and diverge with sample size $n$. In addition to the case of known error variance, we define and study versions of the estimators when…

Ridge regression is revisited with debiasing and thresholding, offering advantages over Lasso.

This study optimizes multi-modal learning thresholds and algorithms in high dimensions.

Optimal rank-adaptive matrix estimation from linear measurements.

Starting from an exact relationship between news, threshold and price return distributions in the stationary state, I discuss the ability of the Ghoulmie-Cont-Nadal model of traders to produce fat-tailed price returns. Under normal conditions, this model is not able to transform Gaussian news into fat-tailed price retu…

Paper proposes a new activation function to reduce overfitting and large weight update issues.

Paper develops algorithms to maximize AUC in imbalanced classification.

Paper analyzes adaptive ISTA with MAD for LASSO problem.

We consider the problem of sparsity-constrained $M$-estimation when both explanatory and response variables have heavy tails (bounded 4-th moments), or a fraction of arbitrary corruptions. We focus on the $k$-sparse, high-dimensional regime where the number of variables $d$ and the sample size $n$ are related through $…

Hard Thresholding Pursuit (HTP) is an iterative greedy selection procedure for finding sparse solutions of underdetermined linear systems. This method has been shown to have strong theoretical guarantee and impressive numerical performance. In this paper, we generalize HTP from compressive sensing to a generic problem …

Geometric framework for signed multivariate tail-dependence compatibility at various thresholds.

A new algorithm improves sample complexity for thresholding in Monte Carlo Tree Search.

The paper analyzes methods for sparse Bayesian regression in nonlinear system identification.

We present a framework and analysis of consistent binary classification for complex and non-decomposable performance metrics such as the F-measure and the Jaccard measure. The proposed framework is general, as it applies to both batch and online learning, and to both linear and non-linear models. Our work follows recen…

Variable selection in linear models plays a pivotal role in modern statistics. Hard-thresholding methods such as $l_0$ regularization are theoretically ideal but computationally infeasible. In this paper, we propose a new approach, called the LAGS, short for "least absulute gradient selector", to this challenging yet i…

The interplay between computational efficiency and statistical accuracy in high-dimensional inference has drawn increasing attention in the literature. In this paper, we study computational and statistical boundaries for submatrix localization. Given one observation of (one or multiple non-overlapping) signal submatrix…