Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,694 papers · 148 categories

Trend · papers per month

61123184245 · Jun 202019922001200920172026
48 results for regression oracle

Oracle inequality for sparse neural nets adapts to unknown structure.

problem Sparse deep neural nets in nonparametric regression.
method Gibbs posterior distribution with Metropolis-adjusted Langevin algorithms and mixture of uniform priors.
result Oracle inequality showing adaptation to unknown regularity and structure, achieving minimax-optimal rate of convergence.

Folded concave penalization methods have been shown to enjoy the strong oracle property for high-dimensional sparse estimation. However, a folded concave penalization problem usually has multiple local solutions and the oracle property is established only for one of the unknown local solutions. A challenging fundamenta…

2012-10-22abs ↗pdf ↗

The paper explores MAB strategies for very short horizons, introducing new methods and showing improved performance.

problem Short horizon multi-armed bandit problems in games.
method Regression oracles, forced exploration, UCBT strategy.
result Combination of epsilon-greedy or epsilon-decreasing with regression oracles outperforms other strategies.

A major challenge in contextual bandits is to design general-purpose algorithms that are both practically useful and theoretically well-founded. We present a new technique that has the empirical and computational advantages of realizability-based approaches combined with the flexibility of agnostic methods. Our algorit…

2018-03-03abs ↗pdf ↗

Study phase transitions in shuffled regression problems.

problem Phase transitions in shuffled regression problems.
method Transformed permutation recovery into probabilistic graphical model, used message passing (MP) algorithm and branching random walk process.
result Characterized impact of signal-to-noise-ratio ($\snr$) on permutation recovery, proposed Gaussian approximation method.

This paper investigates tradeoffs among optimization errors, statistical rates of convergence and the effect of heavy-tailed errors for high-dimensional robust regression with nonconvex regularization. When the additive errors in linear models have only bounded second moment, we show that iteratively reweighted $\ell_1…

2019-07-09abs ↗pdf ↗

The paper studies the benefits of curriculum learning in linear regression tasks.

problem Theoretical understanding of curriculum learning's benefits in machine learning.
method Theoretical analysis of curriculum learning in structured and unstructured multitask linear regression problems.
result Adaptive learning in the unstructured setting is fundamentally harder than oracle learning, but not in the structured setting.

Probabilistic Bisection Algorithm performs root finding based on knowledge acquired from noisy oracle responses. We consider the generalized PBA setting (G-PBA) where the statistical distribution of the oracle is unknown and location-dependent, so that model inference and Bayesian knowledge updating must be performed s…

2018-06-30abs ↗pdf ↗

Rank regression from pairwise comparisons requires many comparisons to accurately learn model parameters.

problem Learning model parameters for rank regression from noisy pairwise comparisons.
method Uniform random pairwise comparisons to estimate model parameters with a given accuracy.
result Learning model parameters requires a number of comparisons proportional to dNlog3N/ε2dN\log^3 N/ε^2.

New algorithms adapt to model misspecification in contextual bandits.

problem Design efficient algorithms for contextual bandits that handle model misspecification gracefully.
method Oracle-efficient algorithms for ε-misspecified contextual bandits using square loss regression.
result First algorithm achieving optimal regret bound for unknown misspecification level in linear contextual bandits.

We investigate the computational complexity of several basic linear algebra primitives, including largest eigenvector computation and linear regression, in the computational model that allows access to the data via a matrix-vector product oracle. We show that for polynomial accuracy, Θ(d)Θ(d) calls to the oracle are nece…

2019-11-06abs ↗pdf ↗

Sparse-penalized deep neural networks improve performance in weakly dependent processes.

problem Nonparametric regression and classification under weak dependence.
method Sparse-penalized deep neural networks with oracle inequalities and convergence rates established.
result The proposed estimators outperform non-penalized ones in simulations.

New algorithm improves gradient-based ERM for smooth convex losses.

problem Empirical risk minimization of smooth, strongly convex loss functions.
method Iterative gradient-based method with local polynomial regression.
result Oracle complexity of O((pε1)d/(2η))O((p ε^{-1})^{d/(2η)}) for our algorithm.

Framework reduces contextual bandit learning to offline regression with near-optimal regret.

problem Efficient learning with large action spaces and complex reward functions.
method Offline Estimation to Decisions (OE2D) algorithm that minimizes regret with near-optimal oracle calls.
result Near-optimal regret for contextual bandits with large action spaces and O(log(T))O(log(T)) offline oracle calls.

Paper tackles distributed quantile regression with improved efficiency and support recovery.

problem Challenges in distributed estimation and support recovery for high-dimensional linear quantile regression.
method Transformed quantile regression into least-squares optimization, applied double-smoothing approach, developed efficient algorithm.
result Achieved near-oracle convergence rate and high support recovery accuracy.

Flexible estimation of heterogeneous treatment effects lies at the heart of many statistical challenges, such as personalized medicine and optimal resource allocation. In this paper, we develop a general class of two-step algorithms for heterogeneous treatment effect estimation in observational studies. We first estima…

2017-12-13abs ↗pdf ↗

OE2D framework reduces contextual bandits to offline regression for near-optimal regret.

problem Efficiently learning contextual bandits with large action spaces and complex reward functions.
method Offline Estimation to Decisions (OE2D) algorithm that reduces contextual bandits to offline regression.
result Near-optimal regret for contextual bandits with large action spaces and O(logT)O(\log T) calls to an offline regression oracle.

Localized transfer learning improves nonparametric regression performance.

problem Improving nonparametric regression performance on target tasks.
method Localized transfer learning framework that models heterogeneity and partition covariate space into cells.
result Sharp minimax rates show local transfer mitigates the curse of dimensionality.

Proposes a new robust expectile regression method for high-dimensional data.

problem Heterogeneity in high-dimensional data with heteroscedastic variance or inhomogeneous covariate effects.
method Iteratively reweighted ℓ1-penalization for robust expectile regression (retire).
result Oracle convergence rate after log(log d) iterations in high-dimensional settings.

Faster algorithm reduces contextual bandit regret with fewer offline regression calls.

problem Optimizing reward in contextual bandits with unknown functions.
method Designing a simple algorithm with O(logT){O}(\log T) offline regression calls.
result Achieves statistically optimal regret with minimal offline calls.

TA-CQR predicts regression intervals with exact coverage, splitting miscoverage between endpoints.

problem Predicting regression intervals with exact coverage under reporting constraints.
method TA-CQR uses tail allocation to parameterize the oracle, estimating the allocation by searching quantile cores and applying nonnegative additive split-conformal calibration.
result TA-CQR achieves exact finite-sample marginal coverage under exchangeability, with theoretical guarantees on calibration and length.

New algorithms for IV regression with streaming data, avoiding matrix inversions.

problem Instrumental variable regression with streaming data.
method Viewing IV regression as a stochastic optimization problem, developing algorithms that avoid matrix inversions and mini-batches.
result Rates of convergence of order O(logT/T)\mathcal{O}(\log T/T) and O(1/T1ι)\mathcal{O}(1/T^{1-ι}) for linear models.

New weighted Lasso estimates improve logistic regression performance with measurement error.

problem Improper Lasso estimates in sparse logistic regression with equal penalties.
method Proposed weighted Lasso estimates using McDiarmid inequality for non-asymptotic oracle inequalities.
result Finite sample behavior illustrated by non-asymptotic oracle inequalities for estimation and prediction errors.

The paper analyzes sparse high-dimensional linear regression with random design and unknown error variance, providing adaptiveness and concentration rates.

problem Sparse high-dimensional linear regression with random design and unknown error variance.
method Analysis of posterior concentration rates, employing techniques to address model misspecification.
result Adaptiveness and concentration rates of the posterior for sparse high-dimensional linear regression.

New method tackles endogeneity in online learning with improved regret bounds.

problem Endogeneity in real data due to omitted variables, strategic behaviors, etc.
method O2SLS (Online Two-Stage Least Squares) for Instrumental Variable (IV) regression.
result O2SLS achieves identification and oracle regret bounds for stochastic online learning.

Researchers compare different gradient methods for ridge regression, finding conjugate gradients have similar performance.

problem Comparing statistical properties of different gradient methods in ridge regression.
method Explicit non-standard error decomposition to bound prediction error of conjugate gradient iterates.
result Conjugate gradient iterates share optimality properties with gradient flow and ridge regression up to a constant factor.

Statistical analysis of regularization in continual learning tasks.

problem Understanding how regularization affects model performance in sequential learning.
method Derivation of convergence rates, iterative update formula, and optimal hyperparameters for generalized ℓ2-regularization.
result Optimal hyperparameters balance forward and backward knowledge transfer, improving model performance.

Optimal CATE estimation with structured contrast functions using KRR.

problem Estimating CATEs with complex response functions in RKHS.
method Unified two-stage kernel ridge regression method for structured contrast functions.
result Minimax rates governed by contrast function complexity, enabling adaptation.

Study on discrepancy principle for learning algorithms in nonparametric regression.

problem Determining optimal iteration number in nonparametric regression with unknown optimal iteration.
method Investigates discrepancy principle and modified principles for kernelized spectral filters, using deviation inequalities and change-of-norm arguments.
result Classical discrepancy principle is adaptive for slow rates, while modified principles are adaptive for faster rates.

Optimal algorithm for maximizing rewards in contextual bandits with resource constraints.

problem Maximizing rewards in contextual bandits with resource constraints.
method Proposed a universal and optimal algorithmic framework for CBwK by reducing it to online regression.
result Established the optimality of the proposed algorithm for various function classes.

Develops a new fairness learning approach for multi-task regression models.

problem Fairness in multi-task regression models with biased datasets.
method Uses rank-based non-parametric independence test (Mann Whitney U statistic) and reformulates as non-convex optimization problem.
result Outperforms state-of-the-art methods on fairness metrics.

Paper introduces SGD for nonparametric additive models with optimal risk.

problem Training nonparametric additive models efficiently and accurately.
method Iterative algorithm based on stochastic gradient descent for truncated basis expansions.
result Estimator achieves minimax optimal risk in well-specified settings.

The paper tackles deep learning from dependent data, achieving optimal performance.

problem Deep learning from strongly mixing observations, especially with regularization and optimality.
method Sparse-penalized regularization for deep neural networks, oracle inequality for expected excess risk.
result Deep neural network estimator achieves minimax optimal rate for nonparametric autoregression.