The paper develops adaptive deep learning methods for nonlinear time series models.
problem Estimating mean functions of non-stationary and nonlinear time series models.
method Develops non-penalized and sparse-penalized DNN estimators for general non-stationary time series, derives minimax lower bounds, and shows the sparse-penalized DNN estimator is adaptive and optimal.
result Sparse-penalized DNN estimator achieves minimax optimal rates for many nonlinear AR models.
Study nonparametric density estimation via measure transport, achieving optimal rates.
problem Nonparametric density estimation with optimal rates.
method Measure transport, penalized maximum likelihood, and sieved wavelet estimators.
result Achieve minimax optimal convergence rates over Hölder classes of densities.
The paper tackles deep learning from dependent data, achieving optimal performance.
problem Deep learning from strongly mixing observations, especially with regularization and optimality.
method Sparse-penalized regularization for deep neural networks, oracle inequality for expected excess risk.
result Deep neural network estimator achieves minimax optimal rate for nonparametric autoregression.
New method for inference on strongly identified functionals even when nuisance functions are weakly identified.
problem Inference on continuous linear functionals of weakly identified nuisance functions defined by conditional moment restrictions.
method Proposes penalized minimax estimators for both the primary and debiasing nuisance functions, which can converge to fixed limits regardless of nuisance identifiability.
result Proves the asymptotic normality of a debiased estimator for the functional of interest, leading to asymptotically valid confidence intervals.
Paper optimizes prediction in semi-functional linear models using kernel methods.
problem Optimizing prediction in semi-functional linear models with functional and nonparametric components.
method Double-penalized least squares method in reproducing kernel Hilbert spaces, with regularization parameter selection via generalized cross validation.
result Achieves minimax optimal rates of convergence for both functional and nonparametric components.
The matrix completion problem consists in reconstructing a matrix from a sample of entries, possibly observed with noise. A popular class of estimator, known as nuclear norm penalized estimators, are based on minimizing the sum of a data fitting term and a nuclear norm penalization. Here, we investigate the case where …
Paper proposes a new method to optimize deep neural networks with sparse regularization.
problem Difficulty in achieving optimal convergence rates for deep neural networks due to sparsity constraints.
method Introduces a novel penalized estimation method for sparse DNNs, resolving computational and theoretical issues.
result Establishes an oracle inequality for the excess risk of the proposed sparse-penalized DNN estimator and derives convergence rates.
The density matrices are positively semi-definite Hermitian matrices of unit trace that describe the state of a quantum system. The goal of the paper is to develop minimax lower bounds on error rates of estimation of low rank density matrices in trace regression models used in quantum state tomography (in particular, i…
Paper proposes deep neural networks for nonparametric regression from dependent data.
problem Nonparametric regression from strongly mixing observations.
method Minimum error entropy principle applied to deep neural networks.
result Deep neural networks achieve minimax optimal convergence rates for Gaussian errors.
We study trend filtering, a recently proposed tool of Kim et al. [SIAM Rev. 51 (2009) 339-360] for nonparametric regression. The trend filtering estimate is defined as the minimizer of a penalized least squares criterion, in which the penalty term sums the absolute kth order discrete derivatives over the input points…
This paper deals with the trace regression model where n entries or linear combinations of entries of an unknown m1×m2 matrix A0 corrupted by noise are observed. We propose a new nuclear norm penalized estimator of A0 and establish a general sharp oracle inequality for this estimator for arbitrary val…
Develops a deep learning framework for various data types.
problem Handling nonparametric regression and classification across different data types.
method Introduces a general framework with two estimators: NPDNN and SPDNN, based on data satisfying generalized Bernstein-type inequalities.
result Both NPDNN and SPDNN estimators are minimax optimal in many classical settings.
Sparse multinomial logistic regression for multiclass classification with feature selection.
problem High-dimensional multiclass classification with a focus on sparse models.
method Penalized maximum likelihood with complexity penalty, feature selection using group Lasso and Slope classifiers.
result Achievement of minimax order in both small and large number of classes regimes.
New method avoids IV limitations for flexible estimation.
problem Nonparametric estimation of IV regressions with multiple solutions.
method Minimax penalized estimator avoiding identification and closedness conditions.
result Strong L2 convergence rate without closedness condition. We consider the problem of non-parametric regression with a potentially large number of covariates. We propose a convex, penalized estimation framework that is particularly well-suited for high-dimensional sparse additive models. The proposed approach combines appealing features of finite basis representation and smoot…
New method for multiclass classification reduces error bounds.
problem High-dimensional multiclass classification with sparse coefficients.
method Sparse multinomial logistic regression with convex penalties.
result Plug-in classifiers achieve minimax generalization error bounds.
Matrix completion works well for smooth non-linear structures, even without low-rank assumptions.
problem Matrix completion for smooth non-linear structures.
method Nuclear-norm penalization for matrices lying in a low-dimensional non-linear manifold.
result Nuclear-norm penalization is minimax rate optimal for recovering smooth non-linear matrices with missing data.
This paper studies sparse density estimation via ℓ1 penalization (SPADES). We focus on estimation in high-dimensional mixture models and nonparametric adaptive density estimation. We show, respectively, that SPADES can recover, with high probability, the unknown components of a mixture of probability densities an…
New algorithms find optimal policies without knowing MDP span.
problem Finding optimal policies in MDPs without knowing span.
method Horizon calibration and span penalization techniques.
result First algorithms achieving optimal span-based complexity without prior knowledge.
We present a unified framework for estimation and analysis of generalized additive models in high dimensions. The framework defines a large class of penalized regression estimators, encompassing many existing methods. An efficient computational algorithm for this class is presented that easily scales to thousands of ob…
We study Empirical Risk Minimizers (ERM) and Regularized Empirical Risk Minimizers (RERM) for regression problems with convex and L-Lipschitz loss functions. We consider a setting where $|\cO|$ malicious outliers contaminate the labels. In that case, under a local Bernstein condition, we show that the L2-error rat…
Support vector machines (SVMs) with sparsity-inducing nonconvex penalties have received considerable attentions for the characteristics of automatic classification and variable selection. However, it is quite challenging to solve the nonconvex penalized SVMs due to their nondifferentiability, nonsmoothness and nonconve…
We consider the problem of comparing probability densities between two groups. A new probabilistic tensor product smoothing spline framework is developed to model the joint density of two variables. Under such a framework, the probability density comparison is equivalent to testing the presence/absence of interactions.…
Sparse-penalized deep neural networks improve performance in weakly dependent processes.
problem Nonparametric regression and classification under weak dependence.
method Sparse-penalized deep neural networks with oracle inequalities and convergence rates established.
result The proposed estimators outperform non-penalized ones in simulations.
In this paper we extend the results of "A strong minimax property of nondegenerate minimal submanifolds" by White, where it is proved that any smooth, compact submanifold, which is a strictly stable critical point for an elliptic parametric functional, is the unique minimizer in a certain geodesic tubular neighbourhood…
Paper addresses covariate shift in deep learning regression models.
problem Covariate shift in dependent data from different distributions.
method Sparse-penalized deep neural network (SPDNN) estimator for nonparametric regression.
result Adaptive convergence rates for quantile and Huber regression.
We investigate implicit regularization schemes for gradient descent methods applied to unpenalized least squares regression to solve the problem of reconstructing a sparse signal from an underdetermined system of linear measurements under the restricted isometry assumption. For a given parametrization yielding a non-co…
In high-dimensional data analysis, penalized likelihood estimators are shown to provide superior results in both variable selection and parameter estimation. A new algorithm, APPLE, is proposed for calculating the Approximate Path for Penalized Likelihood Estimators. Both the convex penalty (such as LASSO) and the nonc…
We consider high-dimensional binary classification by sparse logistic regression. We propose a model/feature selection procedure based on penalized maximum likelihood with a complexity penalty on the model size and derive the non-asymptotic bounds for the resulting misclassification excess risk. The bounds can be reduc…
The MM algorithm improves robust penalized estimation for outlier-contaminated data.
problem Outliers in data affect the reliability of penalized estimation.
method Innovative MM algorithm for both convex and nonconvex loss functions.
result Established convergence theory for MM algorithm with various loss functions.
A new algorithm speeds up sparse-penalized quantile regression solving non-convex penalties.
problem Sparse-penalized quantile regression with non-convex penalties.
method Single-loop smoothing ADMM (SIAD) algorithm for faster convergence.
result SIAD method outperforms existing approaches in solving sparse-penalized quantile regression.
New bounds on learning shared representations improve model performance and efficiency.
problem Improving model performance and efficiency through shared representations across clients.
method Established new upper and lower bounds on statistical error, designed a spectral estimator for non-convex least-squares solutions.
result Optimal statistical rate achieved when shared representation is well covered across clients.
Unified framework for pattern recovery in penalized and thresholded estimation.
problem Pattern recovery in penalized and thresholded estimation methods.
method Defining a novel pattern notion based on subdifferentials, introducing accessibility and noiseless recovery conditions.
result Unified and extended conditions for pattern recovery in a broad class of penalized estimators.
Study robust distribution estimation with Wasserstein distance, achieving optimal risk.
problem Robust distribution estimation under adversarial corruption.
method Combining partial OT and minimum distance estimation, proving structural properties and deriving a novel dual form.
result Achieves minimax-optimal robust estimation risk in many settings.
Study on estimating invertible functions with minimax analysis.
problem Minimizing risk of estimating invertible functions on a plane.
method Introduce two types of L2-risks, derive lower and upper rates for minimax values, develop an asymptotically almost everywhere invertible estimator. result Invertibility does not reduce the complexity of the estimation problem in terms of the rate.
The paper develops a new algorithm for constructing minimax estimators using online learning techniques.
problem Designing minimax estimators for probability distribution parameters.
method Viewing the problem as a zero-sum game and using online learning with non-convex losses to find a Nash equilibrium.
result The algorithm constructs both a minimax estimator and a least favorable prior.
In this paper, we study the performance of extremum estimators from the perspective of generalization ability (GA): the ability of a model to predict outcomes in new samples from the same population. By adapting the classical concentration inequalities, we derive upper bounds on the empirical out-of-sample prediction e…
Adversarial meta-learning computes Gamma-minimax estimators for vague prior knowledge.
problem Estimating parameters with vague prior knowledge.
method Adversarial meta-learning algorithms for Gamma-minimax estimators.
result Convergence guarantees and neural network class for selection.
Corrects GCV for inconsistent risk estimation in finite ensembles of penalized estimators.
problem Inconsistent risk estimation of GCV for finite ensembles of penalized estimators.
method Identifies a correction involving an additional scalar correction based on degrees of freedom adjusted training errors from each ensemble component.
result CGCV maintains computational advantages of GCV and is model-free uniformly consistent for ridge regression.
We develop a maximum penalized quasi-likelihood estimator for estimating in a nonparametric way the diffusion function of a diffusion process, as an alternative to more traditional kernel-based estimators. After developing a numerical scheme for computing the maximizer of the penalized maximum quasi-likelihood function…
We propose a minimax concave penalized multi-armed bandit algorithm under generalized linear model (G-MCP-Bandit) for a decision-maker facing high-dimensional data in an online learning and decision-making process. We demonstrate that the G-MCP-Bandit algorithm asymptotically achieves the optimal cumulative regret in t…
Study online learning with set-valued feedback, showing differences between deterministic and randomized approaches.
problem Online learning with set-valued feedback, where labels are sets rather than single labels.
method Introduced new combinatorial dimensions (Set Littlestone and Measure Shattering) to characterize learnability.
result Characterized deterministic and randomized online learnability, and established bounds for various learning settings.
Gradient descent outperforms ridge regression under certain covariance matrix decay conditions.
problem Comparing the performance of gradient descent and ridge regression in linear models.
method Investigated gradient descent and ridge regression for linear regression with random isotropic ground truth.
result Gradient descent outperforms ridge regression under specific covariance matrix decay conditions.
XGBoost learns a broad class of functions bridging empirical success and theory.
problem Understanding the function class implicitly learned by XGBoost.
method Introduced an infinite-dimensional function class and a complexity measure, showing optimizers of XGBoost are also optimizers of a penalized regression problem.
result Proved the least squares estimator achieves nearly minimax-optimal convergence rate over the function class.
Develops a method to predict stock returns with time-varying risk premia.
problem Predicting stock returns with time-varying risk premia while maintaining no-arbitrage restrictions.
method Penalized two-pass regression with time-varying factor loadings, incorporating penalization in the first pass and grouping in the second pass.
result The proposed method reduces prediction errors compared to other approaches.
The task of estimating a matrix given a sample of observed entries is known as the \emph{matrix completion problem}. Most works on matrix completion have focused on recovering an unknown real-valued low-rank matrix from a random sample of its entries. Here, we investigate the case of highly quantized observations when …
Paper estimates differences in multi-attribute Gaussian graphical models using non-convex penalties.
problem Estimating differences in multi-attribute Gaussian graphical models with similar structure.
method Penalized D-trace loss function with non-convex (log-sum and SCAD) penalties, proximal gradient descent methods.
result Theoretical analysis and numerical examples support consistency in support recovery and estimation.
The Lasso is suboptimal in sparse linear regression due to design matrix constraints.
problem The suboptimality of the Lasso estimator in sparse linear regression.
method Characterization of optimal estimation rates and analysis of the Lasso estimator.
result The Lasso is provably minimax rate-suboptimal when the minimum singular value is small.