We consider support recovery in the quadratic logistic regression setting - where the target depends on both p linear terms and up to quadratic terms . Quadratic terms enable prediction/modeling of higher-order effects between features and the target, but when incorporated naively may involve solvi…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “quadratic regression”
Quadratic regression involves modeling the response as a (generalized) linear function of not only the features but also of quadratic terms . The inclusion of such higher-order "interaction terms" in regression often provides an easy way to increase accuracy in already-high-dimensional problem…
A Piecewise Lyapunov Analysis of Sub-quadratic SGD: Applications to Robust and Quantile Regressionstat.ML
New analysis improves SGD for robust and quantile regression with sub-quadratic convergence.
problem Improving SGD for robust and quantile regression with sub-quadratic convergence.
method Piecewise Lyapunov function for first-order differentiable functions.
result First geometrical convergence result for sub-quadratic SGD.
New conic quadratic formulations improve outlier detection in regression models.
problem Detecting outliers in regression models with corrupted data.
method Deriving stronger second-order conic relaxations without big-M constraints.
result Proposed formulations are significantly faster than existing methods.
Gradient descent dynamics in quadratic regression models are analyzed, revealing five phases: monotonic, catapult, periodic, chaotic, and divergent.
problem Analyzing the dynamics of gradient descent in quadratic regression models.
method Fine-grained bifurcation analysis of gradient descent dynamics using a cubic map parameterized by the step-size.
result Gradient descent dynamics in quadratic regression models exhibit five distinct phases: monotonic, catapult, periodic, chaotic, and divergent.
New findings on kernel regression in the quadratic regime, improving understanding of machine learning models.
problem Understanding kernel ridge regression in the quadratic asymptotic regime.
method Extended study of kernel regression to the quadratic regime, establishing approximation bounds and spectral distributions.
result Broad class of inner-product kernels exhibit behavior similar to a quadratic kernel, with precise asymptotic training and test errors characterized.
Improved SVRG for quadratic functions achieves better performance and running times.
problem Minimizing quadratic functions with a specific type of Hessian matrix.
method Variant of SVRG algorithm for quadratic functions with improved analysis.
result Improved performance and running times for quadratic functions compared to state-of-the-art methods.
Quantum algorithm speeds up Lasso regression by quadratically faster per iteration.
problem Efficiently solving high-dimensional linear regression with L1-penalty.
method Pathwise LARS algorithm adapted for quantum computing, using minimum-finding subroutines.
result Quadratic speedup in computation time for both number of features and observations.
We study the problem of variable selection in convex nonparametric regression. Under the assumption that the true regression function is convex and sparse, we develop a screening procedure to select a subset of variables that contains the relevant variables. Our approach is a two-stage quadratic programming method that…
New lower bounds improve logistic log-likelihood optimization and inference.
problem Designing computationally tractable lower bounds for logistic log-likelihoods.
method Developed a piece-wise quadratic lower bound that uniformly improves tangent quadratic minorizers.
result Improves the speed of convergence and accuracy of variational Bayes approximations.
New method uses DC functions for piecewise linear regression.
problem Regression with piecewise linear constraints.
method Estimates piecewise linear convex functions using a difference of convex functions.
result Method achieves close to minimax statistical risk and comparable performance to existing methods.
Deep neural networks enforce non-crossing quantile regression curves.
problem Estimating quantile regression curves without crossing.
method Penalized deep ReQU neural networks with a non-crossing penalty.
result Established non-asymptotic risk and error bounds for the estimated QRP.
This paper aims at refined error analysis for binary classification using support vector machine (SVM) with Gaussian kernel and convex loss. Our first result shows that for some loss functions such as the truncated quadratic loss and quadratic loss, SVM with Gaussian kernel can reach the almost optimal learning rate, p…
Paper proposes a statistical test for transfer learning in linear regression.
problem Theoretical framework for parameter transfer in linear regression.
method Developed a statistical test to predict transfer quality.
result The test can predict if a fine-tuned model has lower prediction risk.
New algorithm for robust regression with subgaussian error bound.
problem Linear regression in the presence of outliers and finite moments.
method Adaptation of spectral method to linear regression problem.
result Optimal sub-gaussian error bound for robust regression.
A Bernstein-type inequality for stochastic processes of quadratic forms of Gaussian variablesmath.ST
We introduce a Bernstein-type inequality which serves to uniformly control quadratic forms of gaussian variables. The latter can for example be used to derive sharp model selection criteria for linear estimation in linear regression and linear inverse problems via penalization, and we do not exclude that its scope of a…
The least absolute shrinkage and selection operator (lasso) and ridge regression produce usually different estimates although input, loss function and parameterization of the penalty are identical. In this paper we look for ridge and lasso models with identical solution set. It turns out, that the lasso model with shri…
Support vector regression (SVR) is one of the most popular machine learning algorithms aiming to generate the optimal regression curve through maximizing the minimal margin of selected training samples, i.e., support vectors. Recent researchers reveal that maximizing the margin distribution of whole training dataset ra…
Novel approximation hierarchy for sparse quadratic programs.
problem Sparse Quadratic Programs with Cardinality Constraints.
method Exploits rank-dominating eigenvectors for min-max optimization over binary variables.
result Efficient screening of nonzero elements with scalable optimization algorithms.
New findings show GD converges to a linear interpolator even with quadratic loss function under certain conditions.
problem Understanding convergence of Gradient Descent with quadratic loss functions.
method Parameterized linear regression with quadratic loss function, empirical and theoretical analysis.
result Gradient Descent converges to a linear interpolator even with quadratic loss function under the Edge of Stability regime.
This paper considers online convex optimization (OCO) problems - the paramount framework for online learning algorithm design. The loss function of learning task in OCO setting is based on streaming data so that OCO is a powerful tool to model large scale applications such as online recommender systems. Meanwhile, real…
Most of machine learning approaches have stemmed from the application of minimizing the mean squared distance principle, based on the computationally efficient quadratic optimization methods. However, when faced with high-dimensional and noisy data, the quadratic error functionals demonstrated many weaknesses including…
Study shows efficient algorithms for noiseless linear regression require quadratic sample complexity in contamination rate.
problem Efficient algorithms for noiseless linear regression under Gaussian covariates with oblivious contamination.
method Formal evidence using Statistical Query complexity.
result Any efficient Statistical Query algorithm requires VSTAT complexity at least Ω(d^(1/2)/α^2).
BOKE optimizes expensive functions with reduced computational costs.
problem High computational cost of Gaussian process-based Bayesian optimization.
method Kernel regression and density-based exploration integrated into confidence bounds.
result BOKE achieves global convergence and superior computational efficiency.
Extends quadratic loss for SVM and deep learning to improve pattern correlation.
problem Improving generalization in supervised binary classification and regression tasks.
method Extends quadratic loss, restarts from problem (8) in [3], proposes new algorithms, uses multiple kernel learning.
result Comparable results with standard losses and parameterized quadratic loss.
Paper tackles multivariate shape-constrained convex regression problems.
problem Fitting a convex function to data with component-wise monotonicity and uniform Lipschitz continuity.
method Least squares estimator via solving a constrained convex quadratic programming problem. Efficient algorithms designed: sGS-ADMM and pALM.
result Both proposed algorithms outperform state-of-the-art methods in numerical experiments.
The runtime for Kernel Partial Least Squares (KPLS) to compute the fit is quadratic in the number of examples. However, the necessity of obtaining sensitivity measures as degrees of freedom for model selection or confidence intervals for more detailed analysis requires cubic runtime, and thus constitutes a computationa…
Partition functions arise in a variety of settings, including conditional random fields, logistic regression, and latent gaussian models. In this paper, we consider semistochastic quadratic bound (SQB) methods for maximum likelihood inference based on partition function optimization. Batch methods based on the quadrati…
Paper optimizes estimation of quadratic functionals in nonparametric IV models.
problem Optimal estimation of a nonlinear functional in ill-posed inverse regression.
method Adaptive, minimax estimation using leave-one-out, sieve NPIV estimator with data-driven sieve dimension selection.
result Adaptive estimator achieves minimax optimal rate in various ill-posed cases.
Itô processes are the most common form of continuous semimartingales, and include diffusion processes. This paper is concerned with the nonparametric regression relationship between two such Itô processes. We are interested in the quadratic variation (integrated volatility) of the residual in this regression, over a un…
Local polynomial regression (Fan and Gijbels 1996) is an important class of methods for nonparametric density estimation and regression problems. However, straightforward implementation of local polynomial regression has quadratic time complexity which hinders its applicability in large-scale data analysis. In this pap…
The paper explores fair regression and classification under demographic parity constraints.
problem Ensuring fairness in regression and classification models under demographic parity constraints.
method Characterizes the optimal fair regression function using a barycenter problem with optimal transport costs and studies the connection between fair classification and regression.
result The optimal fair regression function is derived from the solution to a barycenter problem with optimal transport costs, and the optimal fair cost-sensitive classifiers can be derived by applying thresholds to this function.
Batches Stabilize the Minimum Norm Risk in High Dimensional Overparameterized Linear Regressioncs.LG
Batching stabilizes risk in high-dimensional linear regression models.
problem Stability and risk behavior in high-dimensional overparameterized linear regression.
method Minimum-norm overparameterized linear regression model with batch-partitioning.
result Optimal batch size is inversely proportional to noise level and overparametrization ratio, leading to stable risk behavior.
New method for insurance valuation combining hedging and risk minimization.
problem Current insurance valuation methods do not reflect regulatory risk measures.
method Two-step hedging procedure using generalised regression.
result The method produces portfolios neutral to risk measures like VaR or expectiles.
Unified analysis of multi-task functional linear regression with manifold and composite penalties.
problem Estimating slope functions from functional data with multi-task learning.
method Penalized splines with manifold constraint and composite quadratic penalty.
result Unified convergence upper bound and phase transition behaviors for estimators.
Paper introduces a method for operator learning using random features.
problem Estimating maps between infinite-dimensional spaces using input-output pairs.
method Function-valued random features method, building a linear combination of random operators.
result The method provides convergence guarantees and error bounds for nonlinear problems.
Sparse linear regression with compressed and low-precision data via concave quadratic programmingmath.OC
We consider the problem of the recovery of a k-sparse vector from compressed linear measurements when data are corrupted by a quantization noise. When the number of measurements is not sufficiently large, different -sparse solutions may be present in the feasible set, and the classical l1 approach may be unsuccessfu…
Paper develops new spot regression estimators using candlesticks for asset pricing.
problem Estimation of spot betas in asset pricing and risk management.
method Develops a new estimation and inference framework for spot regressions using high-frequency candlesticks.
result The proposed candlestick-based estimators reduce estimation risk and achieve higher power in hypothesis testing.
Optimal algorithm for LQR control with improved regret bound.
problem Nonstochastic control with quadratic losses (LQR control).
method Online algorithm with optimal dynamic regret of .
result Improves the best known rate of for general convex losses.
This tutorial explains Linear Discriminant Analysis (LDA) and Quadratic Discriminant Analysis (QDA) as two fundamental classification methods in statistical and probabilistic learning. We start with the optimization of decision boundary on which the posteriors are equal. Then, LDA and QDA are derived for binary and mul…
Bayes-optimal learning of deep random networks with Gaussian weights is studied.
problem Learning a target function corresponding to a deep, extensive-width, non-linear neural network with random Gaussian weights.
method Closed-form expressions for Bayes-optimal test error, ridge regression, kernel and random features regression are computed.
result Optimally regularized ridge regression and kernel regression achieve Bayes-optimal performances, while logistic loss yields a near-optimal test error for classification.
Bayes-optimal learning of a neural network with quadratic activations is achieved with GAMP-RIE.
problem Learning a neural network with quadratic activations from quadratic samples.
method Combining approximate message passing with rotationally invariant matrix denoising.
result Derives a closed-form expression for Bayes-optimal test error.
We consider the minimization of composite objective functions composed of the expectation of quadratic functions and an arbitrary convex function. We study the stochastic dual averaging algorithm with a constant step-size, showing that it leads to a convergence rate of O(1/n) without strong convexity assumptions. This …
COVID-19 mortality analysis from soft-data multivariate curve regression and machine learningstat.ML
Study uses regression and ML for COVID-19 mortality forecasting.
problem Forecasting COVID-19 mortality during the first wave in Spain.
method Cyclical curve log-regression, multivariate time series spatial residual correlation analysis, Bayesian approach, machine learning.
result Empirical analysis shows ML regression models perform better than traditional methods.
The Baire metric induces an ultrametric on a dataset and is of linear computational complexity, contrasted with the standard quadratic time agglomerative hierarchical clustering algorithm. We apply the Baire distance to spectrometric and photometric redshifts from the Sloan Digital Sky Survey using, in this work, about…
New random forest algorithm improves regression with missing data.
problem Regression with missing data values.
method New random forest algorithm compared to existing techniques.
result Improved performance in quadratic errors and bias compared to existing methods.
Paper develops RGN method for estimating low-rank tensors from noisy measurements.
problem Estimating low-rank tensors from noisy linear measurements.
method Riemannian Gauss-Newton (RGN) method for efficient low-rank tensor estimation.
result First local quadratic convergence guarantee of RGN for low-rank tensor estimation in noisy settings.
New method solves constrained stochastic optimization problems efficiently.
problem Online statistical inference of constrained stochastic nonlinear optimization problems.
method Stochastic Sequential Quadratic Programming (StoSQP) with iterative sketching solver.
result The rescaled primal-dual sequence converges to a mean-zero Gaussian distribution.