In this paper, we propose a one-pass algorithm on MapReduce for penalized linear regression \[f_λ(α, β) = \|Y - α\mathbf{1} - Xβ\|_2^2 + p_λ(β)\] where is the intercept which can be omitted depending on application; is the coefficients and is the penalized function with penalizing parameter . $f_λ(α, β…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “penalized algorithm”
In this work we establish the equivalence of algorithmic regularization and explicit convex penalization for generic convex losses. We introduce a geometric condition for the optimization path of a convex function, and show that if such a condition is satisfied, the optimization path of an iterative algorithm on the un…
In this paper we purpose a blockwise descent algorithm for group-penalized multiresponse regression. Using a quasi-newton framework we extend this to group-penalized multinomial regression. We give a publicly available implementation for these in R, and compare the speed of this algorithm to a competing algorithm --- w…
Develops a fast algorithm for high-dimensional LASSO penalized quantile regression.
problem Computational challenges in high-dimensional penalized quantile regression.
method Pathwise coordinate descent algorithm to solve exact coordinatewise minimum of the nonsmooth loss function.
result Algorithm runs faster than existing alternatives and maintains estimation accuracy.
In high-dimensional data analysis, penalized likelihood estimators are shown to provide superior results in both variable selection and parameter estimation. A new algorithm, APPLE, is proposed for calculating the Approximate Path for Penalized Likelihood Estimators. Both the convex penalty (such as LASSO) and the nonc…
Algorithm samples from Wasserstein barycenter of measures.
problem Sampling from Wasserstein barycenter of measures.
method Gradient flow of multimarginal formulation with penalization.
result Algorithm samples close to Wasserstein barycenter.
Narendra-Shapiro (NS) algorithms are bandit-type algorithms that have been introduced in the sixties (with a view to applications in Psychology or learning automata), whose convergence has been intensively studied in the stochastic algorithm literature. In this paper, we adress the following question: are the Narendra-…
New insights into balancing reward and fairness in stochastic MAB.
problem Balancing reward and fairness in stochastic multi-armed bandits.
method Formulated a penalization framework and proposed a hard-threshold UCB-like algorithm.
result Asymptotic fairness, nearly optimal regret, better reward-fairness tradeoff.
MM for Penalized Estimationstat.CO
Penalized estimation can conduct variable selection and parameter estimation simultaneously. The general framework is to minimize a loss function subject to a penalty designed to generate sparse variable selection. The majorization-minimization (MM) algorithm is a computational scheme for stability and simplicity, and …
Support vector machines (SVMs) with sparsity-inducing nonconvex penalties have received considerable attentions for the characteristics of automatic classification and variable selection. However, it is quite challenging to solve the nonconvex penalized SVMs due to their nondifferentiability, nonsmoothness and nonconve…
A new algorithm speeds up sparse-penalized quantile regression solving non-convex penalties.
problem Sparse-penalized quantile regression with non-convex penalties.
method Single-loop smoothing ADMM (SIAD) algorithm for faster convergence.
result SIAD method outperforms existing approaches in solving sparse-penalized quantile regression.
Developed a new algorithm to improve dynamic treatment regimens.
problem Non-convergence of Q-learning-based Q-shared algorithm in dynamic treatment regimens.
method Penalized Q-shared algorithm to address convergence issues.
result The penalized Q-shared algorithm converges and outperforms the original in various settings.
Paper reformulates UOT as non-negative penalized linear regression for efficient algorithms.
problem Optimal transport with relaxed marginal conditions.
method Reformulate UOT as non-negative penalized linear regression, propose multiplicative updates.
result Efficient algorithms for UOT with quadratic penalties, continuity of solutions.
Proposes a new robust expectile regression method for high-dimensional data.
problem Heterogeneity in high-dimensional data with heteroscedastic variance or inhomogeneous covariate effects.
method Iteratively reweighted ℓ1-penalization for robust expectile regression (retire).
result Oracle convergence rate after log(log d) iterations in high-dimensional settings.
Folded concave penalization methods have been shown to enjoy the strong oracle property for high-dimensional sparse estimation. However, a folded concave penalization problem usually has multiple local solutions and the oracle property is established only for one of the unknown local solutions. A challenging fundamenta…
In this paper we study nonconvex penalization using Bernstein functions. Since the Bernstein function is concave and nonsmooth at the origin, it can induce a class of nonconvex functions for high-dimensional sparse estimation problems. We derive a threshold function based on the Bernstein penalty and give its mathemati…
Recursive Group lassostat.ME
We introduce a recursive adaptive group lasso algorithm for real-time penalized least squares prediction that produces a time sequence of optimal sparse predictor coefficient vectors. At each time index the proposed algorithm computes an exact update of the optimal -penalized recursive least squares (R…
A l1-norm penalized orthogonal forward regression (l1-POFR) algorithm is proposed based on the concept of leaveone- out mean square error (LOOMSE). Firstly, a new l1-norm penalized cost function is defined in the constructed orthogonal space, and each orthogonal basis is associated with an individually tunable regulari…
A new Branch-and-Bound solver tackles L0-penalized problems with flexible loss functions.
problem Solving L0-penalized optimization problems with a broader class of loss functions.
method Generic Branch-and-Bound procedure with closed-form expressions for key quantities.
result El0ps solver achieves state-of-the-art performance and extends computational feasibility.
Optimal transport (OT) distances are finding evermore applications in machine learning and computer vision, but their wide spread use in larger-scale problems is impeded by their high computational cost. In this work we develop a family of fast and practical stochastic algorithms for solving the optimal transport probl…
We propose an algorithm, semismooth Newton coordinate descent (SNCD), for the elastic-net penalized Huber loss regression and quantile regression in high dimensional settings. Unlike existing coordinate descent type algorithms, the SNCD updates each regression coefficient and its corresponding subgradient simultaneousl…
We present an approach for penalized tensor decomposition (PTD) that estimates smoothly varying latent factors in multi-way data. This generalizes existing work on sparse tensor decomposition and penalized matrix decompositions, in a manner parallel to the generalized lasso for regression and smoothing problems. Our ap…
Distance weighted discrimination (DWD) was originally proposed to handle the data piling issue in the support vector machine. In this paper, we consider the sparse penalized DWD for high-dimensional classification. The state-of-the-art algorithm for solving the standard DWD is based on second-order cone programming, ho…
Researchers develop a method to learn CTBN structure using penalized maximum likelihood.
problem Learning the structure of continuous-time Bayesian networks (CTBNs) when the dependence structure is unknown.
method Penalized maximum likelihood method.
result The algorithm can accurately identify the network structure with high probability under certain conditions.
G-computation improves clinical trial power with machine learning.
problem Balancing prognostic factors in randomized trials to prevent near-confounders.
method G-computation with penalized models (Lasso, Elasticnet) and algorithm-based methods (neural network, SVM, super learner).
result G-computation with Elasticnet and splines reduces variance and increases power in RCTs.
We introduce a computationally effective algorithm for a linear model selection consisting of three steps: screening--ordering--selection (SOS). Screening of predictors is based on the thresholded Lasso that is l_1 penalized least squares. The screened predictors are then fitted using least squares (LS) and ordered wit…
Estimation in generalized linear models (GLM) is complicated by the presence of constraints. One can handle constraints by maximizing a penalized log-likelihood. Penalties such as the lasso are effective in high dimensions, but often lead to unwanted shrinkage. This paper explores instead penalizing the squared distanc…
High-dimensional data pose challenges in statistical learning and modeling. Sometimes the predictors can be naturally grouped where pursuing the between-group sparsity is desired. Collinearity may occur in real-world high-dimensional applications where the popular technique suffers from both selection inconsisten…
New priors improve robustness and interpretability in penalized regression.
problem Improper priors in penalized regression lead to suboptimal solutions.
method Developed non-zero priors inspired by human decision heuristics.
result Robust priors yield excellent worst-case performance across various tasks.
Alternating direction method of multipliers for penalized zero-variance discriminant analysisstat.ML
We consider the task of classification in the high dimensional setting where the number of features of the given data is significantly greater than the number of observations. To accomplish this task, we propose a heuristic, called sparse zero-variance discriminant analysis (SZVD), for simultaneously performing linear …
We propose a new sparsity-smoothness penalty for high-dimensional generalized additive models. The combination of sparsity and smoothness is crucial for mathematical theory as well as performance for finite-sample data. We present a computationally efficient algorithm, with provable numerical convergence properties, fo…
Many traditional signal recovery approaches can behave well basing on the penalized likelihood. However, they have to meet with the difficulty in the selection of hyperparameters or tuning parameters in the penalties. In this article, we propose a global adaptive generative adjustment (GAGA) algorithm for signal recove…
A new method for high-dimensional classification using Bernstein polynomials.
problem Computational difficulties in high-dimensional SVM hinge loss.
method Proposes Bernstein support vector machine (BernSVM) and two efficient algorithms.
result Achieves a prediction accuracy rate of with high probability.
CD converges linearly for MCP/SCAD penalized least squares.
problem Recovering sparse signals from data.
method Coordinate descent for MCP/SCAD penalized least squares.
result CD converges linearly to solutions of MCP/SCAD penalized least squares.
Paper discusses prediction errors for penalized regressions using GAMP and LOOCV.
problem Prediction accuracy of penalized regression models.
method Derives prediction error estimators using GAMP and LOOCV.
result Information criteria and LOOCV error estimators differ in large parameter regions.
In this paper we study nonconvex penalization using Bernstein functions whose first-order derivatives are completely monotone. The Bernstein function can induce a class of nonconvex penalty functions for high-dimensional sparse estimation problems. We derive a thresholding function based on the Bernstein penalty and di…
Solving linear programs by using entropic penalization has recently attracted new interest in the optimization community, since this strategy forms the basis for the fastest-known algorithms for the optimal transport problem, with many applications in modern large-scale machine learning. Crucial to these applications h…
New algorithms find optimal policies without knowing MDP span.
problem Finding optimal policies in MDPs without knowing span.
method Horizon calibration and span penalization techniques.
result First algorithms achieving optimal span-based complexity without prior knowledge.
Existing methods for sparse channel estimation typically provide an estimate computed as the solution maximizing an objective function defined as the sum of the log-likelihood function and a penalization term proportional to the l1-norm of the parameter of interest. However, other penalization terms have proven to have…
A framework estimates multiple precision matrices with shared structures.
problem Estimating multiple precision matrices with shared structures.
method Penalized likelihood framework with iterative algorithm alternating between convex and clustering problems.
result The method outperforms competitors and performs similarly to methods using prior information.
Increasing variance of losses improves learning with noisy labels.
problem Learning with noisy labels and the need to penalize variance of losses.
method Designing regularizers based on the label noise transition matrix to increase variance of losses.
result Increasing variance of losses significantly improves generalization ability.
Robustness in sparse linear models: relative efficiency based on robust approximate message passingmath.ST
Understanding efficiency in high dimensional linear models is a longstanding problem of interest. Classical work with smaller dimensional problems dating back to Huber and Bickel has illustrated the benefits of efficient loss functions. When the number of parameters is of the same order as the sample size , $p \…
Range penalization enhances statistical accuracy and resource efficiency in federated learning.
problem Statistical accuracy and resource efficiency in federated learning.
method Range regularization and polar clustering.
result Enhanced statistical accuracy and reduced iteration complexity.
It is well known that the out-of-sample performance of Markowitz's mean-variance portfolio criterion can be negatively affected by estimation errors in the mean and covariance. In this paper we address the problem by regularizing the mean-variance objective function with a weighted elastic net penalty. We show that the…
Improved Kriging model reduces prediction errors.
problem Improving prediction accuracy in Kriging models.
method Theta-regularized Kriging model with Lasso, Ridge, and Elastic-net penalties.
result The Theta-regularized Kriging model outperforms other penalized Kriging models in accuracy and stability.
Efficient algorithm reduces communication costs in sparse regression.
problem Sparse linear regression with massive data.
method CESDAR algorithm, communication-efficient surrogate likelihood.
result Achieves same statistical accuracy as global estimator with reduced communication.
Large-scale generalized linear array models (GLAMs) can be challenging to fit. Computation and storage of its tensor product design matrix can be impossible due to time and memory constraints, and previously considered design matrix free algorithms do not scale well with the dimension of the parameter vector. A new des…
We propose a general technique for improving alternating optimization (AO) of nonconvex functions. Starting from the solution given by AO, we conduct another sequence of searches over subspaces that are both meaningful to the optimization problem at hand and different from those used by AO. To demonstrate the utility o…