SR3 framework improves sparse regression solutions.
problem Sparse regression problems in various fields.
method SR3 framework solves relaxed regularized regression problems.
result SR3 provides superior solutions with faster algorithms.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,181 papers · 148 categories
Trend · papers per month
48 results for “relaxed regularized regression”
Safe screening rules reduce computation time in logistic regression with regularization.
problem Efficiently solving logistic regression with many features and regularization.
method Screening rules based on Fenchel dual lower bounds of strong conic relaxations.
result A high percentage of features can be safely removed before solving, leading to substantial speed-up.
In this note we compare two recently proposed semidefinite relaxations for the sparse linear regression problem by Pilanci, Wainwright and El Ghaoui (Sparse learning via boolean relaxations, 2015) and Dong, Chen and Linderoth (Relaxation vs. Regularization A conic optimization perspective of statistical variable select…
New convex relaxations solve sparse regression problems efficiently.
problem Sparse regression with constraint is NP-hard.
method Rank-one convexification for semidefinite optimization.
result Stronger and more general convex relaxations for sparse regression.
Improved robustness in multivariate regression and classification with DRO under Wasserstein metric.
problem Outliers in covariates and responses.
method Distributionally Robust Optimization (DRO) with Wasserstein metric ambiguity set and regularization.
result Significant improvement in predictive error and robustness.
Safe screening rules reduce -regression computation by fixing 76% of variables.
problem Efficiently solving -regression problems with large datasets.
method Convex relaxation and safe screening rules to eliminate variables.
result 76% of variables can be fixed to their optimal values, reducing computational burden.
Relaxed Sparse Eigenvalue Conditions for Sparse Estimation via Non-convex Regularized Regressioncs.LG
Non-convex regularizers usually improve the performance of sparse estimation in practice. To prove this fact, we study the conditions of sparse estimations for the sharp concave regularizers which are a general family of non-convex regularizers including many existing regularizers. For the global solutions of the regul…
New method clusters variables using robust nodewise regression.
problem Variable clustering in multi-factor models.
method Distributionally robust nodewise regression with convex relaxation and ADMM.
result Superior performance in numerical studies.
Joint sparsity regularization in multi-task learning has attracted much attention in recent years. The traditional convex formulation employs the group Lasso relaxation to achieve joint sparsity across tasks. Although this approach leads to a simple convex formulation, it suffers from several issues due to the loosenes…
NPMR uses nuclear norm penalty for multinomial regression, predicting baseball outcomes.
problem Predicting at bat outcomes in baseball with improved accuracy.
method Nuclear penalized multinomial regression (NPMR) applied to MLB data.
result NPMR provides better prediction probabilities for batter-pitcher matchups.
Improving the Tightness of Convex Relaxation Bounds for Training Certifiably Robust Classifierscs.LG
New regularizers tighten convex relaxation bounds for neural networks.
problem Large gap between certifiable and empirical robustness in neural networks.
method Two regularizers to train neural networks yielding tighter convex relaxation bounds.
result Higher certified accuracy with proposed regularizers.
New regularizer for machine learning using private data.
problem Machine learning with private data.
method Distributionally-robust optimization with locally-differentially-private datasets.
result New regularizer for training linear regression models.
The paper analyzes trace regression with low-rank matrices under various regularization methods.
problem Estimating low-rank matrices with near-optimal error bounds under unknown regularization parameters.
method General spikiness notion, restricted strong convexity of sampling operator, cross-validation for parameter selection.
result Cross-validated estimators select near-optimal penalty parameters and outperform theory-inspired approaches.
Regularization helps protect machine learning models from poisoning attacks.
problem Mitigating the impact of poisoned data on machine learning models.
method Distributionally-robust optimization using Wasserstein distance to find an upper bound for worst-case fitness.
result The regularizer is equal to the dual norm of the model parameters for regression models.
Sparse Mixed Linear Regression with Guarantees: Taming an Intractable Problem with Invex Relaxationcs.LG
Paper solves NP-hard sparse mixed linear regression problem with provable guarantees.
problem Sparse mixed linear regression on unlabeled data.
method Invex relaxation for intractable problem with theoretical guarantees.
result Exact recovery of data labels and close approximation of regression parameters.
Paper reformulates UOT as non-negative penalized linear regression for efficient algorithms.
problem Optimal transport with relaxed marginal conditions.
method Reformulate UOT as non-negative penalized linear regression, propose multiplicative updates.
result Efficient algorithms for UOT with quadratic penalties, continuity of solutions.
Variable (feature, gene, model, which we use interchangeably) selections for regression with high-dimensional BIGDATA have found many applications in bioinformatics, computational biology, image processing, and engineering. One appealing approach is the L0 regularized regression which penalizes the number of nonzero fe…
We study the problem of learning a sparse linear regression vector under additional conditions on the structure of its sparsity pattern. This problem is relevant in machine learning, statistics and signal processing. It is well known that a linear regression can benefit from knowledge that the underlying regression vec…
We present a Distributionally Robust Optimization (DRO) approach to estimate a robustified regression plane in a linear regression setting, when the observed samples are potentially contaminated with adversarially corrupted outliers. Our approach mitigates the impact of outliers through hedging against a family of dist…
Develops robust learning framework under distributional perturbations.
problem Learning robust to data distributional changes.
method Distributionally Robust Optimization (DRO) under Wasserstein metric.
result Establishes performance guarantees and tractable formulations.
Develops a robust multiclass classification method for deep image classifiers.
problem Tackles data contamination and robustness to outliers in deep image classifiers.
method Uses Distributionally Robust Optimization (DRO) with Wasserstein metric ambiguity sets and regularized learning.
result Reduces test error rate by up to 83.5% and loss by up to 91.3% in image classification tasks.
New conic quadratic formulations improve outlier detection in regression models.
problem Detecting outliers in regression models with corrupted data.
method Deriving stronger second-order conic relaxations without big-M constraints.
result Proposed formulations are significantly faster than existing methods.
Braids can be represented geometrically as laminations of punctured disks. The geometric complexity of a braid is the minimal complexity of a lamination that represents it, and tight laminations are representatives of minimal complexity. These laminations give rise to a normal form of braids, via a relaxation algorithm…
A relaxed proximal gradient descent algorithm for convergent plug-and-play with proximal denoiserstat.ML
New PnP algorithm converges with relaxed proximal gradient descent.
problem Convergence issues in PnP methods with deep denoisers.
method Relaxed proximal gradient descent for PnP with weakly convex regularization.
result Proposed PnP-PGD converges for a wider range of regularization parameters.
Finding efficient and provable methods to solve non-convex optimization problems is an outstanding challenge in machine learning and optimization theory. A popular approach used to tackle non-convex problems is to use convex relaxation techniques to find a convex surrogate for the problem. Unfortunately, convex relaxat…
Introduces Soft-SVM for binary classification bridging logistic and SVM.
problem Data separability issues in binary classification.
method Soft-SVM regression using convex relaxation of hinge loss with softness and class-separation parameters.
result Soft-SVM performs well in classification and prediction errors.
Optimal transport relaxations improve Wasserstein GAN training efficiency.
problem Training inefficiency in Wasserstein GANs.
method Optimal transport relaxations with a minimization step over a small region.
result Running time improvements with no performance degradation.
Robust Nonparametric Regression via Sparsity Control with Application to Load Curve Data Cleansingstat.ML
Nonparametric methods are widely applicable to statistical inference problems, since they rely on a few modeling assumptions. In this context, the fresh look advocated here permeates benefits from variable selection and compressive sampling, to robustify nonparametric regression against outliers - that is, data markedl…
Study identifies key differences in convex relaxations for combinatorial penalties.
problem Understanding which structures are preserved by convex relaxations for combinatorial penalties.
method Examined homogeneous and non-homogeneous convex relaxations, introduced lower combinatorial envelope, and proposed adaptive estimator.
result Identified new necessary and sufficient conditions for support recovery in convex monotone regularizers.
New MIP framework solves high-dimensional -regularized regression problems.
problem Exact computation of -regularized regression estimators is challenging for large .
method Specialized nonlinear branch-and-bound (BnB) framework with first-order optimization.
result Achieves speedups of at least 5000x compared to state-of-the-art exact methods.
Study introduces weak elastic energy for curves on Riemannian surfaces.
problem Detecting curvature of curves on Riemannian surfaces.
method Relaxation starting from inscribed geodesic polygonals, defined in normalized isothermal coordinates.
result Relaxed energy detects intrinsic second-order Sobolev regularity and agrees with geodesic curvature.
Algorithm recovers function samples from noisy modulo samples with high probability.
problem Recovering function samples from noisy modulo samples.
method Two-stage algorithm involving k-NN regression and SDP relaxation.
result Uniform error rate of for function samples.
Regularization vs. Relaxation: A conic optimization perspective of statistical variable selectioncs.LG
Variable selection is a fundamental task in statistical data analysis. Sparsity-inducing regularization methods are a popular class of methods that simultaneously perform variable selection and model estimation. The central problem is a quadratic optimization problem with an l0-norm penalty. Exactly enforcing the l0-no…
For a bounded smooth domain in the plane and smooth boundary data we consider the minimisation of the Willmore functional for graphs subject to Dirichlet or Navier boundary conditions. For -regular graphs we show that bounds for the Willmore energy imply area and diameter bounds. We then consider the -lower s…
Proposes ESCFR to estimate treatment effects from biased data.
problem Treatment selection bias in observational data.
method Stochastic optimal transport with relaxed mass-preserving and proximal factual outcome regularizers.
result Significantly better performance in estimating treatment effects.
Paper optimizes private ReLU regression with relaxed assumptions.
problem Private ReLU regression with relaxed assumptions.
method One-pass mini-batch Generalized Linear Model Perceptron algorithm (DP-MBGLMtron).
result Achieves optimal utility bound up to logarithmic factors.
A new generative model relaxes the bijectivity requirement for invertible flows.
problem Challenges of invertible flow-based models in scaling to large datasets.
method Proposes a generative model based on relaxed injective probability flows.
result Improves sample quality over VAEs and AEs.
SDHR improves data hashing for better classification accuracy.
problem Efficient data hashing for high-dimensional data retrieval.
method Supervised Discrete Hashing with Relaxation (SDHR) using optimized regression targets.
result SDHR outperforms traditional methods in classification accuracy.
New method for fair regression using optimal transport.
problem Learning fair regression models under counterfactual fairness constraints.
method Causal uncertainty view, optimal transport, post-processing method.
result High-probability fairness guarantees with decay.
A number of recent work studied the effectiveness of feature selection using Lasso. It is known that under the restricted isometry properties (RIP), Lasso does not generally lead to the exact recovery of the set of nonzero coefficients, due to the looseness of convex relaxation. This paper considers the feature selecti…
We study the problem of learning a tensor from a set of linear measurements. A prominent methodology for this problem is based on a generalization of trace norm regularization, which has been used extensively for learning low rank matrices, to the tensor setting. In this paper, we highlight some limitations of this app…
We propose a novel adversarial training method in feature space that improves model robustness and computational efficiency.
problem Improving model robustness against adversarial input perturbations with computational efficiency.
method Shift from input to feature-space perturbations, reformulating the adversarial training problem in reproducing kernel Hilbert spaces, enabling exact solution of inner maximization and efficient optimization.
result The feature-perturbed formulation is a relaxation of the original problem and provides a regularized estimator that adapts to noise and function smoothness.
DARTS fails to generalize well; adding regularization improves robustness.
problem DARTS fails to find architectures that generalize well across different tasks.
method Identified failure modes, added regularization, proposed variations.
result Regularization robustifies DARTS to find better generalizing architectures.
Renet: Principled and Efficient Relaxation for the Elastic Net via Dynamic Objective Selectionstat.ME
Renet improves Elastic Net by dynamically selecting between convex blending and refitting, enhancing prediction accuracy.
problem Elastic Net's shrinkage bias limits its prediction accuracy in high-dimensional settings.
method Adaptive relaxation procedure that dynamically dispatches between convex blending and efficient sub-path refitting.
result Renet consistently outperforms standard Elastic Net and Adaptive Elastic Net in high-dimensional, low signal-to-noise ratio, and high-multicollinearity scenarios.
VaSST uses soft symbolic trees for probabilistic symbolic regression.
problem Efficiently recover symbolic expressions from noisy data.
method Variational inference with soft symbolic trees.
result Superior performance in structural recovery and predictive accuracy.
The paper analyzes the convergence rates of smooth message passing algorithms in entropy-regularized MAP inference.
problem Finding the most likely configuration in graphical models with combinatorial optimization.
method Entropy-regularized linear programming relaxations and smooth message passing algorithms.
result The number of iterations sufficient to recover the true integral MAP solution is determined.
RQR improves prediction intervals for skewed data.
problem Invalid prediction intervals for skewed noise.
method Relaxed Quantile Regression (RQR) for asymmetric noise.
result Improved prediction intervals with desirable qualities.
New nonconvex regularizers improve low-rank matrix recovery efficiency and accuracy.
problem Efficiently recover low-rank matrices from incomplete data.
method Factor group-sparse regularization, related to Schatten-p norms.
result Improved generalization error bounds for Schatten-p norms as p decreases.