The CUR decomposition provides an approximation of a matrix that has low reconstruction error and that is sparse in the sense that the resulting approximation lies in the span of only a few columns of . In this regard, it appears to be similar to many sparse PCA methods. However, CUR takes a randomized algorithm…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “Sparse Optimization”
In this paper, a sparse Markov decision process (MDP) with novel causal sparse Tsallis entropy regularization is proposed.The proposed policy regularization induces a sparse and multi-modal optimal policy distribution of a sparse MDP. The full mathematical analysis of the proposed sparse MDP is provided.We first analyz…
New DP optimization methods for sparse gradients, improving on existing algorithms.
problem Differentially private optimization with sparse gradients in high-dimensional settings.
method Improved bounds for mean estimation, pure- and approximate-DP algorithms for stochastic convex optimization.
result First nearly dimension-independent rates for DP optimization with sparse gradients.
New methods solve sparse estimation robustly, even with outliers.
problem Sparse estimation in high-dimensional data with outliers.
method Non-convex optimization formulations for robust sparse mean estimation and PCA.
result Any approximate stationary point yields near-optimal solutions.
Optimizes sparse mean-reverting portfolios for higher returns.
problem Finding optimal stock weights for mean-reverting portfolios.
method Transformed optimization problem into SDP, added constraints.
result Sparse mean-reverting portfolios provide higher returns with transaction costs.
New method improves robust sparse association estimation.
problem Outliers in high-dimensional data.
method Splitting robust estimation into optimization phases, using augmented Lagrangian and adaptive gradient descent.
result Improved precision over existing methods.
We study the sparse entropy-regularized reinforcement learning (ERL) problem in which the entropy term is a special form of the Tsallis entropy. The optimal policy of this formulation is sparse, i.e.,~at each state, it has non-zero probability for only a small number of actions. This addresses the main drawback of the …
A new framework compresses neural networks using sparse optimization.
problem Efficiently reducing the size of deep neural networks for practical deployment.
method Sparse optimization for model compression, tailored for stochastic learning.
result Up to 7.2 and 2.9 times FLOPs reduction with comparable accuracy.
Paper tackles NAS problem by modeling it as a sparse supernet.
problem Neural Architecture Search (NAS) problem, particularly Mixed-Path Search.
method Model NAS as a sparse supernet with sparsity constraints. Use hierarchical accelerated proximal gradient algorithm for optimization.
result Proposed method finds compact, general, and powerful neural architectures.
Paper connects neural network hyperparameter optimization and NAS to structured sparse recovery.
problem Hyperparameter optimization and neural architecture search in neural networks.
method Structured sparse recovery methods applied to HPO and NAS.
result Improvements in hyperparameter optimization and discovery of novel neural architectures.
This work introduces a method to compare sparse neural network topologies using graph theory.
problem Comparing and understanding sparse neural network topologies, especially during training.
method Introducing Neural Network Sparse Topology Distance (NNSTD) to measure distances between different sparse neural networks.
result Sparse neural networks can outperform over-parameterized models without further structure optimization.
Sparse perturbations improve convergence in SZO methods for faster training.
problem Dependency of SZO methods on function dimensionality limits their convergence speed.
method Sparse perturbations reduce the effective dimensionality of the optimization problem.
result Sparse SZO optimization leads to faster convergence in training loss and test accuracy.
BO method identifies sparse subspaces for efficient high-dimensional optimization.
problem Efficient optimization of high-dimensional black-box functions.
method Sparse Gaussian process surrogate models on axis-aligned subspaces with Hamiltonian Monte Carlo inference.
result SAASBO achieves excellent performance on synthetic and real-world problems.
Picasso is a new library for sparse learning problems in R and Python.
problem Sparse learning problems in high-dimensional data analysis.
method Unified framework of pathwise coordinate optimization with efficient active set selection strategies.
result picasso can efficiently handle large-scale problems.
Paper proposes a new method to optimize deep neural networks with sparse regularization.
problem Difficulty in achieving optimal convergence rates for deep neural networks due to sparsity constraints.
method Introduces a novel penalized estimation method for sparse DNNs, resolving computational and theoretical issues.
result Establishes an oracle inequality for the excess risk of the proposed sparse-penalized DNN estimator and derives convergence rates.
OKRidge solves sparse ridge regression problems for nonlinear systems.
problem Identifying sparse governing equations for nonlinear dynamical systems.
method OKRidge algorithm using saddle point formulation and ADMM-based approach with efficient proximal operators.
result OKRidge achieves provable optimality with significantly faster run times than Gurobi.
Dynamic Sparse Training: Find Efficient Sparse Network From Scratch With Trainable Masked Layerscs.LG
Dynamic Sparse Training finds efficient sparse networks from scratch.
problem Finding efficient sparse neural networks.
method Jointly optimizes network parameters and sparsity with trainable thresholds.
result Achieves state-of-the-art performance with minimal performance loss.
Gradient descent with early stopping achieves optimal sparse recovery.
problem Sparse regression with gradient descent and early stopping.
method Gradient descent on depth-N networks with early stopping.
result Implicit sparse regularization occurs with early stopping for general depth N.
Sparse codes improve optimal control tasks with correlated inputs.
problem Optimal control tasks with correlated feature inputs.
method Used a sparse code to represent natural images in an optimal control task solved with neuro-dynamic programming.
result An over-complete sparse code increases memory capacity and learning speed beyond a complete code.
New method solves sparse approximation problem using trimmed lasso and generalized soft-min penalties.
problem Sparse approximation or best subset selection problem.
method Regularized approach with trimmed lasso and generalized soft-min penalties.
result The trimmed lasso provides sparse recovery guarantees and a practical optimization algorithm.
The generalized linear model (GLM) plays a key role in regression analyses. In high-dimensional data, the sparse GLM has been used but it is not robust against outliers. Recently, the robust methods have been proposed for the specific example of the sparse GLM. Among them, we focus on the robust and sparse linear regre…
Solving l1 regularized optimization problems is common in the fields of computational biology, signal processing and machine learning. Such l1 regularization is utilized to find sparse minimizers of convex functions. A well-known example is the LASSO problem, where the l1 norm regularizes a quadratic function. A multil…
This paper addresses the problem of sparsity penalized least squares for applications in sparse signal processing, e.g. sparse deconvolution. This paper aims to induce sparsity more strongly than L1 norm regularization, while avoiding non-convex optimization. For this purpose, this paper describes the design and use of…
We study sparse approximate solutions to convex optimization problems. It is known that in many engineering applications researchers are interested in an approximate solution of an optimization problem as a linear combination of elements from a given system of elements. There is an increasing interest in building such …
Accelerates optimization in asynchronous systems with sparse updates.
problem Optimizing finite-sum objectives in asynchronous lock-free environments.
method New accelerated SVRG variant with sparse updates.
result Achieves optimal incremental gradient complexity.
A new knot selection method speeds up sparse Gaussian process approximations.
problem Efficiently selecting knots for sparse Gaussian processes.
method One-at-a-time Bayesian optimization for knot selection.
result Competitive performance with reduced computational cost.
Improved Sparse Polyak for high-dimensional M-estimation with sparser solutions.
problem High-dimensional M-estimation problems with potential loss of sparsity and accuracy.
method Variant of Sparse Polyak with optimal thresholding operators.
result Retains desirable scaling properties while achieving sparser and more accurate solutions.
Paper optimizes privacy-preserving distribution estimation for sparse data.
problem Sparse distribution estimation under local differential privacy constraints.
method Compressive sensing approaches for privacy-preserving estimation.
result Significant reduction in sample complexity for approximately sparse distributions.
ACOWA improves distributed sparse classification with extra communication round.
problem Efficiently optimizing sparse classification with limited communication.
method Introducing ACOWA, a new technique with an extra communication round.
result ACOWA achieves better approximation quality and higher accuracy.
We perform a finite sample analysis of the detection levels for sparse principal components of a high-dimensional covariance matrix. Our minimax optimal test is based on a sparse eigenvalue statistic. Alas, computing this test is known to be NP-complete in general, and we describe a computationally efficient alternativ…
New methods show sparse portfolios offer no advantage over mean-variance in diversification.
problem Investment diversification and risk management with sparse portfolios.
method Developed and implemented a new estimation procedure for sparse second-order stochastic spanning using a greedy algorithm and Linear Programming.
result No benefit from expanding a sparse opportunity set beyond 45 assets; optimal sparse portfolio reduces tail risk.
Principal components analysis (PCA) is the optimal linear auto-encoder of data, and it is often used to construct features. Enforcing sparsity on the principal components can promote better generalization, while improving the interpretability of the features. We study the problem of constructing optimal sparse linear a…
Robust estimators for Gaussian sparse tasks with optimal error under contamination.
problem Robust mean estimation, PCA, and linear regression in the presence of Huber contamination.
method Novel multidimensional filtering method for sparse regime.
result Optimal error guarantees within constant factors for Gaussian robust -sparse mean estimation.
DFR reduces the computational cost of sparse-group lasso and adaptive sparse-group lasso.
problem Sparse-group lasso's computational expense and need for tuning.
method Dual Feature Reduction (DFR) using strong screening rules and dual norms.
result DFR drastically reduces computational cost without affecting solution optimality.
Sparse optimization refers to an optimization problem involving the zero-norm in objective or constraints. In this paper, nonconvex approximation approaches for sparse optimization have been studied with a unifying point of view in DC (Difference of Convex functions) programming framework. Considering a common DC appro…
Sparse Bayesian Optimization (SEBO) finds interpretable configurations.
problem Optimizing black-box functions for recommendation systems while maintaining interpretability.
method Regularization-based approaches, including a differentiable relaxation for regularization, and a hyperparameter-free method SEBO.
result SEBO efficiently optimizes for sparsity without hyperparameters.
Scalable Inference of Sparsely-changing Markov Random Fields with Strong Statistical Guaranteescs.LG
Efficiently infers time-varying sparse MRFs with strong statistical guarantees.
problem Inference of time-varying sparse MRFs with strong statistical guarantees.
method Constrained optimization with exact regularization, near-linear time and memory complexity.
result Sharp statistical guarantees for sparsely-changing Gaussian MRFs with as few as one sample per time.
We study how well one can recover sparse principal components of a data matrix using a sketch formed from a few of its elements. We show that for a wide class of optimization problems, if the sketch is close (in the spectral norm) to the original data matrix, then one can recover a near optimal solution to the optimiza…
Sparse generalized eigenvalue problem (GEP) plays a pivotal role in a large family of high-dimensional statistical models, including sparse Fisher's discriminant analysis, canonical correlation analysis, and sufficient dimension reduction. Sparse GEP involves solving a non-convex optimization problem. Most existing met…
L0Learn solves sparse learning problems with millions of features.
problem Sparse learning problems with millions of features.
method Approximate algorithms based on coordinate descent and local combinatorial optimization.
result Achieves competitive run times and statistical performance.
We propose and study a general framework for regularized Markov decision processes (MDPs) where the goal is to find an optimal policy that maximizes the expected discounted total reward plus a policy regularization term. The extant entropy-regularized MDPs can be cast into our framework. Moreover, under our framework, …
Sparsity-constrained optimization has wide applicability in machine learning, statistics, and signal processing problems such as feature selection and compressive Sensing. A vast body of work has studied the sparsity-constrained optimization from theoretical, algorithmic, and application aspects in the context of spars…
EFS: Evolutionary Factor Searching for Sparse Portfolio Optimization Using Large Language Modelsq-fin.PM
EFS uses LLMs to optimize sparse portfolios by evolving alpha factors.
problem Sparse portfolio optimization in dynamic market regimes.
method Evolutionary feedback loop with LLM-generated alpha factors.
result Significantly outperforms baselines in diverse datasets.
We present a robust alternative to principal component analysis (PCA) --- called elliptical component analysis (ECA) --- for analyzing high dimensional, elliptically distributed data. ECA estimates the eigenspace of the covariance matrix of the elliptical data. To cope with heavy-tailed elliptical distributions, a mult…
The paper examines properties of GW optimal transport plans, showing they can be sparse and permutation-supported.
problem Properties of Gromov-Wasserstein optimal transport plans.
method Exploration of sparsity, permutation support, and cyclical monotonicity properties.
result GW optimal plans can be sparse and permutation-supported under certain conditions.
Nonnegative CANDECOMP/PARAFAC (NCP) decomposition is an important tool to process nonnegative tensor. Sometimes, additional sparse regularization is needed to extract meaningful nonnegative and sparse components. Thus, an optimization method for NCP that can impose sparsity efficiently is required. In this paper, we co…
PopArt: Efficient Sparse Regression and Experimental Design for Optimal Sparse Linear Banditsstat.ML
PopArt efficiently solves sparse linear bandits with tighter recovery guarantees.
problem Sparse linear bandits where rewards depend on a few covariates.
method PopArt: a simple, computationally efficient sparse linear estimation method.
result Improved regret bounds compared to state-of-the-art algorithms.
We investigate the difficulties of training sparse neural networks and make new observations about optimization dynamics and the energy landscape within the sparse regime. Recent work of \citep{Gale2019, Liu2018} has shown that sparse ResNet-50 architectures trained on ImageNet-2012 dataset converge to solutions that a…