We propose a DC proximal Newton algorithm for solving nonconvex regularized sparse learning problems in high dimensions. Our proposed algorithm integrates the proximal Newton algorithm with multi-stage convex relaxation based on the difference of convex (DC) programming, and enjoys both strong computational and statist…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “proximal DC algorithm”
A new algorithm solves signed Fréchet regression on manifolds with bounded curvature.
problem Signed Fréchet regression on Riemannian manifolds with bounded curvature.
method Proximal DC algorithm (FRIDA) for computing signed Fréchet regression fits.
result Existence and interiority of minimizers, strong convexity of proximal subproblems, and convergence to stationary points.
We introduce a novel algorithm for solving learning problems where both the loss function and the regularizer are non-convex but belong to the class of difference of convex (DC) functions. Our contribution is a new general purpose proximal Newton algorithm that is able to deal with such a situation. The algorithm consi…
An inexact LPA for DC composite optimization and application to matrix completions with outliersmath.OC
Paper proposes iLPA for solving DC composite optimization problems, with applications to matrix completion with outliers.
problem Solving nonconvex and nonsmooth DC composite optimization problems.
method Inexact linearized proximal algorithm (iLPA) for DC composite optimization problems.
result The iLPA achieves local R-linear convergence rate under the Kurdyka-Łöjasiewicz property.
Difference of convex (DC) functions cover a broad family of non-convex and possibly non-smooth and non-differentiable functions, and have wide applications in machine learning and statistics. Although deterministic algorithms for DC functions have been extensively studied, stochastic optimization that is more suitable …
Paper solves high-order portfolio optimization with cardinality constraint.
problem Solving non-convex cardinality constrained high-order portfolio optimization.
method Transformed cardinality constraint into penalty term, proposed pDCA, pDCAe, and SCA algorithms.
result Proposed algorithms achieve high utility and sparse solutions efficiently.
We consider the problem of minimizing a difference-of-convex (DC) function, which can be written as the sum of a smooth convex function with Lipschitz gradient, a proper closed convex function and a continuous possibly nonsmooth concave function. We refine the convergence analysis in [38] for the proximal DC algorithm …
Paper proposes DC functions for better regularization of inverse problems with theoretical guarantees.
problem Improving regularization for ill-posed inverse problems.
method Introduces difference-of-convex (DC) functions and uses them with optimization algorithms like DCA and PSM.
result DC functions yield improved performance and theoretical guarantees compared to weakly convex functions.
New algorithms improve submodular minimization via DC programming.
problem Minimizing the difference of two submodular functions.
method Introducing variants of the DC algorithm (DCA) and its complete form (CDCA) for DC programs corresponding to DS minimization.
result Our algorithms outperform existing baselines on speech corpus selection and feature selection.
KL property of exponent of -norm and DC regularized factorizations for low-rank matrix recoverymath.OC
This paper is concerned with the factorization form of the rank regularized loss minimization problem. To cater for the scenario in which only a coarse estimation is available for the rank of the true matrix, an -norm regularized term is added to the factored loss function to reduce the rank adaptively; and…
Joint blind source separation (J-BSS) is an emerging data-driven technique for multi-set data-fusion. In this paper, J-BSS is addressed from a tensorial perspective. We show how, by using second-order multi-set statistics in J-BSS, a specific double coupled canonical polyadic decomposition (DC-CPD) problem can be formu…
Sparse optimization refers to an optimization problem involving the zero-norm in objective or constraints. In this paper, nonconvex approximation approaches for sparse optimization have been studied with a unifying point of view in DC (Difference of Convex functions) programming framework. Considering a common DC appro…
: A Divide-and-conquer Algorithm for Large-scale Kernel Learning with Application to Clusteringcs.LG
Divide-and-conquer is a general strategy to deal with large scale problems. It is typically applied to generate ensemble instances, which potentially limits the problem size it can handle. Additionally, the data are often divided by random sampling which may be suboptimal. To address these concerns, we propose the $DC^…
Unified SVM algorithm for various losses with fast training.
problem Training SVM models with different convex or nonconvex losses.
method Introducing LS-DC loss, proposing DCA-based UniSVM algorithm.
result Unified algorithm solves SVM models with any convex or nonconvex LS-DC loss efficiently.
This work builds a sensor graph from DC sensors for anomaly detection.
problem Anomaly detection in data centers with complex sensor relationships.
method Data-driven pipeline (ts2graph) to build a sensor graph from sensor time series.
result Graph neural network (GNN) outperforms existing methods by 2-3 times in anomaly detection.
S3VDC improves DC methods for scalability, stability, and simplicity.
problem Poor scalability, instability, and lack of simplicity in DC methods.
method Four algorithmic improvements: initial -training, periodic -annealing, mini-batch GMM initialization, and inverse min-max transform. S3VDC incorporates all improvements.
result S3VDC outperforms state-of-the-art methods on benchmark and industrial datasets.
Data parallelism has become the de facto standard for training Deep Neural Network on multiple processing units. In this work we propose DC-S3GD, a decentralized (without Parameter Server) stale-synchronous version of the Delay-Compensated Asynchronous Stochastic Gradient Descent (DC-ASGD) algorithm. In our approach, w…
Novel coordinate descent (CD) methods are proposed for minimizing nonconvex functions consisting of three terms: (i) a continuously differentiable term, (ii) a simple convex term, and (iii) a concave and continuous term. First, by extending randomized CD to nonsmooth nonconvex settings, we develop a coordinate subgradi…
Majorization-minimization algorithms consist of iteratively minimizing a majorizing surrogate of an objective function. Because of its simplicity and its wide applicability, this principle has been very popular in statistics and in signal processing. In this paper, we intend to make this principle scalable. We introduc…
DCA algorithm applied to SVR with RBF kernel for nonconvex optimization.
problem Nonconvex optimization of SVR with Gaussian RBF kernel.
method DC algorithm with analytical DC decomposition of SVR objective.
result Convergence properties of DCA on RBF-SVR can be assessed through .
The Data Clustering (DC) problem is of central importance for the area of Machine Learning (ML), given its usefulness to represent data structural similarities from input spaces. Differently from Supervised Machine Learning (SML), which relies on the theoretical frameworks of the Statistical Learning Theory (SLT) and t…
New method uses momentum to converge in DC optimization with small batches.
problem Lack of convergence properties for stochastic difference-of-convex optimization with small batch sizes.
method Introduces momentum to enable convergence under standard assumptions for any batch size.
result Proves convergence of the algorithm under smoothness and bounded variance assumptions.
Group-Lasso (gLasso) identifies important explanatory factors in predicting the response variable by considering the grouping structure over input variables. However, most existing algorithms for gLasso are not scalable to deal with large-scale datasets, which are becoming a norm in many applications. In this paper, we…
This paper reports applications of Difference of Convex functions (DC) programming to Learning from Demonstrations (LfD) and Reinforcement Learning (RL) with expert data. This is made possible because the norm of the Optimal Bellman Residual (OBR), which is at the heart of many RL and LfD algorithms, is DC. Improvement…
Non-convex sparsity-inducing penalties have recently received considerable attentions in sparse learning. Recent theoretical investigations have demonstrated their superiority over the convex counterparts in several sparse learning settings. However, solving the non-convex optimization problems associated with non-conv…
Interactive recommender systems that enable the interactions between users and the recommender system have attracted increasing research attentions. Previous methods mainly focus on optimizing recommendation accuracy. However, they usually ignore the diversity of the recommendation results, thus usually results in unsa…
By exploiting the property that the RBM log-likelihood function is the difference of convex functions, we formulate a stochastic variant of the difference of convex functions (DC) programming to minimize the negative log-likelihood. Interestingly, the traditional contrastive divergence algorithm is a special case of th…
DC Calculusmath.DG
In this paper, we extend the DC Calculus introduced by Perelman on finite dimensional Alexandrov spaces with curvature bounded below. Among other things, our results allow us to define the Hessian and the Laplacian of DC functions (including distance functions as a particular instance) as a measure-valued tensor and a …
DC-Check: A Data-Centric AI checklist to guide the development of reliable machine learning systemscs.LG
DC-Check helps guide ML development by considering data-centric aspects.
problem Lack of standardized framework for data-centric considerations in ML.
method DC-Check is a checklist-style framework for data-centric AI at ML pipeline stages.
result Promotes thoughtfulness and transparency in ML development.
When will a server fail catastrophically in an industrial datacenter? Is it possible to forecast these failures so preventive actions can be taken to increase the reliability of a datacenter? To answer these questions, we have studied what are probably the largest, publicly available datacenter traces, containing more …
Investigates risk measures for DC pension decumulation.
problem Develop optimal decumulation strategies for DC plan holders.
method Formulates decumulation as a control problem, studies risk measures (expected shortfall, linear shortfall, probability of shortfall).
result Optimal controls for expected reward and expected shortfall are identical to those for expected reward and linear shortfall.
X-DC improves speech separation by making DNNs more interpretable.
problem Black-box nature of DNNs in speech separation tasks.
method Introduces X-DC, a DNN architecture that interprets as spectrogram template fitting followed by Wiener filtering.
result X-DC achieves comparable speech separation performance to DC but with enhanced interpretability.
A new nonparametric approach for system identification has been recently proposed where the impulse response is modeled as the realization of a zero-mean Gaussian process whose covariance (kernel) has to be estimated from data. In this scheme, quality of the estimates crucially depends on the parametrization of the cov…
Introduces PPMM algorithm for nonconvex robust regression problems.
problem Nonconvex tuning-free robust regression problems.
method PPMM algorithm with inner subproblems solved by SSN-PPA.
result Converges to d-stationary point with KL property.
In machine learning research, the proximal gradient methods are popular for solving various optimization problems with non-smooth regularization. Inexact proximal gradient methods are extremely important when exactly solving the proximal operator is time-consuming, or the proximal operator does not have an analytic sol…
This paper tackles noise in raw datasets to improve representation learning efficiency.
problem Noise in real-world datasets degrades representation learning quality.
method Proposes denoising Cosine-Similarity (dCS) loss to learn robust representations.
result Empirical results show the dCS loss outperforms baseline objective functions.
FGTSVA improves Thompson Sampling for contextual bandits with optimal variance-aware regret.
problem Optimizing regret bounds for Thompson Sampling in contextual bandits.
method Developed FGTSVA, a variance-aware Thompson Sampling algorithm for contextual bandits with a new decoupling coefficient.
result Achieved optimal regret bound of .
Paper analyzes convergence of proximal algorithm in metric spaces without geodesic convexity.
problem Analyzing convergence of proximal algorithm in general metric spaces.
method Analysis of the Wasserstein proximal algorithm without geodesic convexity assumption.
result Establishes unbiased and linear convergence rate for proximal algorithm under natural Wasserstein inequality.
The Thresholding Bandit Problem (TBP) aims to find the set of arms with mean rewards greater than a given threshold. We consider a new setting of TBP, where in addition to pulling arms, one can also \emph{duel} two arms and get the arm with a greater mean. In our motivating application from crowdsourcing, dueling two a…
The classification of multi-class microarray datasets is a hard task because of the small samples size in each class and the heavy overlaps among classes. To effectively solve these problems, we propose novel Error Correcting Output Code (ECOC) algorithm by Enhance Class Separability related Data Complexity measures du…
The paper proposes a new method to predict VaR using DCS and generalized distributions.
problem Improving VaR prediction models in financial risk management.
method Dynamic Conditional Score (DCS) model combined with generalized distributions (GD).
result The proposed model outperforms traditional models in high-risk VaR prediction.
This paper tackles multi-marginal optimal transport problems using DC programming.
problem Multi-marginal optimal transport problems in machine learning.
method Promoting structural information in MMOT leads to a DC programming problem.
result Solutions from DC optimization are as qualitative as current methods.
SDF-Bayes finds safe drug combinations safely, balancing optimism and caution.
problem Finding safe drug combinations in clinical trials with multiple drugs and patient heterogeneity.
method SDF-Bayes uses Bayesian statistics to choose the most likely MTD while ensuring safety constraints.
result SDF-Bayes outperforms existing methods in both accuracy and safety for drug combination trials.
Proximal algorithms applied to current deformation into cycles.
problem Deformation of de Rham currents into cycles.
method Proximal algorithms, total variation denoising for differential forms.
result Calibrated cycles constructed in calibrated manifolds.
The optimal investment strategy of a DC pension plan under deposit loan spread and the O-U processq-fin.PM
This paper optimizes DC pension plan investments using O-U process and loan.
problem Optimizing investment strategy for DC pension plans under specific market conditions.
method Dynamic programming and Hamilton-Jacobi-Bellman equation to derive optimal investment strategy.
result Explicit expression for optimal investment strategy derived.
Faster algorithms solve convex function learning problems.
problem Approximating arbitrary convex functions in various learning tasks.
method 2-block ADMM method with closed-form block computations.
result Converges with for convex Lipschitz regression.
A relaxed proximal gradient descent algorithm for convergent plug-and-play with proximal denoiserstat.ML
New PnP algorithm converges with relaxed proximal gradient descent.
problem Convergence issues in PnP methods with deep denoisers.
method Relaxed proximal gradient descent for PnP with weakly convex regularization.
result Proposed PnP-PGD converges for a wider range of regularization parameters.
Improved bounds for proximal gradient algorithms with computational errors.
problem Analyzing convergence of proximal gradient algorithms with inaccuracies.
method Deriving new tighter deterministic and probabilistic bounds for convex composite problems.
result Probabilistic bounds are more robust and accurate for algorithm verification and performance guarantees.