New method tackles nonconvex-nonconcave problems with local KL condition.
problem Nonconvex-nonconcave minimax problems under varying KL conditions.
method Inexact proximal gradient method for KL-structured subproblems.
result Complexity guarantees for approximate stationary points.
This paper improves inverse problem solving with weakly convex regularisers and proves convergence.
problem Improving solution methods for inverse problems.
method Generalised formulation of convergent regularisation using weakly convex regularisers, and proof of convergence for primal-dual hybrid gradient method.
result Proves convergence of primal-dual hybrid gradient method for variational problems and shows improved performance with IWCNNs.
Introduces PPMM algorithm for nonconvex robust regression problems.
problem Nonconvex tuning-free robust regression problems.
method PPMM algorithm with inner subproblems solved by SSN-PPA.
result Converges to d-stationary point with KL property.
Paper analyzes convergence rates of SGD for non-convex functions under various assumptions.
problem Analyzing convergence rates of SGD for non-convex functions.
method Studied convergence properties of Stochastic Gradient Descent (SGD) for invex functions under weaker and stronger hypotheses.
result Derives estimates on the rate of convergence of $J(oldsymbolθ_t)$ to its limit for functions satisfying the Polyak-Lojasiewicz (PL) condition.
In this paper, we consider the convergence of an abstract inexact nonconvex and nonsmooth algorithm. We promise a pseudo sufficient descent condition and a pseudo relative error condition, which are both related to an auxiliary sequence, for the algorithm; and a continuity condition is assumed to hold. In fact, a lot o…
PPGD solves nonconvex nonsmooth optimization problems without KL property.
problem Nonconvex and nonsmooth optimization problems in statistics and machine learning.
method Projective Proximal Gradient Descent (PPGD) for solving a class of nonconvex and nonsmooth problems.
result PPGD achieves a fast convergence rate of O(1/k^2) for k ≥ k_0.
New method solves complex constrained optimization problems.
problem Constrained nonconvex-nonconcave minimax optimization problems.
method Inexact proximal gradient method using sequential convex programming.
result Established complexity guarantees for approximate stationary points.
We derive bounds on the path length ζ of gradient descent (GD) and gradient flow (GF) curves for various classes of smooth convex and nonconvex functions. Among other results, we prove that: (a) if the iterates are linearly convergent with factor (1−c), then ζ is at most O(1/c); (b) under the Polyak-K…
Cubic-regularized Newton's method (CR) is a popular algorithm that guarantees to produce a second-order stationary solution for solving nonconvex optimization problems. However, existing understandings of the convergence rate of CR are conditioned on special types of geometrical properties of the objective function. In…
New algorithm solves ℓ0-norm constrained multilinear logistic regression for tensor data.
problem Non-convex and nonsmooth ℓ0-norm constraints in multilinear logistic regression. method APALM+ method for globally convergent optimization. result APALM+ ensures convergence to a first-order critical point. Training deep neural networks (DNNs) efficiently is a challenge due to the associated highly nonconvex optimization. The backpropagation (backprop) algorithm has long been the most widely used algorithm for gradient computation of parameters of DNNs and is used along with gradient descent-type algorithms for this optim…
The paper proves a margin inequality for separating hyperplanes, useful for analyzing algorithmic bias.
problem Analyzing the implicit bias of algorithms in machine learning.
method Proves a nonsmooth Kurdyka-Lojasiewicz inequality for margin function.
result The bias of algorithm iterates converges at least as fast as the square-root of the margin convergence rate.
Although ADAM is a very popular algorithm for optimizing the weights of neural networks, it has been recently shown that it can diverge even in simple convex optimization examples. Several variants of ADAM have been proposed to circumvent this convergence issue. In this work, we study the ADAM algorithm for smooth nonc…
In this paper we study nonconvex penalization using Bernstein functions whose first-order derivatives are completely monotone. The Bernstein function can induce a class of nonconvex penalty functions for high-dimensional sparse estimation problems. We derive a thresholding function based on the Bernstein penalty and di…
The paper proposes an efficient algorithm for solving Schatten-p quasi-norm problems.
problem Finding low-rank solutions of linear inverse problems with Schatten-p quasi-norm regularization. method Dynamic proximal gradient algorithm using Cayley transformation and adaptive step size selection.
result The algorithm converges to a stationary point of the objective function under mild assumptions.
Kurdyka-Lojasiewicz (KL) exponent plays an important role in estimating the convergence rate of many contemporary first-order methods. In particular, a KL exponent of 21 for a suitable potential function is related to local linear convergence. Nevertheless, KL exponent is in general extremely hard to estimate. I…
New method recovers matrices with nonlinear structures using optimization on Grassmann manifold.
problem Recovering high-rank matrices with nonlinear structures like subspaces or clusters.
method Formulated as rank minimization of a nonlinear feature map, approximated by constrained non-convex optimization on the Grassmann manifold, using Riemannian and alternating minimization schemes.
result Global convergence and worst-case complexity bounds for alternating minimization scheme, leading to unique limit point.
Nonconvex and nonsmooth problems have recently attracted considerable attention in machine learning. However, developing efficient methods for the nonconvex and nonsmooth optimization problems with certain performance guarantee remains a challenge. Proximal coordinate descent (PCD) has been widely used for solving opti…
Boosted Difference of Convex Functions Algorithm solves VaR constrained portfolio optimization.
problem Designing VaR optimal portfolios under financial regulations.
method Boosted Difference of Convex Functions Algorithm (BDCA) with a novel line search framework.
result BDCA linearly converges to a Karush-Kuhn-Tucker point for VaR constrained portfolio problems.
The great success of deep neural networks is built upon their over-parameterization, which smooths the optimization landscape without degrading the generalization ability. Despite the benefits of over-parameterization, a huge amount of parameters makes deep networks cumbersome in daily life applications. Though techniq…
Generates samples conditioned on labels using optimal transport.
problem Estimating conditional distributions for specific labels.
method Wasserstein geodesic generator based on optimal transport theory.
result Learned conditional distributions and optimal transport maps.
The paper classifies Finsler surfaces satisfying the T-condition or σT-condition.
problem Characterizing Finsler surfaces based on specific tensor conditions.
method Analyzing Finsler surfaces in dimensions n≥3, proving conditions equivalence, and solving PDEs.
result All Finsler surfaces satisfying the T-condition or σT-condition are classified.
The paper develops a new approach to conditional risk measures using modular convex analysis.
problem Developing a new method for conditional risk measures.
method Random modular approach to conditional certainty equivalents and niveloids in the conditional L∞-space. result Retrieves a conditional variational formula for optimized certainty equivalents and applies it to the conditional entropic risk measure.
Paper constructs solutions to Bogomolny equations with specific boundary and asymptotic conditions.
problem Constructing solutions to Bogomolny equations with given boundary and asymptotic conditions.
method Using generalized Nahm pole boundary condition and real symmetry breaking condition.
result Solutions analogous to instanton solutions, satisfying different asymptotic conditions.
We extend probabilistic programming to handle conditioning on marginal distributions.
problem Conditioning probabilistic programs on marginal distributions of observable variables.
method We define and implement stochastic conditioning, allowing inference in probabilistic programs conditioned on marginal distributions.
result We demonstrate the effectiveness of stochastic conditioning in various real-life scenarios.
New tests for conditional copulas based on decision trees.
problem Testing constancy of conditional dependence structure given conditioning events.
method Data-driven decision trees to maximize differences in conditional Kendall's tau.
result Asymptotic distributions of test statistics under the null hypothesis.
Paper finds necessary condition for logarithmic Minkowski problem in higher dimensions.
problem Logarithmic Minkowski problem in higher dimensions.
method Established a necessary condition through generalization and refinement of previous work.
result Generalizes and refines necessary condition for logarithmic Minkowski problem.
This paper introduces a neural operator for probabilistic conditioning.
problem Probabilistic conditioning of random variables X given Y. method Develops a single operator that maps any joint density to its conditional, approximated by neural operators.
result Neural operators can approximate the conditioning operator to arbitrary accuracy.
CSI method learns conditional distributions by estimating flow equations.
problem Learning conditional distributions in generative models.
method Estimates probability flow equations to transport reference to target distribution.
result Derives explicit expressions for conditional drift and score functions.
New conditional risk measures called conditional generalized quantiles defined and characterized.
problem Developing new risk measures for dynamic risk assessment.
method Propose and characterize conditional generalized quantiles using expected utility model and equivalent conditions.
result Characterized conditional generalized quantiles as well-defined and equivalent to a conditional first order condition.
A new method for learning conditional distributions using ODEs and neural networks.
problem Learning conditional distributions efficiently and accurately.
method Conditional Föllmer Flow, discretized with Euler's method, using nonparametric velocity estimation.
result Effective approximation of target conditional distributions, with convergence results for Wasserstein-2 distance.
Sharp statistical theory for conditional diffusion models.
problem Lack of theoretical foundation for conditional diffusion models.
method Sharp statistical theory with approximation of conditional score function.
result Sample complexity bound that adapts to data distribution smoothness.
An analysis is made of reality conditions within the context of noncommutative geometry. We show that if a covariant derivative satisfies a given left Leibniz rule then a right Leibniz rule is equivalent to the reality condition. We show also that the matrix which determines the reality condition must satisfy the Yang-…
New conditions prevent gaps in optimal control problems.
problem Preventing gaps in optimal control problems with state constraints.
method Developed new sufficient conditions not relying on convexity.
result Derived bounds for the size of the relaxation gap.
We identify conditional parity as a general notion of non-discrimination in machine learning. In fact, several recently proposed notions of non-discrimination, including a few counterfactual notions, are instances of conditional parity. We show that conditional parity is amenable to statistical analysis by studying ran…
We consider families of strongly consistent multivariate conditional risk measures. We show that under strong consistency these families admit a decomposition into a conditional aggregation function and a univariate conditional risk measure as introduced Hoffmann et al. (2016). Further, in analogy to the univariate cas…
Proposes a new method for interpreting feature importance and effects in dependent feature models.
problem Challenges in interpreting feature importance when features are dependent and interactions are present.
method Conditional Subgroup Approach
result Conditional PFI and PDP estimates based on this approach often outperform existing methods.
New boundary conditions solve Cauchy problem for Dirac operators on spacetimes.
problem Understanding non-local boundary conditions for Dirac operators on spacetimes.
method Define and analyze a class of Lorentzian boundary conditions that are local in time and non-local in spatial directions.
result Well-posed Cauchy problem for the Dirac operator is established under these conditions.
We extend CS divergence to conditional distributions and show its advantages in time series data and sequential decision making.
problem Quantifying the closeness between conditional distributions.
method Developed and estimated a conditional Cauchy-Schwarz divergence using kernel density estimation.
result Conditional CS divergence outperforms previous methods in time series clustering and sequential decision making.
The Samuelson condition is not satisfied by tangent lines of quadratic curves.
problem Area condition for Lagrangian 2-web
method Show that the Samuelson condition is not satisfied
result The Samuelson condition is not satisfied by tangent lines of quadratic curves.
We describe a Groebner basis of relations among conditional probabilities in a discrete probability space, with any set of conditioned-upon events. They may be specialized to the partially-observed random variable case, the purely conditional case, and other special cases. We also investigate the connection to generali…
A new method tests conditional independence by transforming it into an unconditional problem using transport maps.
problem Testing conditional independence between two random vectors given a third.
method Constructing transport maps to transform conditional independence into unconditional independence, estimating these maps from data using conditional continuous normalizing flow models.
result The proposed method is validated through simulations and real-data analysis, demonstrating practical effectiveness.
Generalizes moment-matching for exponential families with conditioning or hidden data.
problem Generalizing moment-matching conditions for exponential families with conditioning or hidden data.
method First-principles explanation and self-contained derivation of generalized moment-matching conditions.
result Derives generalized moment-matching conditions for conditional exponential families and hidden data.
DG algorithms often fail to generalize well in limited domains, highlighting necessary vs. sufficient conditions.
problem DG algorithms fail to consistently outperform ERM in limited domains.
method Examined necessary and sufficient conditions for DG, proposing a subspace alignment method.
result DG methods focus on sufficient conditions, often neglecting necessary conditions, leading to generalization failures.
In this paper, we examine higher order difference problems. Using the "squeezing" argument, we derive both Euler's condition and the transversality condition. In order to derive the two conditions, two needed assumptions are identified. A counterexample, in which the transversality condition is not satisfied without th…
Study on estimating conditional risk in machine learning.
problem Estimating expected loss of prediction models given input features.
method Analyzed in classification and regression settings, showing equivalence to standard regression. Developed theoretical insights and empirical validation.
result Conditional risk calibration is distinct from existing uncertainty quantification problems.
The paper extends static Systemic Risk Measures to a conditional setting.
problem Investigating how static Systemic Risk Measures can be adapted to a conditional framework.
method Providing a general dual representation result, analyzing Conditional Shortfall Systemic Risk Measures, and providing explicit formulas for exponential preferences.
result Explicit formulas for Conditional Shortfall Systemic Risk Measures and a time consistency property.
This paper simplifies conditional Sobol' indices calculation using PCE bases.
problem Computational inefficiency and lack of consistency in evaluating conditional Sobol' indices.
method Analytical extraction of conditional Sobol' indices via basis decomposition of PCE expansions.
result Derives closed-form expressions for conditional Sobol' indices.