arXiv research

Study on when RLVR can learn compositional problems.

A generally intelligent learner should generalize to more complex tasks than it has previously encountered, but the two common paradigms in machine learning -- either training a separate learner per task or training a single learner for all tasks -- both have difficulty with such generalization because they do not leve…

This work is an analytical and numerical study of the composition of several fractals into one and of the relation between the composite dimension and the dimensions of the component fractals. In the case of composition of standard IFS with segments of equal size, the composite dimension can be expressed as a function …

C-ADAM is a new adaptive solver for complex nested problems.

Many machine learning, statistical inference, and portfolio optimization problems require minimization of a composition of expected value functions (CEVF). Of particular interest is the finite-sum versions of such compositional optimization problems (FS-CEVF). Compositional stochastic variance reduced gradient (C-SVRG)…

Consider the stochastic composition optimization problem where the objective is a composition of two expected-value functions. We propose a new stochastic first-order method, namely the accelerated stochastic compositional proximal gradient (ASC-PG) method, which updates based on queries to the sampling oracle using tw…

Tree-SMU enables strong compositional generalization in neural networks.

Develops consistent approximations for composite optimization problems.

Improved subgradient method tackles ill-conditioned composite optimization problems.

New method reduces complexity for nonconvex optimization problems.

AdaGrad fails to adapt to Hölder-smoothness in composite optimization problems.

Framework for lifelong learning of compositional structures.

The stochastic gradient descent has been widely used for solving composite optimization problems in big data analyses. Many algorithms and convergence properties have been developed. The composite functions were convex primarily and gradually nonconvex composite functions have been adopted to obtain more desirable prop…

Paper analyzes stability and generalization of SCO algorithms.

Unified algorithm for minimizing composite functions with flexible design.

FeDualEx tackles saddle point optimization in federated learning with composite objectives.

Paper proposes iLPA for solving DC composite optimization problems, with applications to matrix completion with outliers.

In this paper, we consider the convex and non-convex composition problem with the structure $\frac{1}{n}\sum\nolimits_{i = 1}^n {{F_i}( {G( x )} )}$, where $G( x )=\frac{1}{n}\sum\nolimits_{j = 1}^n {{G_j}( x )}$ is the inner function, and $F_i(\cdot)$ is the outer function. We explore the variance reduction based met…

Model predicts composite structures assembly quality with input uncertainty.

New algorithm reduces complexity for optimizing complex machine learning tasks.

New MCMC method for generating composition ratios.

We consider the composition optimization with two expected-value functions in the form of $\frac{1}{n}\sum\nolimits_{i = 1}^n F_i(\frac{1}{m}\sum\nolimits_{j = 1}^m G_j(x))+R(x)$, { which formulates many important problems in statistical learning and machine learning such as solving Bellman equations in reinforcement l…

Extends knockoff filter for composite null hypotheses in variable selection.

RICH models scenes as hierarchical tree to learn and generate complex compositions.

Routing networks tackle challenges in modular and compositional computation.

Study challenges neural models in compositional learning tasks.

This paper advances FL algorithms for composite optimization and statistical recovery.

Adapts Altman's model to compositional data for bankruptcy prediction.

New method for optimizing complex composite functions with reduced variance.

This paper introduces compositional data analysis for financial ratios, improving industry-level analysis.

Greedy coordinate descent achieves linear convergence for non-smooth composite problems.

New algorithm framework solves stochastic composite nonconvex optimization problems efficiently.

Paper solves robust convex problems with heavy-tailed noise.

Paper analyzes word embedding composition using tensor decomposition.

New samplers improve compositional generation with diffusion models.

New sparse GP model learns compositional kernels efficiently.

Classical stochastic gradient methods are well suited for minimizing expected-value objective functions. However, they do not apply to the minimization of a nonlinear function involving expected values or a composition of two expected-value functions, i.e., problems of the form $\min_x \mathbf{E}_v [f_v\big(\mathbf{E}_…

Here we study non-convex composite optimization: first, a finite-sum of smooth but non-convex functions, and second, a general function that admits a simple proximal mapping. Most research on stochastic methods for composite optimization assumes convexity or strong convexity of each function. In this paper, we extend t…

System composes polyphonic music using LSTM and RL.

PICLE uses probabilistic models to efficiently evaluate and compose modules for continual learning.

New algorithm solves composite optimization problems with unknown expectations.

Unified tractability conditions for various compositional inference queries.

We consider in this paper a class of composite optimization problems whose objective function is given by the summation of a general smooth and nonsmooth component, together with a relatively simple nonsmooth term. We present a new class of first-order methods, namely the gradient sliding algorithms, which can skip the…

Adaptive sampling method solves constrained and composite optimization problems.

New sampling methods for constrained and composite distributions.

In many applications one may acquire a composition of several signals that may be corrupted by noise, and it is a challenging problem to reliably separate the components from one another without sacrificing significant details. Adding to the challenge, in a compressive sensing framework, one is given only an undersampl…

Optimizes convergence rate of stochastic proximal algorithms for composite convex problems.

New conditions for weighted composition operators in group homomorphisms.