arXiv research

Study on when RLVR can learn compositional problems.

A generally intelligent learner should generalize to more complex tasks than it has previously encountered, but the two common paradigms in machine learning -- either training a separate learner per task or training a single learner for all tasks -- both have difficulty with such generalization because they do not leve…

This work is an analytical and numerical study of the composition of several fractals into one and of the relation between the composite dimension and the dimensions of the component fractals. In the case of composition of standard IFS with segments of equal size, the composite dimension can be expressed as a function …

C-ADAM is a new adaptive solver for complex nested problems.

Many machine learning, statistical inference, and portfolio optimization problems require minimization of a composition of expected value functions (CEVF). Of particular interest is the finite-sum versions of such compositional optimization problems (FS-CEVF). Compositional stochastic variance reduced gradient (C-SVRG)…

Consider the stochastic composition optimization problem where the objective is a composition of two expected-value functions. We propose a new stochastic first-order method, namely the accelerated stochastic compositional proximal gradient (ASC-PG) method, which updates based on queries to the sampling oracle using tw…

Develops consistent approximations for composite optimization problems.

Improved subgradient method tackles ill-conditioned composite optimization problems.

AdaGrad fails to adapt to Hölder-smoothness in composite optimization problems.

Framework for lifelong learning of compositional structures.

The stochastic gradient descent has been widely used for solving composite optimization problems in big data analyses. Many algorithms and convergence properties have been developed. The composite functions were convex primarily and gradually nonconvex composite functions have been adopted to obtain more desirable prop…

We study compositional generalization, viz., the problem of zero-shot generalization to novel compositions of concepts in a domain. Standard neural networks fail to a large extent on compositional learning. We propose Tree Stack Memory Units (Tree-SMU) to enable strong compositional generalization. Tree-SMU is a recurs…

Paper analyzes stability and generalization of SCO algorithms.

Unified algorithm for minimizing composite functions with flexible design.

FeDualEx tackles saddle point optimization in federated learning with composite objectives.

Paper proposes iLPA for solving DC composite optimization problems, with applications to matrix completion with outliers.

In this paper, we consider the convex and non-convex composition problem with the structure $\frac{1}{n}\sum\nolimits_{i = 1}^n {{F_i}( {G( x )} )}$, where $G( x )=\frac{1}{n}\sum\nolimits_{j = 1}^n {{G_j}( x )}$ is the inner function, and $F_i(\cdot)$ is the outer function. We explore the variance reduction based met…

Model predicts composite structures assembly quality with input uncertainty.

We consider the composition optimization with two expected-value functions in the form of $\frac{1}{n}\sum\nolimits_{i = 1}^n F_i(\frac{1}{m}\sum\nolimits_{j = 1}^m G_j(x))+R(x)$, { which formulates many important problems in statistical learning and machine learning such as solving Bellman equations in reinforcement l…

Extends knockoff filter for composite null hypotheses in variable selection.

Study challenges neural models in compositional learning tasks.

Stochastic compositional optimization arises in many important machine learning tasks such as value function evaluation in reinforcement learning and portfolio management. The objective function is the composition of two expectations of stochastic functions, and is more challenging to optimize than vanilla stochastic o…

This paper advances FL algorithms for composite optimization and statistical recovery.

Adapts Altman's model to compositional data for bankruptcy prediction.

This paper introduces compositional data analysis for financial ratios, improving industry-level analysis.

We propose a novel probabilistic approach to multilevel clustering problems based on composite transportation distance, which is a variant of transportation distance where the underlying metric is Kullback-Leibler divergence. Our method involves solving a joint optimization problem over spaces of probability measures t…

Paper solves robust convex problems with heavy-tailed noise.

New samplers improve compositional generation with diffusion models.

Compositional structures between parts and objects are inherent in natural scenes. Modeling such compositional hierarchies via unsupervised learning can bring various benefits such as interpretability and transferability, which are important in many downstream tasks. In this paper, we propose the first deep latent vari…

New sparse GP model learns compositional kernels efficiently.

We consider the problem of minimizing the composition of a smooth (nonconvex) function and a smooth vector mapping, where the inner mapping is in the form of an expectation over some random variable or a finite sum. We propose a stochastic composite gradient method that employs an incremental variance-reduced estimator…

Here we study non-convex composite optimization: first, a finite-sum of smooth but non-convex functions, and second, a general function that admits a simple proximal mapping. Most research on stochastic methods for composite optimization assumes convexity or strong convexity of each function. In this paper, we extend t…

Classical stochastic gradient methods are well suited for minimizing expected-value objective functions. However, they do not apply to the minimization of a nonlinear function involving expected values or a composition of two expected-value functions, i.e., problems of the form $\min_x \mathbf{E}_v [f_v\big(\mathbf{E}_…

In the domain of algorithmic music composition, machine learning-driven systems eliminate the need for carefully hand-crafting rules for composition. In particular, the capability of recurrent neural networks to learn complex temporal patterns lends itself well to the musical domain. Promising results have been observe…

PICLE uses probabilistic models to efficiently evaluate and compose modules for continual learning.

Unified tractability conditions for various compositional inference queries.

We consider in this paper a class of composite optimization problems whose objective function is given by the summation of a general smooth and nonsmooth component, together with a relatively simple nonsmooth term. We present a new class of first-order methods, namely the gradient sliding algorithms, which can skip the…

We consider multi-level composite optimization problems where each mapping in the composition is the expectation over a family of random smooth mappings or the sum of some finite number of smooth mappings. We present a normalized proximal approximate gradient (NPAG) method where the approximate gradients are obtained v…

Adaptive sampling method solves constrained and composite optimization problems.

In many applications one may acquire a composition of several signals that may be corrupted by noise, and it is a challenging problem to reliably separate the components from one another without sacrificing significant details. Adding to the challenge, in a compressive sensing framework, one is given only an undersampl…

New sampling methods for constrained and composite distributions.

Optimizes convergence rate of stochastic proximal algorithms for composite convex problems.

New conditions for weighted composition operators in group homomorphisms.

New algorithm tackles nested bi-level optimization problems for robust feature learning.

This paper explores optimising acquisition functions in Bayesian optimisation.

This is a survey of our research on geometric structures of projective embeddings and includes some topics of our talks in several symposia during 1990-99. We clarify our main problem, which is to construct a kind of geometric composition series of projective embeddings. The concept of "geometric composition series" is…

Deep learning can learn compositional functions more efficiently by breaking them into stages.

Word embedding is a powerful tool in natural language processing. In this paper we consider the problem of word embedding composition \--- given vector representations of two words, compute a vector for the entire phrase. We give a generative model that can capture specific syntactic relations between words. Under our …