Generative model learns to compose images of objects from different distributions.
problem Capturing complex interactions between objects in scenes.
method Composition-by-Decomposition (CoDe) network.
result Model generates realistic composite images capturing interactions between input objects.
RICH models scenes as hierarchical tree to learn and generate complex compositions.
problem Learning compositional structures between parts and objects in natural scenes.
method RICH uses a latent scene graph to organize entities into a tree structure and employs a top-down inference approach.
result RICH learns and generates complex scene hierarchies from unlabeled data.
FeDualEx tackles saddle point optimization in federated learning with composite objectives.
problem Saddle point optimization with constraints and non-smooth regularization in federated learning.
method Federated Dual Extrapolation (FeDualEx) algorithm for saddle point optimization and composite objectives.
result FeDualEx effectively solves saddle point optimization problems with composite objectives in federated learning.
SCL discovers compositional structures in analogical reasoning tasks.
problem Discovering compositional structures in analogical reasoning tasks like Raven's Progressive Matrices.
method Proposes Scattering Compositional Learner (SCL) that composes neural networks in sequence.
result Achieves state-of-the-art performance on RPM datasets with significant improvements.
AdaGrad fails to adapt to Hölder-smoothness in composite optimization problems.
problem AdaGrad's convergence rate is suboptimal for composite objectives.
method Exhibited a simple one-dimensional convex problem to highlight AdaGrad's limitations.
result AdaGrad does not achieve the classical convergence rate for Hölder-smooth objectives.
ROOTS learns to represent and render 3D scenes with object-centric models.
problem Learning to represent and render 3D scenes with object-centric compositionality.
method Probabilistic generative model for learning object representations and scene rendering from partial observations.
result The model can infer 3D object representations and render scenes from arbitrary viewpoints.
Unified tensor model disentangles object appearance factors.
problem Representing hierarchical intrinsic and extrinsic causal factors of object appearance.
method Compositional hierarchical tensor factorization.
result Interpretable object representation robust to occlusion and reduced training data requirements.
Generative model improves scene graph prediction for rare compositions.
problem Challenges in inferring unseen or rare scene graph compositions.
method Synthesizing rare scene graphs by perturbing real ones, using conditional GANs.
result Marginal but consistent improvements in zero- and few-shot metrics.
C-SWMs learn structured world models from raw data.
problem Learning structured world models from raw sensory data.
method Contrastive learning with graph neural networks.
result C-SWMs outperform typical models in structured environments.
Bayesian optimization for composite functions improves efficiency.
problem Optimizing composite functions with expensive derivative-free evaluations.
method Exploits composite structure using multi-output Gaussian process and expected improvement for composite functions.
result Significantly improves sampling efficiency and asymptotically converges to global optimum.
Consider the stochastic composition optimization problem where the objective is a composition of two expected-value functions. We propose a new stochastic first-order method, namely the accelerated stochastic compositional proximal gradient (ASC-PG) method, which updates based on queries to the sampling oracle using tw…
Paper analyzes adaptive optimization algorithms and provides new insights.
problem Adaptive optimization algorithms for non-convex and composite objectives.
method New regret decomposition, Bregman divergences, modular analysis.
result Improved variational bounds and new optimistic MD algorithms.
Slot Attention extracts object-centric representations from images.
problem Learning distributed representations that don't capture natural scene composition.
method Slot Attention module interfaces with CNN outputs to produce task-dependent abstract slots.
result Slot Attention enables generalization to unseen compositions.
BoTier optimizes experiments by balancing multiple objectives hierarchically.
problem Balancing multiple competing objectives in scientific experiments.
method Composite objective that flexibly represents a hierarchy of preferences over outcomes and parameters.
result Demonstrates robust applicability across various use cases and seamless integration.
Unified algorithm for minimizing composite functions with flexible design.
problem Minimizing composite functions with specific structural properties.
method Unified accelerated algorithm for complementary composite minimization.
result Near-optimal algorithms for various optimization problems.
Improved GFlowNets learn more efficiently with trajectory balance.
problem Inefficient credit assignment in GFlowNets leads to suboptimal learning.
method Proposed trajectory balance as a new learning objective.
result Trajectory balance leads to more efficient and robust GFlowNet learning.
Language helps RL agents learn complex tasks by composing sub-skills.
problem Solving complex, temporally-extended tasks in RL.
method Using language as an abstraction to learn and reuse sub-skills.
result Agents can learn and solve diverse, temporally-extended tasks.
We consider the stochastic composition optimization problem proposed in \cite{wang2017stochastic}, which has applications ranging from estimation to statistical and machine learning. We propose the first ADMM-based algorithm named com-SVR-ADMM, and show that com-SVR-ADMM converges linearly for strongly convex and Lipsc…
New algorithm reduces complexity for optimizing complex machine learning tasks.
problem Optimizing complex machine learning objectives like reinforcement learning and portfolio management.
method Developed SARAH-Compositional algorithm using Stochastic Recursive Gradient Descent.
result Achieved optimal IFO complexity bounds for stochastic compositional optimization.
A tutorial on optimizing complex functions with partial knowledge.
problem Optimizing functions with limited or partial information.
method Grey-box Bayesian optimization, blending black-box and white-box approaches.
result Improves optimization performance by leveraging internal information.
New method separates objects from images using deep neural networks trained to inpaint.
problem Fully self-supervised instance separation of occluded objects in images.
method Maximizes independence of two image regions given a fully self-supervised inpainting network.
result Method achieves similar segmentation performance to fully supervised methods on microscopy image datasets.
New composite indicators reveal hidden relationships between indicators.
problem Subjective aggregation of indicators leads to missed information.
method Used dimensionality reduction techniques (PCA, filtering, clustering) to reveal hidden relationships.
result Cluster-driven composite indicators outperform traditional ones in data reconstruction.
New method solves complex optimization problems with reduced sample complexity.
problem Solving nonconvex stochastic nested optimization problems.
method Stochastic ADMM approach to find ε-stationary points.
result Total sample complexity of O(ε^(-3)) for online case and O((2N_1 + N_2) + (2N_1 + N_2)^(1/2)ε^(-2)) for finite sum case.
New algorithm framework solves stochastic composite nonconvex optimization problems efficiently.
problem Solving stochastic composite nonconvex optimization problems.
method ProxSARAH framework using SARAH estimator with proximal gradient and averaging steps.
result Achieves best-known complexity bounds with constant and adaptive step-sizes.
In this work we introduce a new optimisation method called SAGA in the spirit of SAG, SDCA, MISO and SVRG, a set of recently proposed incremental gradient algorithms with fast linear convergence rates. SAGA improves on the theory behind SAG and SVRG, with better theoretical convergence rates, and has support for compos…
Paper solves robust convex problems with heavy-tailed noise.
problem Solving convex compositional problems with heavy-tailed noise.
method Sub-Gaussian confidence bounds under weak heavy-tailed noise assumptions, using boosting strategy.
result Achieves nearly optimal high probability convergence result.
Improved subgradient method tackles ill-conditioned composite optimization problems.
problem Slow convergence of subgradient method for composite optimization problems.
method Preconditioned subgradient method with Levenberg-Marquardt approach.
result Linear convergence rate for composite optimization problems under mild conditions.
Study challenges neural models in compositional learning tasks.
problem Challenges in neural models for compositional and relational learning.
method Introduced ConceptWorld environment for generating images from compositional concepts, tested various neural architectures.
result Neural models struggle with longer compositional chains and substitutivity tests.
A new method for faster optimization of machine learning problems.
problem Minimization of composition of expected value functions.
method C-SAG, a novel extension of SAG for FS-CEVF problems.
result C-SAG achieves lower oracle query complexity per iteration than C-SVRG and converges faster.
A new geometry-preserving method for interpreting compositional data.
problem Statistical challenges in high-dimensional compositional data.
method Geometry-preserving framework for dimension reduction of compositional data.
result Identification of a central compositional subspace for compositional predictors.
Estimates input from output of nonlinear systems using ANN.
problem Estimating unknown compositional input from system output.
method Artificial Neural Networks (ANNs) for nonlinear system inversion.
result ANNs can compete with optimal bounds for linear systems and demonstrate promising results for nonlinear systems.
Within the unmanageably large class of nonconvex optimization, we consider the rich subclass of nonsmooth problems that have composite objectives---this already includes the extensively studied convex, composite objective problems as a special case. For this subclass, we introduce a powerful, new framework that permits…
Paper analyzes stability and generalization of SCO algorithms.
problem Understanding how SCO algorithms perform on unseen data.
method Algorithmic stability analysis in statistical learning theory.
result Derives dimension-independent excess risk bounds for SCGD and SCSC.
New filters match advanced composition for adaptive privacy, with practical constants.
problem Limitations of existing adaptive composition methods.
method Constructed new filters and odometers that match advanced composition rates, including constants.
result Achieved fully adaptive privacy with practical filters and odometers.
Generative Flow Networks use submodular upper bounds to generate more data.
problem Generating data from unknown, complex reward functions efficiently.
method Introduce submodular upper bounds to estimate reward, use Optimism in the Face of Uncertainty principle to train GFNs.
result SUBo-GFN generates significantly more data than classical GFNs.
Develops consistent approximations for composite optimization problems.
problem Significant errors in solutions due to approximations in optimization problems.
method Specifies conditions for well-behaved approximations in minimizers, stationary points, and level-sets for a broad class of composite problems.
result Framework of consistent approximations for composite problems, including stochastic, neural-network, and multi-objective optimization.
Constellation learns group-level visual relationships for abstract reasoning.
problem Learning configurational properties of entire groups of objects.
method Introduces Constellation, a network that learns relational abstractions over static visual scenes.
result Offers a basis for abstract relational reasoning and sensory imagination.
A new method for estimating random utility models using rank-breaking and composite marginal likelihood.
problem Estimating random utility models efficiently and accurately.
method Rank-breaking-then-composite-marginal-likelihood (RBCML) framework.
result RBCML achieves better statistical efficiency and computational efficiency than existing methods.
In this paper we develop proximal methods for statistical learning. Proximal point algorithms are useful in statistics and machine learning for obtaining optimization solutions for composite functions. Our approach exploits closed-form solutions of proximal operators and envelope representations based on the Moreau, Fo…
New method tackles composite optimization with error feedback.
problem Challenges in distributed machine learning training and message compression.
method Combines Dual Averaging with EControl for composite optimization.
result First strong convergence analysis for composite optimization with error feedback.
New framework infers multiple classes per image for one-shot learning.
problem Inferring multiple classes per image in one-shot learning.
method Compositional embedding framework with joint training of embedding and composition/query functions.
result Compositional embedding models outperform existing methods on various datasets.
Proposes a method to learn dynamic models for systems with variable number of objects.
problem Efficiently modeling systems with a variable number of objects.
method Uses graph neural networks and block-wise linear transition matrices to learn compositional Koopman operators.
result The method adapts to new environments and produces better control signals.
New method solves complex optimization problems efficiently.
problem Optimizing complex functions with inner expectations in machine learning.
method Combines variance reduction methods with duality-free techniques.
result Proves linear convergence for convex and non-convex cases.
This paper analyzes self-supervised learning from a multi-view perspective.
problem Understanding and optimizing self-supervised learning from multi-view data.
method Information-theoretical framework to understand and design self-supervised learning objectives.
result Self-supervised representations can extract task-relevant information and discard task-irrelevant information.
Develops minibatch stochastic proximal gradient for large-scale learning models.
problem Finding optimal predictors with complex regularizers in large-scale learning models.
method Minibatch variants of stochastic proximal gradient algorithm for composite objective functions.
result Minibatch size N after O(Nε1) iterations achieves ε−suboptimality in expected quadratic distance. Measures compositionality in machine learning representations.
problem Evaluating how compositional structure is reflected in learned representations.
method Measures compositionality by approximating true representation-producing models with composed primitives.
result Characterizes compositional structure in various settings.
Paper proposes iLPA for solving DC composite optimization problems, with applications to matrix completion with outliers.
problem Solving nonconvex and nonsmooth DC composite optimization problems.
method Inexact linearized proximal algorithm (iLPA) for DC composite optimization problems.
result The iLPA achieves local R-linear convergence rate under the Kurdyka-Łöjasiewicz property.
Simple model outperforms neural networks on language understanding tasks.
problem Neural networks struggle with creating novel expressions from familiar ones.
method Attention-inspired modification of a baseline model, focusing on sequential thinking and acting.
result Simple model achieves good performance on gSCAN tasks, validating the benchmark.