arXiv research

This work is an analytical and numerical study of the composition of several fractals into one and of the relation between the composite dimension and the dimensions of the component fractals. In the case of composition of standard IFS with segments of equal size, the composite dimension can be expressed as a function …

Study on deep neural networks using branching processes and Mehler's formula.

The paper defines functions that induce bounded composition operators on RKHSs with analytic positive definite functions.

The stochastic gradient descent has been widely used for solving composite optimization problems in big data analyses. Many algorithms and convergence properties have been developed. The composite functions were convex primarily and gradually nonconvex composite functions have been adopted to obtain more desirable prop…

New algorithm reduces complexity for optimizing complex machine learning tasks.

This paper explores optimising acquisition functions in Bayesian optimisation.

Paper tackles gradient estimation for large-scale composition problems.

Deep networks learn sparse hierarchical features without CoD.

Bayesian optimization for composite functions improves efficiency.

Improved subgradient method tackles ill-conditioned composite optimization problems.

New method reduces complexity for nonconvex optimization problems.

Gaussian processes struggle with compositional functions, but deep Gaussian processes can outperform.

A new method for faster optimization of machine learning problems.

C-ADAM is a new adaptive solver for complex nested problems.

Deep neural networks can approximate complex functions through repeated compositions of a fixed-size ReLU network.

We consider composite loss functions for multiclass prediction comprising a proper (i.e., Fisher-consistent) loss over probability distributions and an inverse link function. We establish conditions for their (strong) convexity and explore the implications. We also show how the separation of concerns afforded by using …

We study losses for binary classification and class probability estimation and extend the understanding of them from margin losses to general composite losses which are the composition of a proper loss with a link function. We characterise when margin losses can be proper composite losses, explicitly show how to determ…

New method for optimizing complex composite functions with reduced variance.

Paper proposes iLPA for solving DC composite optimization problems, with applications to matrix completion with outliers.

New framework infers multiple classes per image for one-shot learning.

Improved Q-learning for robot control with faster data-efficiency.

DGPs collapse to near-deterministic transformations, limiting their compositional structure discovery.

We consider the composition optimization with two expected-value functions in the form of $\frac{1}{n}\sum\nolimits_{i = 1}^n F_i(\frac{1}{m}\sum\nolimits_{j = 1}^m G_j(x))+R(x)$, { which formulates many important problems in statistical learning and machine learning such as solving Bellman equations in reinforcement l…

Deep learning can learn compositional functions more efficiently by breaking them into stages.

Sublinearly structured DNNs achieve feature learning consistency for compositional functions.

Corrects an error in isospectral lens spaces formula for composite q.

Defines 'nowhere coexpanding functions' and studies their fixed points.

Here we study non-convex composite optimization: first, a finite-sum of smooth but non-convex functions, and second, a general function that admits a simple proximal mapping. Most research on stochastic methods for composite optimization assumes convexity or strong convexity of each function. In this paper, we extend t…

Unified algorithm for minimizing composite functions with flexible design.

New geometric approach for analyzing compositional data like gut microbiomes.

This work studies fairness in systems of multiple algorithms, addressing pitfalls and constructing fair compositions.

Unified analysis of multi-task functional linear regression with manifold and composite penalties.

Model predicts composite structures assembly quality with input uncertainty.

Agents learn new tasks by combining existing skills in RL.

Consider the stochastic composition optimization problem where the objective is a composition of two expected-value functions. We propose a new stochastic first-order method, namely the accelerated stochastic compositional proximal gradient (ASC-PG) method, which updates based on queries to the sampling oracle using tw…

BOIS optimizes complex systems by leveraging structural knowledge.

This work proves composite neural networks outperform their components under certain conditions.

New method for causal inference with complex treatment compositions.

Composite neural network improves PM2.5 prediction.

In this paper we determine a number of meaningful compositions of higher order of a set of functions, which is considered in Malesevic (1998), in implicit and explicit form. Results which are obtained are applied to the vector analysis in order to determine the number of meaningful differential operations of higher ord…

Deep networks better approximate functions with compositional structure.

Curvature penalties improve interpretability of KANs without sacrificing accuracy.

We consider in this work a system of two stochastic differential equations named the perturbed compositional gradient flow. By introducing a separation of fast and slow scales of the two equations, we show that the limit of the slow motion is given by an averaged ordinary differential equation. We then demonstrate that…

Classical stochastic gradient methods are well suited for minimizing expected-value objective functions. However, they do not apply to the minimization of a nonlinear function involving expected values or a composition of two expected-value functions, i.e., problems of the form $\min_x \mathbf{E}_v [f_v\big(\mathbf{E}_…

New samplers improve compositional generation with diffusion models.

New sparse GP model learns compositional kernels efficiently.

New algorithm samples efficiently from complex composite potentials.

In this contribution we describe an approach to evolve composite covariance functions for Gaussian processes using genetic programming. A critical aspect of Gaussian processes and similar kernel-based models such as SVM is, that the covariance function should be adapted to the modeled data. Frequently, the squared expo…