Routing networks tackle challenges in modular and compositional computation.
problem Challenges in learning and training compositional models with module parameters and their composition.
method Routing networks as a general approach to address these challenges, examining the interplay of algorithmic decisions.
result Empirical analysis of routing networks reveals the interplay of challenges and design decisions.
Paper extends FFT-based differential privacy method to heterogeneous compositions.
problem Computing accurate differential privacy guarantees for mixed mechanisms.
method Uses Fast Fourier Transform (FFT) for error analysis and parameter selection.
result Provides tighter bounds for heterogeneous compositions compared to homogeneous cases.
Compositional data have two unique characteristics compared to typical multivariate data: the observed values are nonnegative and their summand is exactly one. To reflect these characteristics, a specific regularized regression model with linear constraints is commonly used. However, linear constraints incur additional…
Quandle colorings and cocycle invariants are studied for composite knots, and applied to chirality and abelian extensions. The square and granny knots, for example, can be distinguished by quandle colorings, so that a trefoil and its mirror can be distinguished by quandle invariants of composite knots. We investigate t…
DGPs collapse to near-deterministic transformations, limiting their compositional structure discovery.
problem Limitations of variational inference in DGPs lead to suboptimal posterior approximations.
method Examine alternative variational inference schemes allowing for dependencies across different layers.
result Alternative variational inference schemes can better capture the compositional structure in DGPs.
We present new computations of tight shapes obtained using the constrained gradient descent code RIDGERUNNER for 544 composite knots with 12 and fewer crossings, expanding our dataset to 943 knots and links. We use the new data set to analyze two outstanding conjectures about tight knots, namely that the ropelengths of…
Gaussian process (GP) models provide a powerful tool for prediction but are computationally prohibitive using large data sets. In such scenarios, one has to resort to approximate methods. We derive an approximation based on a composite likelihood approach using a general belief updating framework, which leads to a recu…
Paper simplifies DP composition for adaptive privacy budgets, enabling better privacy and accuracy in deep learning.
problem Tension between efficiency and flexibility in DP composition theorems.
method Rényi Differential Privacy (RDP) for adaptive privacy budgets, proving simpler composition theorem with smaller constants.
result Practical DP composition for adaptive privacy budgets, enabling better privacy and accuracy in deep learning.
New geometric approach for analyzing compositional data like gut microbiomes.
problem Analyzing non-negative compositional data with relative values only.
method Reinterpret compositional data as quotient topology of a sphere, using spherical harmonics and reflection group actions.
result Construction of Reproducing Kernel Hilbert Space (RKHS) for compositional data.
Heegaard Floer homology study confirms composition maps match up to homotopy.
problem Verifying consistency of composition maps in Heegaard Floer homology.
method Using Auroux and Zemke's results, proving agreement up to homotopy.
result Proves consistency of composition maps in Heegaard Floer homology.
New framework infers multiple classes per image for one-shot learning.
problem Inferring multiple classes per image in one-shot learning.
method Compositional embedding framework with joint training of embedding and composition/query functions.
result Compositional embedding models outperform existing methods on various datasets.
We compute the second coefficient of the composition of two Berezin-Toeplitz operators associated with the spinc Dirac operator on a symplectic manifold, making use of the full-off diagonal expansion of the Bergman kernel.
New insights into privacy guarantees for subsampled mechanisms under composition.
problem Tight privacy guarantees for the composition of subsampled differentially private mechanisms.
method Addressed confusion points in privacy accounting for subsampled mechanisms, providing examples and counterexamples.
result Privacy guarantees for subsampled mechanisms differ significantly between Poisson subsampling and sampling without replacement.
Edgeworth Accountant calculates privacy loss under differential privacy compositions efficiently.
problem Efficiently computing overall privacy loss under composition of private algorithms.
method Analytical approach using f-differential privacy framework and Edgeworth expansion. result Non-asymptotic (ε,δ)-differential privacy bounds with reduced computational cost. PICLE uses probabilistic models to efficiently evaluate and compose modules for continual learning.
problem Challenging search space of module compositions in continual learning.
method Probabilistic framework to cheaply compute module compositions' fitness.
result First modular CL algorithm to achieve perceptual, few-shot, and latent transfer.
New sparse GP model learns compositional kernels efficiently.
problem Learning accurate Gaussian Process models with complex kernel structures.
method MultiSVGP model with Horseshoe prior for kernel selection.
result Our model provides better fit and faster computation for large-scale data.
This paper introduces compositional data analysis for financial ratios, improving industry-level analysis.
problem Statistical issues with standard financial ratios at industry level.
method Compositional data analysis techniques for financial ratios.
result Improved analysis of financial ratios using compositional data methods.
A generally intelligent learner should generalize to more complex tasks than it has previously encountered, but the two common paradigms in machine learning -- either training a separate learner per task or training a single learner for all tasks -- both have difficulty with such generalization because they do not leve…
Cosmos models scenes using neural encodings and symbolic attributes for compositional generalization.
problem Modeling scenes with high performance on unseen input scenes composed of known visual elements.
method Neurosymbolic grounding with neurosymbolic scene encodings and attention mechanisms.
result Establishes a new state-of-the-art for compositional generalization in world modeling.
Deep networks learn sparse hierarchical features without CoD.
problem Overparameterized deep networks struggle with the curse of dimensionality.
method Norm-constrained neural networks for sparse compositional functions.
result Deep networks can learn sparse hierarchical features efficiently.
Adaptive sampling method solves constrained and composite optimization problems.
problem Solving constrained optimization problems with stochastic objectives and deterministic constraints.
method Proximal gradient method with adaptive sampling to improve gradient approximation quality.
result Convergence results established for both strongly convex and general convex objectives.
Word embedding is a powerful tool in natural language processing. In this paper we consider the problem of word embedding composition \--- given vector representations of two words, compute a vector for the entire phrase. We give a generative model that can capture specific syntactic relations between words. Under our …
Composite likelihood inference of fractional Gaussian processes with sequentially optimal subset selection
problem Estimating parameters in time series
method Composite likelihood method
result The method reduces computational cost
Deep learning can learn compositional functions more efficiently by breaking them into stages.
problem Understanding why deep learning performs better than shallow models in learning compositional functions.
method Analyzed learnability of compositional target functions using a three-layer fitting model trained with layer-wise spectral estimators.
result Learning compositional functions can be simplified by breaking them into stages, reducing the complexity of the learning problem.
In this paper, we explicitly construct the Calabi composition of multiple affine hyperspheres possibly including some points viewing as 0-dimensional hypersheres. Then we compute all the basic affine invariants of the composed affine hyperspheres, proving that the composed affine hypersphere is symmetric one if and onl…
Develops consistent approximations for composite optimization problems.
problem Significant errors in solutions due to approximations in optimization problems.
method Specifies conditions for well-behaved approximations in minimizers, stationary points, and level-sets for a broad class of composite problems.
result Framework of consistent approximations for composite problems, including stochastic, neural-network, and multi-objective optimization.
Corrects an error in isospectral lens spaces formula for composite q.
problem Isospectral lens spaces on forms but not on functions for composite q.
method Detailed calculations and reworking of formulas for composite q.
result Formulas (3) and (4) in previous work must be revised for composite q.
Transformer architecture struggles with complex tasks due to limitations in function composition.
problem Transformer architecture's limitations in handling complex tasks.
method Used Communication Complexity to prove limitations in composing functions.
result Transformer layer is incapable of handling large domain functions, even when domains are small.
Text classification is a challenging problem which aims to identify the category of texts. In the process of training, word embeddings occupy a large part of parameters. Under the limitation of limited computing resources, it indirectly limits the ability of subsequent network designs. In order to reduce the number of …
New method solves complex optimization problems with reduced sample complexity.
problem Solving nonconvex stochastic nested optimization problems.
method Stochastic ADMM approach to find ε-stationary points.
result Total sample complexity of O(ε^(-3)) for online case and O((2N_1 + N_2) + (2N_1 + N_2)^(1/2)ε^(-2)) for finite sum case.
Adapts Altman's model to compositional data for bankruptcy prediction.
problem Predicting business default using standard financial ratios has issues.
method Uses compositional data methodology with log-ratios and machine learning.
result Compositional methods improve predictive performance, especially random forests.
CoLA automates efficient numerical linear algebra for complex matrix structures.
problem Efficiently solving large-scale linear algebra problems with complex matrix structures.
method Combining linear operator abstraction with compositional dispatch rules.
result Automatic and efficient numerical algorithms for various linear algebra operations.
Bayesian optimization for composite functions improves efficiency.
problem Optimizing composite functions with expensive derivative-free evaluations.
method Exploits composite structure using multi-output Gaussian process and expected improvement for composite functions.
result Significantly improves sampling efficiency and asymptotically converges to global optimum.
Algorithmic fairness, and in particular the fairness of scoring and classification algorithms, has become a topic of increasing social concern and has recently witnessed an explosion of research in theoretical computer science, machine learning, statistics, the social sciences, and law. Much of the literature considers…
Paper proposes iLPA for solving DC composite optimization problems, with applications to matrix completion with outliers.
problem Solving nonconvex and nonsmooth DC composite optimization problems.
method Inexact linearized proximal algorithm (iLPA) for DC composite optimization problems.
result The iLPA achieves local R-linear convergence rate under the Kurdyka-Łöjasiewicz property.
Efficient logistic regression for aggregated data reduces computation time.
problem Inference for logistic regression models is computationally expensive for large datasets.
method Adapted symbolic data analysis to summarise predictor variables into histograms and use composite likelihoods.
result The method achieves comparable classification rates to full data analysis but at lower computational cost.
Learning embeddings of entities and relations is an efficient and versatile method to perform machine learning on relational data such as knowledge graphs. In this work, we propose holographic embeddings (HolE) to learn compositional vector space representations of entire knowledge graphs. The proposed method is relate…
A framework for designing and evaluating new GCN variants.
problem Designing and evaluating new graph convolutional network (GCN) variants.
method Propose a framework to compose networks using building blocks of GCN.
result Several newly composed variants are useful alternatives and competitive with original GCNs.
New approach for open ad hoc teamwork using graph-based policy learning.
problem Designing autonomous agents to collaborate with changing teams without prior coordination.
method Graph-based policy learning to adapt to dynamic team compositions.
result Successfully models the effects of other agents, leading to robust adaptation and superior performance.
New algorithm samples efficiently from complex composite potentials.
problem Sampling from densities with smooth and non-smooth components.
method Metropolis-Hastings framework with proximal-based proposal.
result Mixes to target density in O(dlog(d/ε)) iterations. Floer cohomology is computed for certain elements of the mapping class group of a surface Σ of genus g>1 which are compositions of positive and negative dehn twists along some loops in Σ. The computations cover a certain class of pseudo-Anasov maps.
A novel optimization-based Gaussian mixture reduction method using composite transportation divergence.
problem Exponential increase in Gaussian mixture order leads to intractable inference.
method Optimization-based Gaussian mixture reduction (GMR) using composite transportation divergence (CTD).
result Unified framework for selecting optimal cost function in various applications.
We consider in this paper a class of composite optimization problems whose objective function is given by the summation of a general smooth and nonsmooth component, together with a relatively simple nonsmooth term. We present a new class of first-order methods, namely the gradient sliding algorithms, which can skip the…
A new GP model for non-Gaussian data with explicit inverse warping.
problem Limited expressiveness and computational complexity of Gaussian processes for non-Gaussian data.
method Compositionally-warped Gaussian processes (CWGP) with explicit inverse warping.
result CWGP provides more accurate predictions and shorter computation times than traditional warped GPs.
In earlier work the Kauffman bracket polynomial was extended to an invariant of marked graphs, i.e., looped graphs whose vertices have been partitioned into two classes (marked and not marked). The marked-graph bracket polynomial is readily modified to handle graphs with weighted vertices. We present formulas that simp…
A new method estimates rare events using tensor trains.
problem Estimating rare event probabilities in high-dimensional problems.
method Approximating optimal importance distribution via tensor-train decompositions and compositions.
result Better variance reduction and efficient computation of rare event probabilities.
We investigate the Khovanov-Rozansky invariant of a certain tangle and its compositions. Surprisingly the complexes we encounter reduce to ones that are very simple. Furthermore, we discuss a "local" algorithm for computing Khovanov-Rozansky homology and compare our results with those for the "foam" version of sl_3-hom…
Improves music composition with user-defined constraints using continuous time models.
problem Combining sequence models with user-defined constraints in continuous time.
method Introduces a novel particle filter scheme for continuous time point processes.
result The particle filter scheme yields superior results in a human listening test.