Paper analyzes word embedding composition using tensor decomposition.
problem Given vector representations of two words, compute a vector for the entire phrase.
method Generative model with low rank Tucker decomposition of word embedding correlations.
result Word embeddings and a core tensor can be derived from the Tucker decomposition.
This work proves composite neural networks outperform their components under certain conditions.
problem Understanding the performance of composite neural networks.
method Theoretical investigation of a composite neural network combining pre-trained models.
result A composite neural network, with high probability, performs better than any of its pre-trained components.
New algorithm framework solves stochastic composite nonconvex optimization problems efficiently.
problem Solving stochastic composite nonconvex optimization problems.
method ProxSARAH framework using SARAH estimator with proximal gradient and averaging steps.
result Achieves best-known complexity bounds with constant and adaptive step-sizes.
Solves tangle equations involving composite links under specific conditions.
problem Solving tangle equations with composite links and 2-bridge links.
method Uses algebraic and double branched cover properties to solve equations.
result Non-hyperbolic solutions to tangle equations involving composite links are found.
Paper solves robust convex problems with heavy-tailed noise.
problem Solving convex compositional problems with heavy-tailed noise.
method Sub-Gaussian confidence bounds under weak heavy-tailed noise assumptions, using boosting strategy.
result Achieves nearly optimal high probability convergence result.
We develop a model to predict effects of sequential interventions, clarifying their combined impact.
problem Uncertainty in generalizing behavioral predictions for combinations of interventions.
method Explicit model for composition of interventions, identifying their combined effect.
result Our compositional model aids prediction in sparse data conditions.
Deep neural networks with ReLU activation achieve optimal nonparametric regression rates.
problem Nonparametric regression with general composition assumptions.
method Sparsely connected deep neural networks with ReLU activation function.
result Achieve minimax rates of convergence under general composition assumption.
New filters match advanced composition for adaptive privacy, with practical constants.
problem Limitations of existing adaptive composition methods.
method Constructed new filters and odometers that match advanced composition rates, including constants.
result Achieved fully adaptive privacy with practical filters and odometers.
DGPs collapse to near-deterministic transformations, limiting their compositional structure discovery.
problem Limitations of variational inference in DGPs lead to suboptimal posterior approximations.
method Examine alternative variational inference schemes allowing for dependencies across different layers.
result Alternative variational inference schemes can better capture the compositional structure in DGPs.
CARE method estimates precision matrix for compositional data, achieving optimality in high dimensions.
problem Challenges in inferring conditional dependence relationships in high-dimensional compositional data.
method Composition adaptive regularized estimation (CARE) method for sparse basis precision matrix.
result CARE estimator achieves minimax optimality in high dimensions, performing as well as if the basis were observed.
This paper studies robust estimation methods in high dimensions, comparing model-averaged and composite quantile estimators.
problem Understanding robustness in high-dimensional regularized estimation.
method Optimal weights are determined by minimizing the asymptotic mean squared error, incorporating regularization effects without perfect selection.
result Model-averaged and composite quantile estimators often outperform least-squares methods in prediction quality.
New method estimates composite outcomes for precision medicine.
problem Balancing multiple outcomes in precision medicine.
method Optimal treatment rule for composite outcomes.
result Estimates composite outcomes and optimal treatment rule.
New findings show disentangled latent representations are not enough for robust compositional generalization.
problem Deep learning models struggle with compositional generalization, especially in out-of-distribution samples.
method Investigated a 2D Gaussian generation task with fully disentangled inputs, then forced disentangled latent representations into full-dimensional output space.
result Forcing disentangled latent representations into full-dimensional output space enables robust compositional generalization.
Optimizes convergence rate of stochastic proximal algorithms for composite convex problems.
problem Solving composite convex optimization problems with composite regularizers.
method Analyzed proximal stochastic gradient method and randomized incremental proximal method under relaxed variance assumptions.
result Proves O ( 1 / T ) O(1/\sqrt{T}) O ( 1/ T ) convergence rate for last iterate of both algorithms under componentwise convexity and smoothness. Developed neural network for predicting mechanical properties of composite materials.
problem Predicting and optimizing mechanical properties of composite materials.
method Convolutional neural network model integrated with a genetic algorithm optimizer.
result Highly accurate predictions and optimal microstructural designs identified.
We consider the minimization of composite objective functions composed of the expectation of quadratic functions and an arbitrary convex function. We study the stochastic dual averaging algorithm with a constant step-size, showing that it leads to a convergence rate of O(1/n) without strong convexity assumptions. This …
This research proves guarantees on sequence models' generalization to longer and novel sequences.
problem Generalization to longer sequences and novel token combinations in sequence models.
method Provable guarantees on length and compositional generalization for various sequence models.
result Limited capacity models achieve both length and compositional generalization with diverse training distributions.
New DP-CD method outperforms DP-SGD in solving composite DP-ERM problems.
problem Privacy-preserving machine learning with differential privacy.
method Differentially Private proximal Coordinate Descent (DP-CD) for composite Empirical Risk Minimization (ERM).
result DP-CD outperforms DP-SGD due to larger step sizes and better gradient exploitation.
New algorithm solves complex optimization problems without needing projections.
problem Optimizing nested functions under convex constraints with noisy evaluations.
method Projection-free conditional gradient-type algorithm for smooth stochastic multi-level composition optimization.
result The algorithm achieves ε ε ε -stationary solutions with complexity bounds independent of ε ε ε and T T T . We study pairs of structures, such as the Poisson-Nijenhuis structures, on the tangent bundle of a manifold or, more generally, on a Lie algebroid or a Courant algebroid. These composite structures are defined by two of the following, a closed 2-form, a Poisson bivector or a Nijenhuis tensor, with suitable compatibilit…
New algorithms solve nonconvex federated learning problems efficiently.
problem Nonconvex federated composite optimization in federated learning.
method FedDR and asyncFedDR algorithms combining Douglas-Rachford splitting, randomized block-coordinate strategies, and asynchronous implementation.
result Match communication complexity lower bound up to a constant factor.
New algorithms solve non-convex optimization problems efficiently.
problem Non-convex stochastic compositional optimization problems.
method Developed two stochastic Gauss-Newton algorithms.
result Established global oracle complexity for stochastic Gauss-Newton methods.
We propose a unified modelling framework that theoretically justifies the main empirical regularities characterizing the international trade network. Each country is associated to a Polya urn whose composition controls the propensity of the country to trade with other countries. The urn composition is updated through t…
The covariance graph (aka bi-directed graph) of a probability distribution p p p is the undirected graph G G G where two nodes are adjacent iff their corresponding random variables are marginally dependent in p p p . In this paper, we present a graphical criterion for reading dependencies from G G G , under the assumption that $…
Develops methods for inference after detecting a change in sequential data.
problem Inference after a detected change in sequential data.
method General framework for constructing confidence sets using only data up to a stopping time.
result First general method for sequential changepoint localization with theoretical guarantees.
Paper analyzes adaptive optimization algorithms and provides new insights.
problem Adaptive optimization algorithms for non-convex and composite objectives.
method New regret decomposition, Bregman divergences, modular analysis.
result Improved variational bounds and new optimistic MD algorithms.
Study develops efficient algorithm for probabilistic penetration response of composite plates.
problem Probabilistic modeling of discrete structural response, focusing on binary events like buckling.
method Adaptive domain-based decomposition, sparse grid sampling, assumption of monotonic behavior.
result Efficient computational framework for probabilistic penetration response of composite plates.
Under Markovian assumptions, we leverage a Central Limit Theorem (CLT) for the empirical measure in the test statistic of the composite hypothesis Hoeffding test so as to establish weak convergence results for the test statistic, and, thereby, derive a new estimator for the threshold needed by the test. We first show t…
New method shows hidden state can significantly improve differential privacy in SGD.
problem Differential privacy in SGD with hidden state.
method Proves converging privacy bounds for hidden state SGD, using privacy amplification techniques.
result Privacy bound converges exponentially fast and is smaller than composition bounds.
We prove that, in general, given a p p p -harmonic map F : M → N F:M\to N F : M → N and a convex function H : N → R H:N\to\mathbb{R} H : N → R , the composition H ∘ F H\circ F H ∘ F is not p p p -subharmonic. By assuming some rotational symmetry on manifolds and functions, we reduce the problem to an ordinary differential inequality. The key of the proof is an asymptotic…
New hybrid stochastic optimization framework tackles nonconvex problems efficiently.
problem Efficiently solving stochastic composite nonconvex optimization problems.
method Combining two stochastic estimators to create a hybrid one, developing several variants of stochastic gradient methods.
result Achieved best-known complexity bounds for various optimization problems.
A new hybrid algorithm reduces stochastic gradient evaluations for nonconvex optimization.
problem Solving stochastic composite nonconvex optimization problems efficiently.
method Proposes a new hybrid variance-reduced proximal gradient method with a stochastic gradient estimator.
result Achieves optimal stochastic oracle complexity bound with one less gradient evaluation.
This work addresses the convergence of SGD's final iterate without restrictive assumptions.
problem Prove optimal convergence rate of SGD's final iterate without compact domains or bounded noise.
method Unified proof for general domains, composite objectives, non-Euclidean norms, etc.
result First unified convergence rates in expectation and high probability.
Two algorithms find optimal points in decentralized optimization.
problem Decentralized non-convex stochastic optimization with composite objective functions.
method Prox-DASA and Prox-DASA-GT algorithms for finding ε-stationary points.
result Achieves comparable complexity without large batch sizes or complex per-iteration operations.
Study non-oblivious adversarial bandits with delayed feedback and propose algorithms with improved regret bounds.
problem Adversarial bandit problem with delayed, composite anonymous feedback.
method Propose wrapper algorithm for non-oblivious delay setting, achieving o ( T ) o(T) o ( T ) policy regret. result Achieve o ( T ) o(T) o ( T ) policy regret for many adversarial bandit problems with bounded memory loss sequences. Deep neural networks achieve optimal classification rates in high dimensions.
problem Binary classification on high-dimensional data with specific smoothness and composition properties.
method Proved optimal convergence rate for ReLU DNNs trained with hinge loss.
result ReLU DNNs achieve optimal classification rates up to a logarithmic factor.
This work is an analytical and numerical study of the composition of several fractals into one and of the relation between the composite dimension and the dimensions of the component fractals. In the case of composition of standard IFS with segments of equal size, the composite dimension can be expressed as a function …
Generative Neuro-Symbolic model learns from raw data with rich conceptual representations.
problem Learning rich, general-purpose conceptual representations from raw perceptual inputs.
method Generative Neuro-Symbolic (GNS) model combining symbolic and neural network approaches.
result Model learns from raw data and generalizes to 4 unique tasks.
New geometric approach for analyzing compositional data like gut microbiomes.
problem Analyzing non-negative compositional data with relative values only.
method Reinterpret compositional data as quotient topology of a sphere, using spherical harmonics and reflection group actions.
result Construction of Reproducing Kernel Hilbert Space (RKHS) for compositional data.
We show that the problem of recognizing that a knot diagram represents a specific torus knot, or any torus knot at all, is in the complexity class N P ∩ c o - N P {\sf NP} \cap {\sf co\text{-}NP} NP ∩ co - NP , assuming the generalized Riemann hypothesis. We also show that satellite knot detection is in N P {\sf NP} NP under the same assumption, and t…
Study on deep neural networks using branching processes and Mehler's formula.
problem Understanding the mathematical role of activation functions in compositional neural networks.
method Connection between compositional kernels and branching processes via Mehler's formula; new random features algorithm.
result Explicit formulas for eigenvalues of compositional kernels quantify complexity.
Uses art composition attributes to guide CycleGAN image translation.
problem Improving image-to-image translation quality.
method Trained ACAN on art composition attributes to influence CycleGAN.
result CycleGAN translations improved with ACAN constraints.
We establish conditions for compositional generalization in machine learning.
problem Achieving compositional generalization in machine learning models.
method We reformulate compositionality as a property of the data-generating process and derive mild conditions on the training distribution and model architecture.
result Our theoretical framework enables compositional generalization under mild conditions.
This paper proves hyperbolicity of virtual knot compositions.
problem Proving hyperbolicity of virtual knot compositions.
method Exploring the composition of hyperbolic virtual knots.
result Strong lower bounds on the volume of compositions.
Develops methods for causal inference in compositional data using instrumental variables.
problem Interpreting summary statistics like diversity indices as causal effects in compositional data.
method Statistical data transformations and regression techniques tailored for compositional data.
result Advantages and limitations of the proposed methods demonstrated on synthetic and real microbiome data.
UCB-TQL learns from multiple tasks with shared dynamics and adapts to task-specific variations.
problem Transfer reinforcement learning with composite MDPs where tasks share core dynamics but have sparse differences.
method UCB-TQL, a novel transfer RL algorithm for composite MDPs.
result Achieved a regret bound of i l d e O ( e H 5 N ) ilde{O}(\sqrt{eH^5N}) i l d e O ( e H 5 N ) that scales independently of the ambient dimension. In classical field theory, the composite fibred manifolds Y -> Z -> X provides the adequate mathematical formulation of gauge models with broken symmetries, e.g., the gauge gravitation theory. This work is devoted to connections on composite fibred manifolds. In particular, we get the horizontal splitting of the vertic…
The p-index improves investment performance for NYSE stocks but not for SSE stocks.
problem Improving investment performance for stocks using the p-index.
method Comparing different p-ratio strategies and empirical efficient frontiers for SSE and NYSE stocks.
result The p-index enhances investment performance for NYSE stocks but not for SSE stocks.