Efficient algorithm for analyzing compositional data.
problem Compositional data analysis with nonnegative values summing to one.
method Proposes an efficient solution path algorithm for l 1 l_1 l 1 regularized regression with compositional data. result The proposed algorithm is faster than existing methods, especially in high-dimensional cases.
Paper analyzes stability and generalization of SCO algorithms.
problem Understanding how SCO algorithms perform on unseen data.
method Algorithmic stability analysis in statistical learning theory.
result Derives dimension-independent excess risk bounds for SCGD and SCSC.
Paper develops momentum schemes with variance reduction for non-convex composition optimization.
problem Lack of convergence guarantee and efficient momentum design in existing algorithms.
method Develops various momentum schemes with SPIDER-based variance reduction.
result Achieves near-optimal sample complexity and linear convergence rate.
New algorithm reduces complexity for optimizing complex machine learning tasks.
problem Optimizing complex machine learning objectives like reinforcement learning and portfolio management.
method Developed SARAH-Compositional algorithm using Stochastic Recursive Gradient Descent.
result Achieved optimal IFO complexity bounds for stochastic compositional optimization.
Routing networks tackle challenges in modular and compositional computation.
problem Challenges in learning and training compositional models with module parameters and their composition.
method Routing networks as a general approach to address these challenges, examining the interplay of algorithmic decisions.
result Empirical analysis of routing networks reveals the interplay of challenges and design decisions.
This work studies fairness in systems of multiple algorithms, addressing pitfalls and constructing fair compositions.
problem Fairness of scoring and classification algorithms in systems of multiple algorithms.
method Identifying and addressing pitfalls of naive composition, constructing fair compositions for individual and group fairness.
result Fairness properties of systems of multiple fair algorithms are not necessarily preserved under composition.
This paper improves neural network explanations by quantifying and visualizing semantic compositions.
problem Improving neural network explanations for natural language processing tasks.
method Proposes a formal way to quantify word and phrase importance, introduces SCD and SOC algorithms.
result Our algorithms outperform prior methods in explaining neural network predictions.
New filters match advanced composition for adaptive privacy, with practical constants.
problem Limitations of existing adaptive composition methods.
method Constructed new filters and odometers that match advanced composition rates, including constants.
result Achieved fully adaptive privacy with practical filters and odometers.
Unified algorithm for minimizing composite functions with flexible design.
problem Minimizing composite functions with specific structural properties.
method Unified accelerated algorithm for complementary composite minimization.
result Near-optimal algorithms for various optimization problems.
Analysis of a stochastic system showing convergence to an averaged model with Gaussian deviations.
problem Convergence analysis of a perturbed compositional gradient flow system.
method Separation of scales and averaging principle applied to stochastic differential equations.
result The slow motion of the system can be approximated by a standard perturbed gradient flow or SCGD algorithm.
Algorithm samples composite logconcave densities efficiently.
problem Sampling from composite logconcave densities efficiently.
method Uses a restricted Gaussian oracle and gradient queries.
result Achieves strong total variation distance guarantees.
System composes polyphonic music using LSTM and RL.
problem Complex polyphonic music composition.
method Divided music into monophonic streams, trained LSTM to generate sequences, used RL to find pleasant compositions.
result System generates intricate melodies, chords, and contrapuntal sequences.
New algorithm framework solves stochastic composite nonconvex optimization problems efficiently.
problem Solving stochastic composite nonconvex optimization problems.
method ProxSARAH framework using SARAH estimator with proximal gradient and averaging steps.
result Achieves best-known complexity bounds with constant and adaptive step-sizes.
A new model for analyzing compositional data using folded structure.
problem Analyzing data defined on the simplex sample space.
method Developed a folded type model involving an extension of the α-transformation for efficient parameter estimation using the EM algorithm.
result The proposed model outperforms the logistic normal distribution in capturing data structure.
In many applications one may acquire a composition of several signals that may be corrupted by noise, and it is a challenging problem to reliably separate the components from one another without sacrificing significant details. Adding to the challenge, in a compressive sensing framework, one is given only an undersampl…
Unified view of gradient-based algorithms for stochastic convex composite optimization.
problem Optimization of stochastic convex composite functions.
method Extend the concept of estimate sequence to cover various gradient-based methods.
result Generic convergence proof and new adaptive SVRG variant.
FeDualEx tackles saddle point optimization in federated learning with composite objectives.
problem Saddle point optimization with constraints and non-smooth regularization in federated learning.
method Federated Dual Extrapolation (FeDualEx) algorithm for saddle point optimization and composite objectives.
result FeDualEx effectively solves saddle point optimization problems with composite objectives in federated learning.
Study on deep neural networks using branching processes and Mehler's formula.
problem Understanding the mathematical role of activation functions in compositional neural networks.
method Connection between compositional kernels and branching processes via Mehler's formula; new random features algorithm.
result Explicit formulas for eigenvalues of compositional kernels quantify complexity.
A new ADMM-based algorithm for stochastic composition optimization.
problem Stochastic composition optimization problems in estimation and machine learning.
method com-SVR-ADMM, converges linearly for strongly convex and Lipschitz smooth objectives, and has improved convergence rates.
result com-SVR-ADMM converges linearly for strongly convex and Lipschitz smooth objectives with a rate of O ( log S / S ) O( \log S/S) O ( log S / S ) . This paper advances FL algorithms for composite optimization and statistical recovery.
problem Federated learning optimization and statistical recovery in composite settings.
method Proposes Fast Federated Dual Averaging for strongly convex and smooth loss, and Multi-stage Federated Dual Averaging for restricted strongly convex and smooth loss.
result Establishes state-of-the-art iteration and communication complexity, and high probability complexity bound with linear speedup.
Accelerates machine learning algorithms for sparse data.
problem Efficiently solving composite convex minimization problems.
method Accelerated dual-averaging primal-dual method for composite convex minimization.
result Demonstrates advantages in handling sparse data both theoretically and empirically.
New algorithm solves composite optimization problems with unknown expectations.
problem Solving composite optimization problems with unknown statistical expectations.
method Proposes a new stochastic primal-dual algorithm for composite optimization problems with unknown statistical expectations.
result Converges to a saddle point of the Lagrangian function.
Adaptive MAB algorithms handle composite, anonymous feedback without reward interval knowledge.
problem Multi-armed bandit with composite and anonymous feedback, especially without reward interval size knowledge.
method Proposed adaptive algorithms for stochastic and adversarial cases, without reward interval knowledge.
result First algorithm for adversarial case handling non-oblivious adversary and unknown reward interval size.
We consider in this paper a class of composite optimization problems whose objective function is given by the summation of a general smooth and nonsmooth component, together with a relatively simple nonsmooth term. We present a new class of first-order methods, namely the gradient sliding algorithms, which can skip the…
Optimizes convergence rate of stochastic proximal algorithms for composite convex problems.
problem Solving composite convex optimization problems with composite regularizers.
method Analyzed proximal stochastic gradient method and randomized incremental proximal method under relaxed variance assumptions.
result Proves O ( 1 / T ) O(1/\sqrt{T}) O ( 1/ T ) convergence rate for last iterate of both algorithms under componentwise convexity and smoothness. Paper extends FFT-based differential privacy method to heterogeneous compositions.
problem Computing accurate differential privacy guarantees for mixed mechanisms.
method Uses Fast Fourier Transform (FFT) for error analysis and parameter selection.
result Provides tighter bounds for heterogeneous compositions compared to homogeneous cases.
New method solves doubly-nonconvex composite optimization problems.
problem Solving composite optimization problems with both functions nonconvex.
method Stochastic gradient descent with quasiconvex penalty function.
result Convergence properties for doubly-nonconvex composite optimization.
Heegaard Floer homology study confirms composition maps match up to homotopy.
problem Verifying consistency of composition maps in Heegaard Floer homology.
method Using Auroux and Zemke's results, proving agreement up to homotopy.
result Proves consistency of composition maps in Heegaard Floer homology.
Developed neural network for predicting mechanical properties of composite materials.
problem Predicting and optimizing mechanical properties of composite materials.
method Convolutional neural network model integrated with a genetic algorithm optimizer.
result Highly accurate predictions and optimal microstructural designs identified.
Paper solves robust convex problems with heavy-tailed noise.
problem Solving convex compositional problems with heavy-tailed noise.
method Sub-Gaussian confidence bounds under weak heavy-tailed noise assumptions, using boosting strategy.
result Achieves nearly optimal high probability convergence result.
New algorithm solves complex optimization problems without needing projections.
problem Optimizing nested functions under convex constraints with noisy evaluations.
method Projection-free conditional gradient-type algorithm for smooth stochastic multi-level composition optimization.
result The algorithm achieves ε ε ε -stationary solutions with complexity bounds independent of ε ε ε and T T T . New algorithm reduces error in regression problems.
problem Minimizing composite objective functions with quadratic and convex components.
method Stochastic dual averaging with constant step-size, proving convergence rate O(1/n).
result Extends least-squares regression to various convex regularizers and geometries.
Sharp privacy bounds for sequential analysis of sensitive data.
problem Privacy degradation under sequential analysis of sensitive data.
method Edgeworth expansion in f-differential privacy framework.
result Improved privacy bounds under composition with refined approximation accuracy.
New MCMC method for generating composition ratios.
problem Combining and selecting knowledge to create new compositions.
method Proposes a new MCMC algorithm with specific constraints.
result Shows the effectiveness of combining MCMC with supervised learning.
New method speeds up optimization for complex functions.
problem Optimizing complex functions with stochastic composition.
method Accelerated stochastic compositional proximal gradient (ASC-PG) method.
result ASC-PG achieves faster convergence and optimal sample-error complexity.
Edgeworth Accountant calculates privacy loss under differential privacy compositions efficiently.
problem Efficiently computing overall privacy loss under composition of private algorithms.
method Analytical approach using f f f -differential privacy framework and Edgeworth expansion. result Non-asymptotic ( ε , δ ) (ε, δ) ( ε , δ ) -differential privacy bounds with reduced computational cost. New algorithm samples efficiently from complex composite potentials.
problem Sampling from densities with smooth and non-smooth components.
method Metropolis-Hastings framework with proximal-based proposal.
result Mixes to target density in O ( d log ( d / ε ) ) O(d \log (d/\varepsilon)) O ( d log ( d / ε )) iterations. New sparse GP model learns compositional kernels efficiently.
problem Learning accurate Gaussian Process models with complex kernel structures.
method MultiSVGP model with Horseshoe prior for kernel selection.
result Our model provides better fit and faster computation for large-scale data.
Paper tackles distributed linear regression with compositional covariates.
problem Solving distributed statistical methodology and computing for massive compositional data.
method Proposes two distributed optimization techniques based on ADMM and CDMM for solving constrained convex optimization problems.
result Established convergence theories for the proposed algorithms under regularity conditions.
Paper introduces a new learner for generalizing complex tasks.
problem Generalizing to new, complex tasks without prior experience.
method Compositional problem graph and compositional recursive learner.
result Compositional approach can generalize to more complex problems than non-compositional learners.
Paper proposes iLPA for solving DC composite optimization problems, with applications to matrix completion with outliers.
problem Solving nonconvex and nonsmooth DC composite optimization problems.
method Inexact linearized proximal algorithm (iLPA) for DC composite optimization problems.
result The iLPA achieves local R-linear convergence rate under the Kurdyka-Łöjasiewicz property.
New adaptive smoothing algorithm solves nonsmooth convex optimization.
problem Solving fully nonsmooth composite convex optimization problems.
method Combines Nesterov's accelerated proximal gradient scheme with a homotopy strategy for smoothness parameter.
result Develops an algorithm with worst-case iteration-complexity of O(1/ε) while maintaining complexity-per-iteration.
This paper characterizes exp-concavity of proper composite losses and transforms mixable losses into exp-concave ones.
problem Understanding and transforming mixable losses into exp-concave ones for better online prediction strategies.
method Characterization of exp-concavity, mixability condition, and approximation approach for multi-class losses.
result Complete characterization of exp-concavity for proper composite losses and transformation of mixable losses into exp-concave ones.
New algorithms detect clusters in multiplex networks.
problem Detecting clusters in networks with multiple layers.
method Network Fusion for Composite Community Extraction (NF-CCE) using non-negative matrix factorization.
result NF-CCE outperforms state-of-the-art methods on various multiplex networks.
New algorithms solve nonconvex federated learning problems efficiently.
problem Nonconvex federated composite optimization in federated learning.
method FedDR and asyncFedDR algorithms combining Douglas-Rachford splitting, randomized block-coordinate strategies, and asynchronous implementation.
result Match communication complexity lower bound up to a constant factor.
A new neural network for text classification reduces parameters with improved accuracy.
problem Reducing the number of parameters in text classification models.
method Compositional coding, capsule network, k-means routing algorithm.
result The proposed method achieves competitive accuracy with significantly fewer parameters.
Paper simplifies DP composition for adaptive privacy budgets, enabling better privacy and accuracy in deep learning.
problem Tension between efficiency and flexibility in DP composition theorems.
method Rényi Differential Privacy (RDP) for adaptive privacy budgets, proving simpler composition theorem with smaller constants.
result Practical DP composition for adaptive privacy budgets, enabling better privacy and accuracy in deep learning.
GFlowNet-EM learns complex latent variable models with discrete structures.
problem Challenges in modeling posteriors over discrete compositional latents with expectation-maximization.
method Uses GFlowNets to learn stochastic policies for sampling from complex posterior distributions.
result GFlowNet-EM enables training expressive LVMs with discrete compositional latents.