New approach to machine learning optimization using distribution space.
problem Optimization challenges in machine learning with non-convex constraints.
method Relaxation to convex optimization in distribution space, numerical algorithm based on mixture distributions.
result Approximate optimization in distribution space is consistent and effective.
Study optimal ridge regularization for out-of-distribution prediction.
problem Optimal ridge regularization for predicting out-of-distribution data.
method Established conditions for optimal regularization under covariate and regression shifts, proving monotonic risk in data aspect ratio.
result Negative regularization can be optimal under shifts, even with isotropic or underparameterized training features.
A new method uses optimal transport to transform latent space distributions without solving a hard Min-Max problem.
problem Latent space data distribution collapse and loss of manifold structure in generative models.
method Proposes a GAN-like method to solve a minimization problem for optimal transport between a simple distribution and a latent-space data distribution.
result Experimental results show that the proposed method can handle multi-cluster distributions and is effective on MNIST and CelebA datasets.
Study optimizes privacy in distributed optimization, balancing accuracy and communication.
problem Privacy-preserving distributed stochastic convex optimization.
method Distributed algorithm using Vaidya's plane cutting method, with privacy guarantees via differential privacy.
result Complete characterization of accuracy-communication-privacy trade-off.
Study learns optimal auctions from corrupted or perturbed bidder valuation samples.
problem Learning revenue-optimal auctions from corrupted or perturbed samples.
method Proves upper bounds, proposes algorithms for learning near-optimal auctions.
result Proves tight upper bounds and proposes algorithms for near-optimal auctions.
Paper optimizes training data distribution for better model performance across various deployment conditions.
problem Improving model accuracy when deployed with parameters far from training data.
method Developed adaptive algorithms based on bilevel or alternating optimization in the space of probability measures.
result Optimized training distributions lead to models with improved sample complexity and robustness to distribution shift.
New approach reduces communication in distributed optimization.
problem Reduces communication in distributed optimization.
method Each node performs multiple local optimization steps before communication, even for an infinite number of steps.
result Communication can be significantly reduced without sacrificing convergence.
Paper characterizes sampling distributions of optimal portfolio weights and characteristics.
problem Characterizing sampling distributions of optimal portfolio weights and characteristics.
method Derives exact sampling distribution by stochastic representation.
result High-dimensional asymptotic distribution of optimal portfolio weights is multivariate normal.
A novel Federated Learning scheme using Optimal Transport for personalized model training.
problem Training models with data from clients having non-identically distributed data.
method Personalized Federated Learning scheme based on Optimal Transport (FedOT).
result FedOT scheme effectively transfers data from multiple distributions to a common domain and optimizes the prediction model.
Proposes MRO to achieve uniformly low regret in distributionally robust learning.
problem Learning under unknown test distributions (distribution shift).
method Minimax Regret Optimization (MRO) for robust machine learning.
result MRO achieves uniformly low regret across all test distributions.
Spark implementation for distributed function minimization.
problem Optimization of functions in distributed computing environments.
method Gradient and quasi-Newton methods on Apache Spark.
result Scalable solution for classification and regression problems.
The paper studies robust risk measures with linear penalties under uncertain distributions.
problem Risk measurement under distributional uncertainty.
method Robust distortion risk measures with linear penalty function under distributional constraints.
result Explicit characterization of optimal quantile distribution and value function.
New method for fully distributed Bayesian optimization with high parallelism.
problem Scalability and parallelization in Bayesian optimization.
method Formulated Bayesian optimization as a partially observable Markov decision process and applied stochastic policies.
result Demonstrated superior performance of the proposed method in various benchmarks and applications.
Private KL distribution estimation improved with instance-optimality.
problem Minimizing KL divergence between true and estimated distributions.
method Construct minimax optimal private estimators, then focus on instance-optimality.
result Achieved instance-optimality up to constant factors for KL estimation.
LOT embeds distributions for linear separability and classification.
problem Distribution discrimination in various scientific fields.
method Linear Optimal Transport (LOT) embedding into L 2 L^2 L 2 space. result LOT embeds distributions into linearly separable spaces for certain transformations and perturbations.
Distributed algorithm finds global solutions for low-rank matrices.
problem Finding global solutions for low-rank matrices in distributed systems.
method Distributed Gradient Descent (DGD+) with LOCAL variables.
result DGD+LOCAL converges to global minimizer with exact consensus.
Method learns conditional distributions using neural entropic optimal transport.
problem Challenges in learning multiple conditional distributions.
method Neural entropic optimal transport method with two networks and regularization.
result Effective learning of conditional distributions with limited samples.
FedAVOT improves federated learning by aligning user distributions.
problem Partial client participation leads to biased and unstable updates in federated learning.
method Formulates aggregation as masked optimal transport to align availability and importance distributions.
result Achieves a standard O(1/√T) rate, independent of the number of participating users per round.
Study binary hypothesis testing with privacy and communication constraints.
problem Binary hypothesis testing under local differential privacy and communication constraints.
method Qualifies results as minimax or instance optimal, develops instance-optimal algorithms.
result Achieves minimum possible sample complexity under both privacy and communication constraints.
New approach for distributed online optimization of non-convex losses with sublinear regret.
problem Regret evaluation and consensus in distributed, multi-agent systems with non-convex losses.
method Composite regret metric and consensus-based online normalized gradient (CONGD) approach for pseudo-convex losses; offline optimization oracle for general non-convex losses.
result First sublinear regret bound for general distributed online non-convex learning.
AOT aligns LLMs on distributional preferences via optimal transport.
problem Current LLM alignment techniques lack distributional level alignment.
method Alignment via Optimal Transport (AOT) aligns LLMs on unpaired preference data.
result AOT enables alignment by penalizing reward distribution violations.
New method AM learns optimal vector fields for entire distribution sequences, matching OT.
problem Optimal Transport (OT) problem in generative modeling.
method Action Matching (AM) method learns optimal vector fields for a sequence of distributions.
result AM method achieves optimal transport by learning vector fields for entire distribution sequences.
Study on distributed nonparametric function estimation with optimal rate and cost of adaptation.
problem Optimal rate of convergence and cost of adaptation in distributed nonparametric function estimation.
method Distributed minimax estimation and adaptive estimation under communication constraints for Gaussian sequence model and white noise model.
result Established minimax rate of convergence and exact communication cost for adaptation.
The paper optimizes k-NN for distributed learning with minimax optimal performance.
problem Minimizing error rates in classification, regression, and density estimation.
method Optimal aggregation of fixed-k nearest neighbors from multiple subsets of data.
result Achieves minimax optimal error rates up to a logarithmic factor.
This paper tackles cost-sensitive portfolio optimization under ambiguous return distributions.
problem Tackles cost-sensitive distributionally robust log-optimal portfolio problem with ambiguous return distributions.
method Uses Wasserstein metric for distributional ambiguity, incorporates convex transaction costs, and approximates infinite-dimensional problem with finite convex program.
result Establishes conditions for robustly survivable trades and validates theoretical framework with empirical studies.
Study efficient iterative method for distribution matching using sliced optimal transport.
problem Efficiently match distributions using sliced optimal transport.
method Slice-matching scheme based on sliced optimal transport, with quantitative non-asymptotic rates derived.
result Derive quantitative non-asymptotic rates for convergence to target distribution.
Enhances flexibility in data reweighting with optimal transport and maximum entropy principles.
problem Adapting empirical distributions to predefined constraints on moments, tail behavior, etc.
method Nonparametric distributional constraints, maximum entropy principle, optimal transport.
result Maximum entropy weight adjusted empirical distribution close to a specified distribution in optimal transport metric.
New RL algorithm minimizes distributional learning error.
problem Improving distributional reinforcement learning for better error minimization.
method Proposes a new model-based algorithm with theoretical minimax optimality.
result Proves minimax optimality for approximating return distributions.
Paper proposes a method to solve log-optimal portfolios under ambiguous return distributions.
problem Maximizing wealth growth with unknown return distributions.
method Supporting hyperplane approximation to reformulate the problem into a linear program.
result The problem can be solved efficiently, even with transaction costs and diversification.
This paper develops a scalable Thompson Sampling method using optimal transport.
problem Efficiently approximating posterior distributions for complex models in Thompson Sampling.
method The approach uses distribution optimization techniques via Wasserstein gradient flows to approximate posterior distributions efficiently.
result The proposed method achieves superior performance on both synthetic and real large-scale data.
Generates samples conditioned on labels using optimal transport.
problem Estimating conditional distributions for specific labels.
method Wasserstein geodesic generator based on optimal transport theory.
result Learned conditional distributions and optimal transport maps.
We consider K K K -armed stochastic bandits and consider cumulative regret bounds up to time T T T . We are interested in strategies achieving simultaneously a distribution-free regret bound of optimal order K T \sqrt{KT} K T and a distribution-dependent regret that is asymptotically optimal, that is, matching the κ ln T κ\ln T κ ln T lower b…
Integrates estimation and optimization for uncertain parameters.
problem Optimizing with uncertain parameters whose distributions can be estimated.
method Integrated Conditional Estimation-Optimization (ICEO) framework.
result Asymptotically consistent and provides finite performance guarantees.
Develops unbiased averaging methods for second order optimization in distributed systems.
problem Computing the Hessian is challenging and communication is a bottleneck in distributed optimization.
method Unbiased parameter averaging methods using sampling and sketching of the Hessian.
result Provably minimizes bias for sketched Newton directions.
Near-optimal regret in distributed bandit learning with efficient communication protocols.
problem Minimizing total regret in collaborative bandit learning with limited communication.
method Proposed communication protocols for distributed multi-armed and linear bandits with near-optimal regret and efficient communication costs.
result Achieved near-optimal regret with communication costs independent of time horizon and number of arms.
As the size of modern data sets exceeds the disk and memory capacities of a single computer, machine learning practitioners have resorted to parallel and distributed computing. Given that optimization is one of the pillars of machine learning and predictive modeling, distributed optimization methods have recently garne…
Distributed Lion optimizes large model training by reducing communication costs.
problem Training large AI models efficiently with reduced communication costs.
method Adapted Lion optimizer for distributed training, using binary or lower-precision vectors for communication.
result Distributed Lion achieves comparable performance to standard optimizers but with significantly reduced communication bandwidth.
A new method for conditional sampling using paired Wasserstein Autoencoders.
problem Conditional sampling from complex data distributions.
method Derive a novel loss function for Wasserstein Autoencoders to enable sampling from OT-type couplings.
result Learned cost-optimal transport maps and conditional sampling from an OT-type coupling.
The thesis presents a new perspective on high-dimensional optimization.
problem The failure point of classical optimization methods in high dimensions.
method A distributional view of optimization, focusing on random objective functions and Bayesian Optimization.
result The distributional view explains predictable progress in high-dimensional optimization and provides insights into optimal step size control.
The paper solves the problem of optimal portfolio choice when the parameters of the asset returns distribution, like the mean vector and the covariance matrix are unknown and have to be estimated by using historical data of the asset returns. The new approach employs the Bayesian posterior predictive distribution which…
New insights into noise distribution for self-supervised learning.
problem Challenges the assumption that optimal noise should match data distribution.
method Turns to Noise-Contrastive Estimation (NCE) to define optimality of noise distribution.
result Optimal noise distribution is different from data distribution, challenging GANs assumption.
Optimizes risk measures given known marginal distributions of two unknown factors.
problem Determining an upper bound for spectral risk measures with unknown joint distribution.
method Introduces Maximum Spectral Measure (MSP) as a worst-case risk measure, formulated as an optimization problem with a more general objective function.
result Characterizes the continuity properties of the optimal value function and optimal solution set with respect to marginal distributions.
Optimal transport for vector Gaussian mixtures improves efficiency and structure preservation.
problem Optimal mass transport for vector-valued Gaussian mixtures.
method Vectorizing Gaussian mixture models and studying optimal mass transport problems.
result Computational efficiency and structure preservation in optimal mass transport.
Optimizes UAV deployment for VLC-enabled UAVs considering illumination distribution.
problem Optimizing UAV deployment for VLC-enabled UAVs with illumination distribution consideration.
method Formulated as an optimization problem, solved using GRUs and Gaussian mixture model.
result Achieves up to 22.1% reduction in transmit power compared to conventional methods.
Optimal estimator for discrete distributions from faulty batches.
problem Estimating discrete distributions from batches, some of which may be unreliable.
method First polynomial-time estimator achieving optimal accuracy in number of batches.
result Optimal estimation accuracy in polynomial time.
Develops a method to efficiently compute Wasserstein barycenters with variational distributions.
problem High computational burden in computing Wasserstein barycenters for high-dimensional and continuous settings.
method Introduces a variational distribution to approximate the continuous Wasserstein barycenter, reformulating the problem as an optimization with c-cyclical monotonicity.
result The method provides a tractable dual formulation for efficient computation of Wasserstein barycenters, demonstrated on real applications.
A new distributed method for convex optimization over networks with fast convergence.
problem Large-scale convex optimization over networks with limited communication.
method Distributed cubic-regularized Newton method.
result Convergence rate of O ( k − 3 ) O(k^{{-}3}) O ( k − 3 ) for convex functions with Lipschitz gradient and Hessian. This paper examines how optimization methods affect the reliability of detecting inputs outside a model's training distribution.
problem The unreliability of deep neural networks on out-of-distribution inputs.
method Analysis of optimization methods' impact on OOD detection approaches.
result Optimization methods significantly influence the robustness of OOD detection approaches.