New algorithms for convex optimization with many constraints using random projections and stochastic gradient descent.
problem Optimization problems with a large number of constraints.
method Stochastic gradient descent combined with random feasibility updates on randomly selected constraint subsets.
result The proposed algorithms converge almost surely and achieve efficient performance.
Analyze convergence of SGD with biased updates using LMI.
problem Convergence analysis of SGD with computation errors.
method Develops sequential minimization approach to analyze trade-offs.
result Obtains convergence formulas for biased SGD.
Convolutional networks improve reinforcement learning for navigating all goals.
problem Expensive parallel updates limit reinforcement learning to small tabular cases.
method Use convolutional neural networks to generate Q-values and updates for all goals simultaneously.
result Demonstrated improved accuracy and generalization on various environments.
A multilevel optimization method for constrained problems.
problem Regularized constrained linear inverse problems with box constraints.
method Geometric multilevel optimization with varying discretization levels.
result Preserves feasibility of updates while speeding up computations.
We propose two new alternating direction methods to solve "fully" nonsmooth constrained convex problems. Our algorithms have the best known worst-case iteration-complexity guarantee under mild assumptions for both the objective residual and feasibility gap. Through theoretical analysis, we show how to update all the al…
Optimizes neural network training by dynamically updating Tucker decomposition ranks.
problem Redundant parameters in neural network architectures.
method Geometry-aware training of factorized layers in tensor Tucker format.
result Optimal locally approximating the original dynamics without initial rank knowledge.
This work introduces a fixed-point optimization for variational inference.
problem Improving quantified uncertainty in predictions by optimizing a simplified distribution over parameters.
method Projective integral updates for high-dimensional variational inference.
result Efficient quasirandom quadrature sequence for mean-field distributions, leading to quasi-Newton variational Bayes (QNVB).
New algorithm for convex optimization with coupled constraints and linear constraints.
problem General multi-block convex optimization with coupled objective and linear constraints.
method Randomized primal-dual proximal block coordinate updates.
result Established O ( 1 / t ) O(1/t) O ( 1/ t ) convergence rate for objective value and feasibility measure. Proposes time-smoothed gradients for more stable online forecasting.
problem Stability and efficiency in online forecasting with SGD.
method Introduces time-smoothed gradients within SGD update rules.
result Time-smoothed gradients yield more stable results than existing methods.
A geometric method optimizes over the intersection of two manifolds.
problem Optimizing over the intersection of two manifolds with coupled geometry.
method Geometric method using retraction on one manifold and orthogonal updates.
result Convergence to first-order stationarity under intrinsic transversality.
A new optimization algorithm for Gaussian Variational Inference on precision matrices.
problem Complex models with positive definite constraints on covariance matrices.
method Manifold Gaussian Variational Bayes (MGVBP) with natural gradient updates.
result Empirically validated as a feasible and efficient solution for VI in complex models.
New algorithm optimizes MCMC sampling for structural dynamic models.
problem Time-consuming retraining of neural networks in MCMC methods.
method Adaptive meta-learning SGHMC algorithm that optimizes sampling strategy.
result Trained sampler can be applied to various problems without retraining.
FedSGM tackles constrained federated learning with unified framework.
problem Functional constraints, communication bottlenecks, local updates, and partial client participation in federated learning.
method Unified framework based on switching gradient method, incorporating bi-directional error feedback, and soft switching for stability.
result Achieves O ( 1 / T ) \boldsymbol{\mathcal{O}}(1/\sqrt{T}) O ( 1/ T ) convergence rate with high-probability bounds decoupling from sampling noise. QBVI uses natural gradients for efficient Bayesian learning.
problem Efficient Bayesian learning in complex models.
method Natural gradient updates in a black-box framework for exponential-family distributions.
result QBVI framework is effective for a wide range of Bayesian inference problems.
EGAB algorithms improve online portfolio selection.
problem Online portfolio selection problem.
method Generalized exponentiated gradient (EG) updates with Alpha-Beta divergence regularization.
result EGAB algorithms enhance portfolio performance, especially with transaction costs.
Hierarchical FL reduces latency in HCNs by sharing model updates.
problem Latency and privacy issues in federated learning across heterogeneous cellular networks.
method Hierarchical federated learning, gradient sparsification, periodic averaging.
result Significant reduction in communication latency without compromising model accuracy.
SGD updates align with a low-rank subspace but do not lead to further loss reduction.
problem Understanding the training dynamics of deep neural networks, particularly the role of the dominant subspace.
method Exploring whether neural networks can be trained within the dominant subspace of the loss Hessian.
result SGD updates, when projected onto the dominant subspace, do not decrease the training loss further, suggesting spurious alignment.
A framework for federated adversarial learning with convergence analysis.
problem Unique vulnerabilities to adversarial attacks in federated learning.
method Formulates a general federated adversarial learning framework with inner and outer loops for client-side adversarial training and server-side model aggregation.
result The minimum loss under the proposed algorithm can converge to ε with chosen learning rate and communication rounds.
Efficient CD algorithms on matrix manifolds for optimization problems.
problem Optimization on Riemannian manifolds with computational efficiency.
method Developed coordinate descent algorithms for various matrix manifolds, updating only a few variables at each iteration.
result Proposed algorithms achieve low cost per iteration and a more efficient variant via first-order approximation.
EGMU optimizes portfolios using KL divergence, ensuring positive solutions.
problem Constructing multi-factor target-exposure portfolios efficiently and accurately.
method Convex optimization framework minimizing KL divergence, with explicit solvers.
result Established feasibility and uniqueness of strictly positive solutions under convex-hull conditions.
The restricted Boltzmann machine (RBM) is a flexible tool for modeling complex data, however there have been significant computational difficulties in using RBMs to model high-dimensional multinomial observations. In natural language processing applications, words are naturally modeled by K-ary discrete distributions, …
An algorithm simplifies optimization with nonnegative and orthogonal constraints.
problem Optimization problems with nonnegative and orthogonal constraints.
method Support-set algorithm exploiting structural sparsity.
result Global convergence to first-order stationary point with iteration complexity O ( ε − 2 ) O(ε^{-2}) O ( ε − 2 ) . FLOP algorithm speeds up causal structure learning for linear models.
problem Efficiently learning causal structures from discrete data.
method FLOP algorithm combines fast parent selection and iterative score updates.
result FLOP finds highly accurate causal structures with near-perfect recovery.
Safe RL for autonomous vehicles using PCPO with trust regions and parallel learners.
problem Unexplainable behaviours and lack of safety guarantees in RL for real vehicles.
method PCPO framework with trust regions and parallel learners.
result Safe learning confirmed for autonomous vehicles with fast convergence.
USAC balances pessimism and optimism in actor-critic training for better exploration and performance.
problem Excessive pessimism limits exploration, while excessive optimism leads to high-risk behaviors.
method Utility Soft Actor-Critic (USAC) dynamically adapts exploration based on critic uncertainty.
result USAC consistently outperforms state-of-the-art algorithms in continuous control tasks.
DNN policies improve stochastic AC OPF for power grid optimization.
problem Optimizing power grid operations under uncertainty.
method Deep neural network (DNN) policies for real-time generator dispatch decisions.
result DNN policies enforce feasibility constraints and produce near optimal solutions.
The SCMU algorithm computes cone factorizations for symmetric cones, improving upon existing methods.
problem Computing cone factorizations for symmetric cones in optimization.
method Introduces and analyzes the symmetric-cone multiplicative update (SCMU) algorithm.
result The SCMU algorithm non-decreases the squared loss objective.
Novel spam filter improves e-mail classification accuracy.
problem Uneven class distribution, unequal error cost, frequent content change, personalized discrimination.
method TFDCR feature selection, incremental learning, dynamic feature update.
result TFDCR outperforms in feature selection, incremental model improves classification accuracy.
Paper addresses feasibility of counterfactual explanations in ML models, especially for critical domains.
problem Feasibility of counterfactual examples in ML models, especially in healthcare and finance.
method Uses partial structural causal models and modified variational autoencoder loss to generate counterfactuals that satisfy feasibility constraints.
result Generated counterfactuals better satisfy feasibility constraints than existing methods.
A scalable inferential method for SGD estimates.
problem Statistical inference for SGD estimates is computationally infeasible.
method Randomly perturbed SGD estimates for scalable inference.
result The proposed method provides theoretical and practical benefits.
RPO uses past and future state-action info for better policy optimization.
problem Sample inefficiency in on-policy reinforcement learning methods.
method Reflective Policy Optimization (RPO) integrates past and future state-action info for policy improvement.
result RPO improves policy performance and contracts the solution space, leading to faster convergence.
New algorithm finds best feasible arm in grouped bandits.
problem Finding the best arm with all attributes above a threshold.
method Feasibility Constrained Successive Rejects (FCSR) algorithm.
result FCSR identifies the best feasible arm with optimal dependence on problem parameters.
Bayesian search optimizes exploration of feasible solutions under expensive constraints.
problem Identifying feasible solutions in computationally expensive constraint spaces.
method Bayesian models with an acquisition function for efficient exploration and exploitation.
result The proposed acquisition function improves the prediction of feasibility.
FISAR uses neural networks to optimize safe reinforcement learning with forward-invariant constraints.
problem Safe reinforcement learning with constraints in safety-critical environments.
method Imposing linear constraints on policy parameters' updating dynamics, using a DNN-based optimizer to satisfy these constraints.
result The policy decreases constraint violation and maximizes cumulative reward monotonically.
Develops a new framework for integrating satellite allocations in small portfolios.
problem Feasibility constraints in small portfolios, not return predictability, are the primary concerns.
method A four-layer feasibility framework: physical, economic, structural, and epistemic.
result Closed-form feasibility bounds on satellite size, turnover, and breadth without return forecasts.
Square Attack efficiently attacks deep models with random updates.
problem Efficiently attacking deep learning models without gradient information.
method Randomized search of localized square updates.
result Significantly more query-efficient and higher success rate compared to state-of-the-art methods.
Unified model explains international trade patterns using reinforced urns.
problem Understanding the complex patterns of international trade networks.
method A unified modelling framework using reinforced urns and the Reinforced Urn Process.
result The model predicts power law behavior and accounts for various network properties.
AES learns feasible domains in unbounded spaces with bounded query budget.
problem Learning feasible domains in unbounded input spaces with limited query budget.
method Active Expansion Sampling (AES) progressively expands knowledge of the input space, switching between learning decision boundaries and searching for new feasible domains.
result AES has a misclassification loss guarantee within the explored region, independent of iterations or labeled samples.
Bayesian principles improve agentic AI decision-making.
problem Decision-making under uncertainty in agentic AI systems.
method Bayesian decision theory applied to the orchestration layer of agentic AI.
result Bayesian principles enhance agentic AI's ability to make decisions under uncertainty.
New algorithm solves complex medical radiation therapy problems.
problem Optimizing radiation therapy treatment plans.
method Majorization-minimization principle applied to non-linear split feasibility problems.
result Euclidean norm in proximity function replaced by Bregman divergences.
VaR-CPO optimizes VaR-constrained RL problems with conservative policy updates.
problem Optimizing VaR-constrained reinforcement learning problems.
method Combines Cantelli's inequality and trust-region framework for efficient and conservative optimization.
result Achieves zero constraint violations during training in feasible environments.
New method dynamically adjusts UTD ratio to balance under- and overfitting in RL.
problem Balancing under- and overfitting in world model learning for RL.
method Dynamic adjustment of UTD ratio based on validation performance on a small subset of experience data.
result Our method improves balance between under- and overfitting compared to default settings and competitive with extensive hyperparameter search.
Enhances OTA FL algorithms by defining inverse feasibility for linear models.
problem Improving security and privacy in over-the-air federated learning.
method Defines inverse feasibility as an upper bound on condition number, analyzes existing model, proposes new model.
result Proposes a new OTA FL model with enhanced characteristics.
A smart method predicts and optimizes decisions online with resource constraints.
problem Online decision-making with resource constraints.
method Combines prediction and optimization with dual update using mirror descent.
result Regret bounds and convergence rates for general convex feasible regions.
SFLS method finds feasible solutions faster with less data.
problem Efficiently solving SOECs with near-feasibility and near-optimality.
method SFLS method that emphasizes feasibility before convergence.
result SFLS maintains high-probability feasibility at each iteration.
Paper develops inference methods for low-rank tensors without debiasing.
problem Statistical inference for low-rank tensor models.
method Two-iteration alternating minimization for asymptotic distribution.
result Asymptotic distributions and confidence regions for singular subspaces.
We introduce a multivariate stochastic volatility model for asset returns that imposes no restrictions to the structure of the volatility matrix and treats all its elements as functions of latent stochastic processes. When the number of assets is prohibitively large, we propose a factor multivariate stochastic volatili…
New algorithm exploits curvature of feasible sets for fast online convex optimization.
problem Online convex optimization with fast rates.
method Adapting FTL algorithm to curvature of feasible sets.
result Achieves logarithmic regret bound of O ( ρ log T ) O(ρ\log T) O ( ρ log T ) in stochastic environments.