A new hybrid algorithm reduces stochastic gradient evaluations for nonconvex optimization.
problem Solving stochastic composite nonconvex optimization problems efficiently.
method Proposes a new hybrid variance-reduced proximal gradient method with a stochastic gradient estimator.
result Achieves optimal stochastic oracle complexity bound with one less gradient evaluation.
Hybrid Policy Optimization tackles reinforcement learning in hybrid spaces, improving performance over PPO.
problem Credit assignment issues and biased gradients in hybrid discrete-continuous action spaces.
method Mixed gradient estimator combining pathwise and score-function gradients, reformulating problems in hybrid form.
result HPO substantially outperforms PPO on inventory control and switched systems, with performance gaps increasing with continuous action dimension.
A hybrid strategy forecasts short-term loads using Warm-start Gradient Tree Boosting.
problem Lack of effective short-term load forecasting methods.
method Hybrid strategy integrating four different inference models: tree-based ensemble method Warm-start Gradient Tree Boosting (WGTB).
result Demonstrates effectiveness of hybrid strategy on real datasets.
New method optimizes 3D training data generation for deep networks.
problem Challenges in generating realistic 3D training data for deep networks.
method Hybrid gradient optimization of design decisions in graphics-based generation pipelines.
result Our approach outperforms prior methods in computational efficiency and performance.
This paper provides a geometrical derivation of the Hybrid Minimum Principle (HMP) for autonomous hybrid systems whose state manifolds constitute Lie groups (G,⋆) which are left invariant under the controlled dynamics of the system, and whose switching manifolds are defined as smooth embedded time invariant subma…
We introduce a hybrid stochastic estimator to design stochastic gradient algorithms for solving stochastic optimization problems. Such a hybrid estimator is a convex combination of two existing biased and unbiased estimators and leads to some useful property on its variance. We limit our consideration to a hybrid SARAH…
A new method for optimizing functions without gradients, improving efficiency and convergence.
problem Optimizing functions without gradient information in machine learning.
method Hybrid Gradient Descent (HGE) using random and coordinate-wise gradient estimates.
result The proposed method achieves optimal convergence rates in convex cases and generalizes to non-convex cases.
Hybrid RL algorithm combines offline and online data for robust and efficient policy learning.
problem Combining robust on-policy methods with efficient offline data for hybrid RL.
method Integrates off-policy training on offline data into on-policy NPG framework.
result Achieves state-of-the-art theoretical guarantees and maintains on-policy NPG guarantees.
Many structured data-fitting applications require the solution of an optimization problem involving a sum over a potentially large number of measurements. Incremental gradient algorithms offer inexpensive iterations by sampling a subset of the terms in the sum. These methods can make great progress initially, but often…
We study a hybrid conditional gradient - smoothing algorithm (HCGS) for solving composite convex optimization problems which contain several terms over a bounded set. Examples of these include regularization problems with several norms as penalties and a norm constraint. HCGS extends conditional gradient methods to cas…
New hybrid stochastic optimization framework tackles nonconvex problems efficiently.
problem Efficiently solving stochastic composite nonconvex optimization problems.
method Combining two stochastic estimators to create a hybrid one, developing several variants of stochastic gradient methods.
result Achieved best-known complexity bounds for various optimization problems.
A new method reduces the complexity of decentralized optimization.
problem Decentralized stochastic non-convex optimization over a network.
method GT-HSGD, a hybrid variance-reduced method.
result Achieves an oracle complexity of O(n^(-1)ε^(-3)) for small ε.
Boosting algorithms are frequently used in applied data science and in research. To date, the distinction between boosting with either gradient descent or second-order Newton updates is often not made in both applied and methodological research, and it is thus implicitly assumed that the difference is irrelevant. The g…
A new method for efficient computation of Knowledge Gradient in Bayesian optimization.
problem Efficient computation of the Knowledge Gradient for Bayesian optimization.
method One-shot Hybrid KG, a new approach combining previous ideas.
result The new method is cheap to compute and preserves theoretical properties of previous methods.
Study compares quantum and classical ML in crypto trading, finding hybrid models outperform.
problem Comparing quantum and classical machine learning in crypto trading strategies.
method Backtesting 10 models across multiple crypto assets using classical ML, quantum ML, hybrid models, and transformer models.
result Hybrid quantum models achieve superior performance with 13.99% return and 1.76 Sharpe ratio.
To train an inference network jointly with a deep generative topic model, making it both scalable to big corpora and fast in out-of-sample prediction, we develop Weibull hybrid autoencoding inference (WHAI) for deep latent Dirichlet allocation, which infers posterior samples via a hybrid of stochastic-gradient MCMC and…
Hybrid method improves sampling from multimodal distributions.
problem Sampling from multimodal posterior distributions efficiently.
method Jump-Diffusion Langevin Dynamics hybrid with Metropolis.
result Calibrated hybrid method outperforms pure methods.
Hybrid model for online nonlinear prediction using LSTM and soft GBDT.
problem Online nonlinear prediction with manual feature selection and model selection issues.
method End-to-end architecture with LSTM for feature extraction and soft GBDT for regression, jointly optimized.
result Significant performance improvements over conventional methods on real datasets.
EnKO combines VI and EnKF for efficient latent dynamics inference.
problem Particle degeneracy and biased gradient estimators in SMC-based methods.
method EnKO: hybrid of VI and EnKF.
result EnKO outperforms SMC-based methods in predictive ability and particle efficiency.
Paper proposes a new method to stabilize noisy gradient algorithms.
problem Stochastic-gradient Langevin algorithms can introduce bias when taming denominators depend on stochastic-gradient realizations.
method Proposes a structure-preserving framework for designing tamed denominators that avoid unnecessary taming and maintain the stabilizing effect of taming.
result The method avoids stationary bias and explains the stationary error split into bias and remaining error.
Coded computation techniques provide robustness against straggling servers in distributed computing, with the following limitations: First, they increase decoding complexity. Second, they ignore computations carried out by straggling servers; and they are typically designed to recover the full gradient, and thus, canno…
Improves BED scalability for implicit models.
problem Designing experiments for implicit models with intractable data distributions.
method Hybrid gradient approach combining variational MI estimator, ES, and SGA.
result Significantly improves scalability of BED for implicit models.
A fast method for Lasso and Logistic Lasso problems.
problem Solving Lasso and Logistic Lasso regression problems efficiently.
method Iterative active set approach using solver updates.
result 31.41 times faster on average for compressed sensing.
A new hybrid Newton algorithm improves convergence in logistic regression.
problem Solving large-scale binary classification problems efficiently.
method Proposes a hybrid stochastic Newton algorithm with two weighted components in the Hessian matrix estimation.
result Proves almost sure convergence to the true parameter of logistic regression.
Improved VI method for deep mixed models in finance.
problem Inaccurate and slow variational inference in high dimensions.
method Natural gradient hybrid VI method targeting joint posterior.
result Natural gradient method is faster and more accurate than existing methods.
Hybridizes CEM and gradient descent for efficient model-predictive control.
problem Efficiently planning optimal action sequences in high-dimensional spaces.
method Interleaves Cross-Entropy Method (CEM) and gradient descent steps.
result Faster convergence and avoidance of local optima compared to CEM.
A common practice in most of deep convolutional neural architectures is to employ fully-connected layers followed by Softmax activation to minimize cross-entropy loss for the sake of classification. Recent studies show that substitution or addition of the Softmax objective to the cost functions of support vector machin…
Hybrid framework predicts Arctic permafrost decline, risks infrastructure, and provides tools.
problem Tackles permafrost decline and infrastructure risk assessment in Arctic territories.
method Hybrid physics-machine learning framework integrating 2.9 million observations.
result Projects mean permafrost fraction decline of -20.3 pp under RCP8.5 forcing, with high-risk zones identified.
A hybrid deep learning model improves ESD diagnosis accuracy.
problem Automated diagnosis of Erythemato-Squamous Disease (ESD) is challenging.
method Proposes Derm2Vec, a hybrid model combining Autoencoders and Deep Neural Networks.
result Derm2Vec outperforms other methods in real-world dermatology dataset.
A new hybrid framework reduces quantum runtime and noise effects.
problem Challenges in deploying deep QFMs on real quantum hardware.
method Iterative Quantum Feature Maps (IQFMs) combining shallow QFMs and classical augmentation weights.
result Numerical experiments show IQFMs outperforming quantum convolutional neural networks.
Hybrid QC system for Bengali questions using smart data balancing.
problem Classifying factoid questions in Bengali.
method Two-stage approach with 1D CNN for coarse classification and SGD for fine classification.
result Effectiveness of smart data balancing technique in improving classification accuracy.
We propose a technique for increasing the efficiency of gradient-based inference and learning in Bayesian networks with multiple layers of continuous latent vari- ables. We show that, in many cases, it is possible to express such models in an auxiliary form, where continuous latent variables are conditionally determini…
In this paper, we propose a stochastic Primal-Dual Hybrid Gradient (PDHG) approach for solving a wide spectrum of regularized stochastic minimization problems, where the regularization term is composite with a linear function. It has been recognized that solving this kind of problem is challenging since the closed-form…
Hybrid model improves COVID-19 case forecasting accuracy.
problem Limited data and simplistic models for accurate prediction.
method Combining SEIR and RNN on a graph structure with local and edge features.
result Improves prediction accuracy on state-level COVID-19 data.
A hybrid method improves Convolutional Neural Networks training.
problem Training Convolutional Neural Networks efficiently and avoiding local minima.
method Combines backpropagation with evolutionary strategies.
result Improves accuracy by 0.61% on CIFAR-10 image classification.
Hybrid tensor networks improve machine learning by combining quantum and classical methods.
problem Limitations of regular tensor networks in machine learning.
method Quantum-classical hybrid tensor networks (HTN) combining tensor networks and classical neural networks.
result HTN overcomes limitations of regular tensor networks and enables deep learning training.
Improves deep neural network training and accuracy with adaptive basis approach.
problem Gap between theoretical and practical performance of deep neural networks.
method Adaptive basis viewpoint, novel initializations, hybrid optimizer.
result Dramatic increases in accuracy and convergence rate for various DNN applications.
A new hybrid-ordered SGD method reduces communication and complexity for non-convex optimization.
problem Balancing communication, computational complexity, and convergence rate in distributed non-convex optimization.
method Hybrid-ordered distributed SGD with pre-shared scalers and periodic vector communication.
result Order-wise faster convergence compared to existing methods.
A novel metric and framework for evaluating gradient norm equality in deep neural networks.
problem Evaluation of gradient norm equality in complex DNNs requires strong assumptions or complex analysis.
method Proposes a novel metric called Block Dynamical Isometry and a modularized statistical framework based on free probability.
result Gradient Norm Equality is a universal philosophy behind initialization, normalization, and network structures.
Paper proposes DCT for efficient hybrid parallel training of large recommendation models.
problem Training large recommendation models at scale with efficient communication.
method Dynamic Communication Thresholding (DCT) for both Data Parallelism and Model Parallelism.
result Reduces communication by 100x and 20x during DP and MP, respectively, improving training time by 37%.
This paper improves inverse problem solving with weakly convex regularisers and proves convergence.
problem Improving solution methods for inverse problems.
method Generalised formulation of convergent regularisation using weakly convex regularisers, and proof of convergence for primal-dual hybrid gradient method.
result Proves convergence of primal-dual hybrid gradient method for variational problems and shows improved performance with IWCNNs.
CANDI solves the gap between continuous and discrete diffusion models for text generation.
problem Underperformance of continuous diffusion models in discrete data domains.
method Introduces token identifiability and a hybrid framework (CANDI) to decouple discrete and continuous corruption.
result CANDI successfully avoids temporal dissonance, enabling continuous diffusion benefits for discrete spaces.
New optimization method combines gradient clipping and non-Euclidean smoothness.
problem Improving optimization in non-Euclidean spaces for machine learning.
method Hybrid of steepest descent and conditional gradient, incorporating weight decay.
result Achieves optimal convergence rate and demonstrates effectiveness in deep learning.
We define a hybrid between Ollvier and Bakry Emery curvature on graphs with dependence on a variable neighborhood. The hexagonal lattice is non-negatively curved under this new curvature notion. Bonnet-Myers diameter bounds and Lichnerowicz eigenvalue estimates follow from the standard arguments. We prove gradient esti…
Statistical inference methods are fundamentally important in machine learning. Most state-of-the-art inference algorithms are variants of Markov chain Monte Carlo (MCMC) or variational inference (VI). However, both methods struggle with limitations in practice: MCMC methods can be computationally demanding; VI methods …
Defines non-parabolic curves in spatial hybrid space with applications.
problem Defining and analyzing non-parabolic spatial hybrid framed curves.
method Definition and proof of existence and uniqueness theorem for non-parabolic spatial hybrid framed curves.
result Existence and uniqueness theorem for non-parabolic spatial hybrid framed curves.
Improved algorithm reduces stochastic gradient complexity for large-scale learning problems.
problem High stochastic gradient complexity for large-scale learning problems.
method Hybrid Stochastic-Deterministic Minibatch Proximal Gradient (HSDMPG) algorithm.
result Achieves nearly optimal generalization in less than a single pass over data.
Enhanced tracking control for AUVs with improved policy gradient method.
problem Trajectory tracking problem for underactuated AUVs with unknown dynamics and constrained inputs.
method Hybrid actors-critics architecture with multiple actors and critics, Pseudo Q-learning, and deterministic policy gradient.
result High-level tracking control accuracy and stable learning of AUVs.