Deep Optimisation (DO) combines evolutionary search with Deep Neural Networks (DNNs) in a novel way - not for optimising a learning algorithm, but for finding a solution to an optimisation problem. Deep learning has been successfully applied to classification, regression, decision and generative tasks and in this paper…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,932 papers · 148 categories
Trend · papers per month
48 results for “deep learning optimisation”
Investigates gradient descent dynamics and introduces new regularisation methods.
problem Understanding and mitigating gradient descent instabilities and interactions with smoothness regularisation.
method Derives continuous-time flows to account for discretisation drift, constructs learning rate schedules and regularisers.
result New regularisation methods improve performance in reinforcement learning.
Automated framework optimizes DNN deployment on Arm CPUs.
problem Lack of globally optimised DNN deployment across software levels.
method Reinforcement Learning search for automated design space exploration.
result Up to 4x improvement in performance and 2x reduction in memory.
Orpheus simplifies deep learning deployment on edge devices.
problem Optimizing deep learning inference on edge devices for efficiency.
method Orpheus is a new framework with a small codebase, minimal dependencies, and easy integration.
result Preliminary results show the effectiveness of Orpheus for inference optimisations.
Bayesian deep learning made practical with variational inference.
problem Impracticality of Bayesian methods in deep learning.
method Natural-gradient variational inference with techniques like batch normalisation, data augmentation, and distributed training.
result Achieves similar performance to Adam optimiser with fewer epochs, preserving Bayesian benefits.
It seems to be a pearl of conventional wisdom that parameter learning in deep sum-product networks is surprisingly fast compared to shallow mixture models. This paper examines the effects of overparameterization in sum-product networks on the speed of parameter optimisation. Using theoretical analysis and empirical exp…
A new optimisation framework for neural networks without hyperparameters.
problem Lack of explicit architectural information in optimisation frameworks.
method Extends mirror descent to account for neural architecture, transforming a Bregman divergence.
result Automatic gradient descent: a first-order optimiser without hyperparameters.
Despite recent innovations in network architectures and loss functions, training RNNs to learn long-term dependencies remains difficult due to challenges with gradient-based optimisation methods. Inspired by the success of Deep Neuroevolution in reinforcement learning (Such et al. 2017), we explore the use of gradient-…
Real world experiments are expensive, and thus it is important to reach a target in minimum number of experiments. Experimental processes often involve control variables that changes over time. Such problems can be formulated as a functional optimisation problem. We develop a novel Bayesian optimisation framework for s…
We present an efficient block-diagonal ap- proximation to the Gauss-Newton matrix for feedforward neural networks. Our result- ing algorithm is competitive against state- of-the-art first order optimisation methods, with sometimes significant improvement in optimisation performance. Unlike first-order methods, for whic…
Learning and adapting to new distributions or learning new tasks sequentially without forgetting the previously learned knowledge is a challenging phenomenon in continual learning models. Most of the conventional deep learning models are not capable of learning new tasks sequentially in one model without forgetting the…
Deep RL algorithms struggle with noisy rewards in portfolio optimisation.
problem Evaluating deep reinforcement learning for portfolio optimisation with market impact.
method Simulated data with geometric Brownian motion and market impact model; Kelly criterion as upper bound; PPO and A2C with GAE; clipping; hidden Markov model for regime changes.
result PPO and A2C with GAE perform better with noisy rewards; PPO with HMM learns different policies for regime changes.
Deep k-means learns clustering and features from unlabeled data.
problem Clustering and feature learning from unlabeled data.
method Gradient-estimator for non-differentiable k-means objective via Gumbel-Softmax reparameterisation.
result Concrete k-means model optimised for canonical k-means objective, end-to-end trainable.
Deep learning is a form of machine learning for nonlinear high dimensional pattern matching and prediction. By taking a Bayesian probabilistic perspective, we provide a number of insights into more efficient algorithms for optimisation and hyper-parameter tuning. Traditional high-dimensional data reduction techniques, …
This paper optimizes investment strategies over time using dynamic mean-variance optimization.
problem Optimizing investment strategies over time in a market with time-inconsistency issues.
method Uses game-theoretical approach to address time-inconsistency in dynamic mean-variance optimization.
result Developed a time-consistent investment strategy that performs well in both real and simulated data.
Convolutional Neural Networks (CNNs) are extremely computationally demanding, presenting a large barrier to their deployment on resource-constrained devices. Since such systems are where some of their most useful applications lie (e.g. obstacle detection for mobile robots, vision-based medical assistive technology), si…
Deep learning has been extended to a number of new domains with critical success, though some traditional orienteering problems such as the Travelling Salesman Problem (TSP) and its variants are not commonly solved using such techniques. Deep neural networks (DNNs) are a potentially promising and under-explored solutio…
New method optimizes black-box functions using generative models and Wasserstein distance.
problem Optimizing black-box functions with stochastic responses in high dimensions.
method Deep generative surrogate models and Wasserstein distance for uncertainty estimation.
result Method outperforms state-of-the-art methods in robustness to function shape and stochasticity.
A new algorithm for deep Q-learning with robustness to state transition uncertainty.
problem Model uncertainty in state transitions for non-tabular, continuous state spaces.
method Distributionally robust approach using worst-case transition ball and dualized Bellman operator with Sinkhorn distance.
result Optimal policy found through solving non-linear Bellman equation with neural network parameterization.
In statistical dialogue management, the dialogue manager learns a policy that maps a belief state to an action for the system to perform. Efficient exploration is key to successful policy optimisation. Current deep reinforcement learning methods are very promising but rely on epsilon-greedy exploration, thus subjecting…
Deep learning methods are reviewed for preserving structure in neural networks.
problem Challenges in applying deep learning, especially in preserving structure.
method Review of existing deep learning methods and new algorithmic frameworks.
result Mathematical understanding and systematic design of deep learning methods to preserve structure.
Unified theory linking Bayesian and ensemble methods in deep learning.
problem Uncertainty quantification in deep learning.
method Reformulating optimisation as convex optimisation in probability measures, studying Wasserstein gradient flows.
result Unified theory explaining success of deep ensembles over variational inference.
Paper revisits Deep Variational Information Bottleneck and proposes a new optimization approach.
problem Limitations of Deep Variational Information Bottleneck in optimizing mutual information.
method Proposes a new optimization approach by circumventing the limitation of requiring both Markov chains during optimisation.
result Shows how to optimise a lower bound for mutual information, circumventing the limitation of requiring both Markov chains.
C-ADAM is a new adaptive solver for complex nested problems.
problem Solving compositional problems involving nested expected values.
method Adaptive solver for non-linear functional nesting of expected values.
result C-ADAM converges to a stationary point in .
We introduce a new algorithm for reinforcement learning called Maximum aposteriori Policy Optimisation (MPO) based on coordinate ascent on a relative entropy objective. We show that several existing methods can directly be related to our derivation. We develop two off-policy algorithms and demonstrate that they are com…
Deep learning optimizes portfolio Sharpe ratio without forecasting returns.
problem Optimizing portfolio weights without accurate expected returns forecasts.
method Using deep learning models to directly optimize ETF portfolios based on market indices.
result Our model outperformed other algorithms over the 2011-2020 period, including financial instabilities.
The paper optimizes air conditioning setpoints using machine learning.
problem Static setpoints waste energy and increase costs.
method Deep-learning model based on RNNs for predicting future temperatures.
result RNNs outperform state-of-the-art models in prediction accuracy.
MTFL improves UA and speeds convergence in personalised DNNs on edge devices.
problem Non-IID user data harms FL convergence and global UA is not always the goal.
method Introduces non-federated BN layers into federated DNNs for personalised training.
result MTFL reduces UA rounds by up to 5x and convergence time by up to 3x.
While time series momentum is a well-studied phenomenon in finance, common strategies require the explicit definition of both a trend estimator and a position sizing rule. In this paper, we introduce Deep Momentum Networks -- a hybrid approach which injects deep learning based trading rules into the volatility scaling …
Method estimates uncertainty in CT reconstructions.
problem Lack of accurate uncertainty estimates in deep-learning CT reconstructions.
method Linearised deep image prior with conjugate Gaussian-linear model error bars and Gaussian surrogate for TV regularisation.
result Method provides superior calibration of uncertainty estimates.
A new method learns priors for Bayesian optimisation to improve performance.
problem Bayesian optimisation tasks often assume strong similarity, which is violated in many cases.
method Replace strong similarity assumption with shape similarity, learn priors for hyperparameters.
result PLeBO and prior transfer find good inputs in fewer evaluations.
The mutual information is a core statistical quantity that has applications in all areas of machine learning, whether this is in training of density models over multiple data modalities, in maximising the efficiency of noisy transmission channels, or when learning behaviour policies for exploration by artificial agents…
Graph auto-encoders improve financial clustering using news and stock data.
problem Improving clustering of financial entities using multiple data sources.
method Applying graph deep learning to a finance graph with news co-occurrence and stock price data.
result Dual data sources (news and stock price) improve clustering purity to 64% compared to 32% and 42% for single data sources.
BOiLS optimizes circuit quality using Bayesian optimization.
problem Optimizing circuits with complex search spaces.
method Adapting Bayesian optimization to logic synthesis, using Gaussian process kernels and trust-region constrained acquisitions.
result Demonstrated superior performance in sample efficiency and QoR values.
Bayesian Optimisation (BO) refers to a class of methods for global optimisation of a function which is only accessible via point evaluations. It is typically used in settings where is expensive to evaluate. A common use case for BO in machine learning is model selection, where it is not possible to analytically…
Develops framework for understanding deep learning in time series data.
problem Understanding and explaining decisions made by deep learning models in time series data.
method Uses deep neural networks to capture and explain temporal dependencies in time series data.
result Framework successfully captures and explains temporal dependencies in various synthetic and real-world datasets.
Study improves Bayesian optimisation with ensemble transfer learning.
problem Improving sample efficiency in Bayesian optimisation of expensive functions.
method Empirical analysis of ensemble-based transfer learning methods and pipeline components.
result Two components (warm start initialisation and positive weight constraint) improve transfer learning Bayesian optimisation performance.
Deep RL optimizes sensor placement in digital twins for dynamic data acquisition.
problem Limited applicability of traditional sensor placement techniques for online applications.
method Formulates sensor placement as a Markov decision process and uses deep reinforcement learning.
result Improves predictive accuracy and reliability of digital twins through adaptive sensor repositioning.
We propose an algorithm for the adaptation of the learning rate for stochastic gradient descent (SGD) that avoids the need for validation set use. The idea for the adaptiveness comes from the technique of extrapolation: to get an estimate for the error against the gradient flow which underlies SGD, we compare the resul…
MTL2L learns to adapt optimisation rules for unseen data.
problem Learners need to adapt to unseen data domains.
method Introduces MTL2L, a context-aware neural optimiser.
result MTL2L can adapt optimisation rules for unseen data.
Deep RL solves complex macroeconomic models.
problem Solving dynamic stochastic general equilibrium models with bounded rationality.
method Using deep reinforcement learning to model agents as neural networks.
result Artificially intelligent agents can solve models in all policy regimes.
In this paper we model the loss function of high-dimensional optimization problems by a Gaussian random field, or equivalently a Gaussian process. Our aim is to study gradient descent in such loss functions or energy landscapes and compare it to results obtained from real high-dimensional optimization problems such as …
AdamZ optimiser improves neural network training efficiency.
problem Challenges in optimisation like overshooting and stagnation.
method Dynamic learning rate adjustment based on overshoot and stagnation factors.
result Consistently minimises loss function, improving model performance.
Sequential Subspace Search for Functional Bayesian Optimization Incorporating Experimenter Intuitioncs.LG
Bayesian optimization guided by experimenter intuition and beliefs.
problem Finding optimal functions with experimenter's beliefs incorporated.
method Sequential Subspace Search using Gaussian Process.
result Algorithm converges in sub-linear time with finite effective dimension.
Develops a new method for risk diversification using dynamic risk measures.
problem Dynamic risk diversification in investment portfolios.
method Introduces dynamic risk contributions and a recursive optimization approach for coherent dynamic distortion risk measures.
result Dynamic risk budgeting strategies can be solved using deep learning.
Nemesyst optimizes deep learning for IoT food refrigeration systems.
problem Optimizing deep learning for large-scale IoT systems with dynamic data.
method Hybrid parallelism framework using databases and model sequentialisation.
result Demonstrated compelling performance of deep learning models in IoT food refrigeration.
Deep learning solves complex volatility equations.
problem Solving path-dependent PDEs in rough volatility.
method Interpreting PDE as BSDE, using neural network reservoir approach.
result Proved theoretical convergence for least-square regression.
RL models outperform traditional methods in certain market conditions.
problem Traditional portfolio management methods rely on accurate forecasts and do not incorporate specific investor preferences.
method Deep reinforcement learning with specific investor preferences incorporated into reward functions, realistic transaction costs modelled.
result RL models can significantly outperform traditional methods in upward trending markets, but not in sideways trending markets.