Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,657 papers · 148 categories

Trend · papers per month

4168321,2481,664 · Jun 202019922001200920172026
48 results for Learning Rate Sensitivity

The paper introduces gapped scale-sensitive dimensions to improve learning rate bounds.

problem Improving lower bounds on rates of convergence in statistical and online learning.
method Introducing and analyzing gapped scale-sensitive dimensions for function classes.
result Gapped dimensions lead to stronger lower bounds on offset Rademacher averages.

The ever-increasing number of parameters in deep neural networks poses challenges for memory-limited applications. Regularize-and-prune methods aim at meeting these challenges by sparsifying the network weights. In this context we quantify the output sensitivity to the parameters (i.e. their relevance to the network ou…

2018-10-28abs ↗pdf ↗

Arguably the biggest challenge in applying neural networks is tuning the hyperparameters, in particular the learning rate. The sensitivity to the learning rate is due to the reliance on backpropagation to train the network. In this paper we present the first application of Implicit Stochastic Gradient Descent (ISGD) to…

2018-08-07abs ↗pdf ↗

Fairness-aware learning involves designing algorithms that do not discriminate with respect to some sensitive feature (e.g., race or gender). Existing work on the problem operates under the assumption that the sensitive feature available in one's training sample is perfectly reliable. This assumption may be violated in…

2019-01-30abs ↗pdf ↗

A deep reinforcement learning method for cost-sensitive portfolio selection.

problem Non-stationary price series and complex asset correlations make feature learning hard, and practical cost constraints are not considered.
method A two-stream portfolio policy network and a cost-sensitive reward function are developed using deep reinforcement learning.
result The method achieves superior performance in profitability, cost-sensitivity, and representation abilities.

The paper develops a filtering framework for estimating hazard rates with jumps in financial and insurance applications.

problem Estimating hazard rates with unobservable change-points in financial and insurance contexts.
method Continuous-time filtering framework using progressive enlargement of filtration, stochastic differential equations, and sensitivity analysis.
result Explicit formula for survival probability conditional on partial information.

This article presents valuation of Treasury Bonds (T-Bonds) on Macedonian Stock Exchange (MSE) and empirical test of duration, modified duration and convexity of the T-bonds at MSE in order to determine sensitivity of bonds prices on interest rate changes. The main goal of this study is to determine how standard valuat…

2012-06-29abs ↗pdf ↗

Hyperparameter tuning is a bothersome step in the training of deep learning models. One of the most sensitive hyperparameters is the learning rate of the gradient descent. We present the 'All Learning Rates At Once' (Alrao) optimization method for neural networks: each unit or feature in the network gets its own learni…

2018-10-02abs ↗pdf ↗

New approach detects sensitive info in text, outperforming previous methods.

problem Detecting sensitive information in unstructured text documents.
method Developed novel recursive neural network approaches for sensitive info detection, assuming only labeled examples.
result Our approaches significantly outperform previous keyword-based methods on real-world data.

In binary classification framework, we are interested in making cost sensitive label predictions in the presence of uniform/symmetric label noise. We first observe that 00-11 Bayes classifiers are not (uniform) noise robust in cost sensitive setting. To circumvent this impossibility result, we present two schemes; un…

2019-01-08abs ↗pdf ↗

The paper develops a method to achieve fairness in predictions using Wasserstein barycenters.

problem Learning a fair real-valued function independent of sensitive attributes.
method Establishing a connection between fair regression and optimal transport theory, deriving a close form expression for the optimal fair predictor as the Wasserstein barycenter of sensitive groups.
result The optimal fair predictor's distribution is the Wasserstein barycenter of sensitive groups' distributions, offering an intuitive interpretation and a simple post-processing algorithm.

Worst-Case Sensitivity measures model sensitivity to uncertainty set size.

problem Model sensitivity to uncertainty set size in Distributionally Robust Optimization.
method Introducing Worst-Case Sensitivity as a measure of model sensitivity, and deriving closed-form expressions for various uncertainty sets.
result DRO solutions can be sensitive to the family and size of the uncertainty set, and worst-case sensitivity reflects these properties.

Noise Sensitivity Exponent controls statistical-computational gaps in learning.

problem Understanding when learning is statistically possible yet computationally hard in high-dimensional statistics.
method Investigating statistical-computational gaps in single- and multi-index models using Noise Sensitivity Exponent.
result Noise Sensitivity Exponent governs statistical-computational gaps in high-dimensional learning.

GOLS-I automatically determines learning rates for various neural network training algorithms.

problem Adapting learning rates in stochastic training algorithms for neural networks.
method Gradient-Only Line Search (GOLS-I) for automatically setting learning rates.
result GOLS-I learning rate schedules are competitive with manually tuned rates across multiple algorithms, architectures, datasets, and loss functions.

The paper optimizes risk-sensitive RL with CVaR, achieving near-minimax-optimal results.

problem Optimizing risk-sensitive reinforcement learning with CVaR objective.
method Developed algorithms for multi-arm bandits and online RL in MDPs, achieving near-minimax-optimal regret.
result Achieved near-minimax-optimal regret of O(τ1SAK)O(τ^{-1}\sqrt{SAK}) for constant ττ.

Learning rate needs to decrease with higher data moments for effective ICA in high dimensions.

problem Slower convergence of ICA in high-dimensional data with high-order moments.
method High-dimensional ODE analysis of ICA algorithm under controlled moment structure.
result Critical learning rate threshold for effective ICA when moments are high.

The paper tackles fair classification with multiple sensitive features.

problem Existing fair classification methods often consider a single sensitive feature, but in practice, individuals are defined by multiple sensitive features.
method Characterizes Bayes-optimal fair classifiers for multiple sensitive features under various fairness measures, proposing in-processing and post-processing algorithms.
result Bayes-optimal fair classifiers for multiple sensitive features are instance-dependent thresholding rules that rely on a weighted sum of group membership probabilities.

Risk management in financial derivative markets requires inevitably the calculation of the different price sensitivities. The literature contains an abundant amount of research works that have studied the computation of these important values. Most of these works consider the well-known Black and Scholes model where th…

2017-05-06abs ↗pdf ↗

Gradient-based meta-learning methods leverage gradient descent to learn the commonalities among various tasks. While previous such methods have been successful in meta-learning tasks, they resort to simple gradient descent during meta-testing. Our primary contribution is the {\em MT-net}, which enables the meta-learner…

2018-01-17abs ↗pdf ↗

Classifiers trained using conventional empirical risk minimization or maximum likelihood methods are known to suffer dramatic performance degradations when tested over examples adversarially selected based on knowledge of the classifier's decision rule. Due to the prominence of Artificial Neural Networks (ANNs) as clas…

2018-02-22abs ↗pdf ↗

New adaptive scheduler improves SAM for better model training.

problem Training machine learning models requires selecting a learning rate, which is often difficult and time-consuming.
method Derive Polyak schedulers tailored to SAM-style updates, proving linear convergence for strongly convex objectives and an O(1/T) rate for convex objectives.
result Polyak schedulers achieve comparable or better performance than tuned SAM baselines, reducing the need for learning-rate tuning.

Algorithm corrects bias in classification data.

problem Underrepresentation and intersectional bias in classification data.
method Estimate group-wise drop-out rates with small unbiased data, construct reweighting scheme, and present algorithm.
result Efficiently approximate loss of any hypothesis on true distribution.

Momentum is a widely used technique for gradient-based optimizers in deep learning. In this paper, we propose a decaying momentum (\textsc{Demon}) rule. We conduct the first large-scale empirical analysis of momentum decay methods for modern neural network optimization, in addition to the most popular learning rate dec…

2019-10-11abs ↗pdf ↗

This work tackles risk-sensitive deep RL by optimizing policies with variance constraints.

problem Risk and aleatoric uncertainty in deep reinforcement learning.
method Lagrangian and Fenchel dualities to transform the problem into an unconstrained saddle-point policy optimization problem, and an actor-critic algorithm to iteratively update policy, Lagrange multiplier, and Fenchel dual variable.
result The proposed actor-critic algorithm finds a globally optimal policy at a sublinear rate.

In this paper incomplete-information models are developed for the pricing of securities in a stochastic interest rate setting. In particular we consider credit-risky assets that may include random recovery upon default. The market filtration is generated by a collection of information processes associated with economic…

2010-01-20abs ↗pdf ↗

The family of temporal difference (TD) methods span a spectrum from computationally frugal linear methods like TD(λ) to data efficient least squares methods. Least square methods make the best use of available data directly computing the TD solution and thus do not require tuning a typically highly sensitive learning r…

2016-11-28abs ↗pdf ↗

Study examines how liquidity constraints impact optimal retirement decisions.

problem Impact of liquidity constraints on optimal retirement decisions.
method Analytical solution using duality method with different liquidity constraints.
result Sensitivity analysis reveals the effect of liquidity constraints on retirement decisions.

This work adapts RDT for mental program construction, showing benefits and costs.

problem Applying RDT to mental programs with trade-offs between description length, error, and computational costs.
method Proposed a three-way trade-off and used simulations and partial information decomposition.
result Constructing a shared program library provides global benefits but is sensitive to curricula.

slimTrain simplifies DNN training by separating features and adapting hyperparameters.

problem Challenges in training deep neural networks, including non-convexity, non-smoothness, and hyperparameter sensitivity.
method slimTrain exploits separability in DNN architectures to reduce hyperparameter sensitivity and improve convergence.
result slimTrain outperforms existing methods with recommended hyperparameters and reduces sensitivity to remaining hyperparameters.

Optimizer memory affects learning rate sensitivity in shuffle order, impacting fine-tuning noise.

problem Optimizer memory affects the learning rate sensitivity in shuffle order, leading to fine-tuning noise.
method Isolated the mechanism of fixed-clock optimizer memory affecting the learning rate sensitivity in shuffle order, deriving a fit-free way to size the noise.
result Fixed-clock optimizers like AdamW produce a larger first-order noise channel compared to memoryless optimizers, affecting fine-tuning comparisons.