Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,695 papers · 148 categories

Trend · papers per month

316394125 · Jun 202019922001200920172026
48 results for Gaussian-like tuning

Minimalistic model captures head direction system properties.

problem Representing head direction system in a high-dimensional space.
method A minimalistic representation model of the rotation group U(1), including fully connected and convolutional versions.
result Emergence of Gaussian-like tuning profiles and 2D circle geometry in both model versions.

We use matricial free energy to regularize autoencoders, producing Gaussian-like codes.

problem Generating Gaussian-like codes for autoencoders.
method Define a differentiable loss function based on singular values of the code matrix, minimizing matricial free energy.
result Minimizing matricial free energy results in Gaussian-like codes that generalize.

Gaussian process regression loses locality in high dimensions, affecting molecular energy surface fitting.

problem Loss of locality in high-dimensional Gaussian process regression.
method Analysis of Matern family kernels and multi-zeta basis functions.
result The property of locality disappears in high dimensions, impacting regression quality.

Gaussian random vectors exhibit the loss of dimension phenomena, which relate to their joint survival tail behaviour. Besides, the fact that the components of such vectors are light-tailed complicates the approximations of various multivariate risk measures significantly. In this contribution we derive precise approxim…

2018-03-14abs ↗pdf ↗

We introduce a class of non-commutative Heisenberg like infinite dimensional Lie groups based on an abstract Wiener space. The Ricci curvature tensor for these groups is computed and shown to be bounded. Brownian motion and the corresponding heat kernel measures, {νt}t>0,\{ν_t\}_{t>0}, are also studied. We show that these he…

2008-05-12abs ↗pdf ↗

We propose a new method of measuring the third and fourth moments of return distribution based on quadratic variation method when the return process is assumed to have zero drift. The realized third and fourth moments variations computed from high frequency return series are good approximations to corresponding actual …

2013-11-20abs ↗pdf ↗

Leaky ReLU activations improve the calibration of Bayesian neural networks.

problem Bayesian neural networks struggle with mean-field variational inference for ReLU activations.
method Investigated the effect of activation functions on the calibration of Bayesian neural networks.
result Leaky ReLU activations lead to more Gaussian-like weight posteriors and lower expected calibration error.

The third moment variation of a financial asset return process is defined by the quadratic covariation between the return and square return processes. The skew and fat tail risk of an underlying asset can be hedged using a third moment variation swap under which a predetermined fixed leg and the floating leg of the rea…

2019-08-14abs ↗pdf ↗

Paper improves few-shot classification accuracy using feature distribution preprocessing.

problem Challenges of few-shot classification due to limited labelled samples.
method Proposes a novel transfer-based method that preprocesses feature vectors to Gaussian-like distributions and uses optimal-transport inspired algorithms.
result Achieves state-of-the-art accuracy on standardized vision benchmarks.

StatQAT optimizes quantization for deep networks, reducing computational cost and memory usage.

problem Optimal quantization parameters selection for deep neural networks with diverse data distributions.
method Statistical error analysis framework for uniform and floating-point quantization, iterative and analytic quantizers designed for arbitrary and Gaussian-like distributions.
result Improved accuracy and stability in training low-precision neural networks.

Sharp bounds for Dirichlet sums lead to improved Bayesian algorithm analysis.

problem Improving Bayesian algorithm performance through precise deviation bounds.
method Novel integral representation of Dirichlet sum density, Gaussian approximation, complex analysis.
result Significantly sharpened regret bounds for Multinomial Thompson Sampling.

Enhances hashing for fast retrieval with correlated bits.

problem Fast retrieval and small memory footprint for large-scale information retrieval.
method Employing Boltzmann machine distribution as variational posterior to model correlations among hash code bits.
result Significant performance gains achieved by effectively modeling correlations among hash code bits.

New method reduces fine-tuning cost for reused models.

problem Repeating fine-tuning costs with outdated foundation models.
method Portable Reward Tuning (PRT) trains a reward model to maximize the same loss function as fine-tuning.
result PRT achieves comparable accuracy to inference-time tuning with less inference cost.

Many applications concern sparse signals, for example, detecting anomalies from the differences between consecutive images taken by surveillance cameras. This paper focuses on the problem of recovering a K-sparse signal x in N dimensions. In the mainstream framework of compressed sensing (CS), the vector x is recovered…

2013-02-04abs ↗pdf ↗

New method quantifies uncertainty in fine-tuned LLMs using LoRA ensembles.

problem Uncertainty in fine-tuned LLMs and how to trust their predictions.
method Posterior approximations using low-rank adaptation ensembles.
result Unexpected retention of acquired knowledge during fine-tuning in overfitting regime.

BERT fine-tuning is unstable due to optimization issues, not forgetting or dataset size.

problem Stability of fine-tuning BERT-based models across different random seeds.
method Analysis of BERT, RoBERTa, and ALBERT fine-tuned on GLUE datasets, identifying optimization difficulties as the cause of instability.
result Fine-tuning instability is due to optimization difficulties leading to vanishing gradients, not forgetting or dataset size.

ADAPT improves robustness of Vision Transformers without full model fine-tuning.

problem Vulnerability of Vision Transformers to adversarial attacks.
method Parameter-efficient prompt tuning with ADAPT framework for adaptive adversarial training.
result ADAPT achieves robust accuracy of ~40% w.r.t. SOTA methods using only ~1% of the parameters.

Paper proposes a reinforcement learning framework for efficient hyper-parameter tuning of stochastic optimization algorithms.

problem Efficient tuning of hyper-parameters for stochastic optimization algorithms.
method Modeling hyper-parameter tuning as a Markov decision process and using policy gradient algorithms.
result The proposed framework significantly reduces the time required for hyper-parameter tuning compared to Bayesian optimization.

Improved code translation by preserving structure with composed fine-tuning.

problem Improving code translation accuracy with unlabeled code outputs.
method Pre-trained denoiser to capture output structure, composed fine-tuning to fine-tune predictor.
result Composed fine-tuning significantly improves generalization over standard fine-tuning.

This paper studies node embeddings of networks, revealing their geometric properties.

problem Understanding the geometric properties of node embeddings in random networks.
method Characterization of ergodic limits, generalization, and convex relaxations of random walk node embedding objectives.
result The optimal node embedding Grammians have rank 1 for a nuclear norm relaxation of the non-randomized objective.

The paper introduces a Hessian-based method to improve generalization in fine-tuned deep neural networks.

problem Improving generalization in fine-tuned deep neural networks, especially in noisy conditions.
method PAC-Bayesian analysis to identify a Hessian-based distance measure, proving generalization bounds, and developing an algorithm with a generalization error guarantee.
result Hessian-based distance measure correlates well with observed generalization gaps and can match the scale of these gaps in practice.

A new robust prefix-tuning framework improves model robustness against adversarial attacks.

problem Lack of robustness in prefix-tuning for adversarial attacks.
method Leveraging layerwise activations of pretrained models for additional prefix finetuning during the test phase.
result Framework substantially improves robustness over strong baselines while maintaining comparable accuracy on clean texts.

Optimizes sparse fine-tuning for privacy in neural networks.

problem Performance gap between DP-SGD and non-private fine-tuning.
method Optimization-based approach using private gradient information for selecting trainable weights.
result Our selection method leads to better prediction accuracy compared to existing approaches.

Single-head transformers with a single self-attention layer can approximate any sequence-to-sequence function and are efficient under certain conditions.

problem Statistical and computational limits of prompt tuning for transformer-based models.
method Investigation of single-head transformers with a single self-attention layer, proving universality and efficiency under SETH.
result Existence of almost-linear time prompt tuning inference algorithms under certain conditions.

Mango automates hyperparameter tuning for large-scale ML training.

problem Manual hyperparameter tuning is tedious and inefficient for large-scale machine learning.
method Parallel hyperparameter tuning with intelligent search strategies and flexible abstractions.
result Mango achieves comparable performance to Hyperopt while supporting distributed computing.

Fine-tuning harms in-context learning, but restricting updates to the value matrix improves zero-shot performance.

problem Fine-tuning harms in-context learning, reducing zero-shot performance on unseen tasks.
method Theoretical analysis of linear attention models, identifying conditions for degraded few-shot performance.
result Restricting updates to the value matrix improves zero-shot performance while preserving in-context learning.

Fine-tuning normalization layers can reconstruct smaller networks.

problem Understanding the expressive power of fine-tuning normalization layers.
method Random ReLU networks and sparsified networks were fine-tuned to reconstruct target networks.
result Fine-tuning normalization layers can reconstruct networks that are O(extwidth)O(\sqrt{ ext{width}}) times smaller.

InfoPrompt improves soft prompt tuning by maximizing mutual information, leading to better performance.

problem High sensitivity of prompt tuning to initial conditions and insufficient task-relevant information.
method Develops an information-theoretic framework to maximize mutual information between prompts and model parameters, using novel loss functions.
result InfoPrompt accelerates convergence and outperforms traditional methods.

Method constructs finance LLMs without instruction data using pretraining and model merging.

problem Developing domain-specific LLMs for finance is resource-intensive.
method Continual pretraining on financial data + model merging of instruction-tuned and domain-specific pretrained vectors.
result Successfully constructs instruction-tuned LLMs for finance without additional instruction data.

The paper develops a theory linking pretraining and fine-tuning in neural networks.

problem Understanding how initialization choices impact feature learning and generalization in neural networks.
method Analytical theory of diagonal linear networks, deriving generalization error as a function of initialization parameters and task statistics.
result Different initialization choices place networks into four fine-tuning regimes with varying abilities to support feature learning and generalization.

Compact models match or exceed GPT's performance in financial news sentiment analysis.

problem Improving financial sentiment analysis models without large computational costs.
method Fine-tuning non-generative, small-sized models (FinBERT, FinDRoBERTa) on a novel market score database.
result Fine-tuned models outperform GPT-3.5 and GPT-4 in zero-shot learning for financial news sentiment analysis.