Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,695 papers · 148 categories

Trend · papers per month

213425638850 · Jun 202019922001200920172026
48 results for Chinchilla Approach 2

Chinchilla Approach 2 biases neural scaling law estimates, leading to unnecessary compute costs.

problem Systematic biases in Chinchilla Approach 2's parabolic fits of neural scaling laws.
method Analyzes three sources of error: IsoFLOP sampling grid width, uncentered sampling, and loss surface asymmetry.
result Chinchilla Approach 3 largely eliminates these biases, offering a more convenient or scalable alternative.

New scaling laws optimize model size, training, and inference for better performance.

problem Trade-off between model size and inference cost in modern LLMs.
method Train-to-Test (T2T^2) scaling laws that jointly optimize model size, training tokens, and inference samples.
result Optimal pretraining decisions shift into overtraining regime, leading to stronger performance.

Seesaw optimizes training by balancing learning rate and batch size, accelerating model pretraining.

problem Optimizing training efficiency for large language models with adaptive optimizers.
method Develops a principled framework for batch-size scheduling, introducing Seesaw which multiplies learning rate by 1/√2 and doubles batch size.
result Empirically, Seesaw reduces wall-clock time by approximately 36% compared to cosine decay, matching theoretical limits.

A method predicts GNS of transformer layers using normalization layer norms.

problem Estimating gradient noise scale with minimal variance.
method Simultaneously compute per-example gradient norms and parameter gradients.
result Total GNS is predicted well by normalization layer GNS.

Researchers study how skills are learned in neural networks using physics principles.

problem Understanding how skills are sequentially learned in neural networks.
method Abstract and simplify the problem using physics principles, proposing three models: Geometry, Resource, and Domino.
result Models reveal insights into neural scaling laws, learning dynamics, and the benefits of modularity.

The study introduces anytime learning schedules for large language models without fixed horizons.

problem Training large language models without knowing the total training horizon.
method Theoretical analysis and weight averaging to create anytime learning schedules.
result Theoretical and empirical evidence shows that weight averaging with simple step sizes can achieve comparable final loss to well-tuned cosine schedules.

Galactica learns from scientific literature to help researchers.

problem Information overload in scientific literature makes it hard to find useful insights.
method Trained on a large corpus of scientific papers, reference material, and knowledge bases.
result Outperforms existing models on various scientific tasks, including LaTeX equations and mathematical reasoning.

New study finds optimal hyperparameter tuning crucial for fair optimizer comparisons.

problem Inadequate hyperparameter tuning and misleading evaluation setups hinder fair comparisons of optimizers.
method Systematic study of ten optimizers across four model scales and data-to-model ratios.
result Optimal hyperparameters for one optimizer may be suboptimal for another, and many claimed speedups are lower than expected.

Geometric approach combines asset returns and investor views for better portfolio optimization.

problem Optimizing portfolios with investor-specific views.
method Generalized Wasserstein barycenter (GWB) to integrate statistical asset returns and investor views.
result The geometric approach offers more flexibility and rewards for correct investor views.

Paper proposes an alternative method to price American options using HJM approach.

problem Price American options efficiently and accurately.
method Utilizes HJM technique to model term structure of volatility for equity markets.
result Proposes a new value function, stopping criteria, and stopping time for American options.

We study inference and learning based on a sparse coding model with `spike-and-slab' prior. As in standard sparse coding, the model used assumes independent latent sources that linearly combine to generate data points. However, instead of using a standard sparse prior such as a Laplace distribution, we study the applic…

2012-11-15abs ↗pdf ↗

We develop a semi-analytic approach to the valuation of auto-callable structures with accrual features subject to barrier conditions. Our approach is based on recent studies of multi-assed binaries, present in the literature. We extend these studies to the case of time-dependent parameters. We compare numerically the s…

2016-08-18abs ↗pdf ↗

Two ML approaches learn local volatility surfaces from option prices, with GP being arbitrage-free.

problem Interpolating European vanilla option prices to create a local volatility surface.
method Gaussian process regression and neural net with arbitrage penalties.
result GP approach is arbitrage-free and yields best out-of-sample calibration error.

This paper critiques the Standardized Measurement Approach (SMA) for operational risk and recommends maintaining Advanced Measurement Approach (AMA).

problem Weaknesses and failures of the Standardized Measurement Approach (SMA) in operational risk.
method Critical review and analysis of SMA and AMA approaches.
result SMA is unstable, insensitive to risk, and implicitly related to systemic risk in the banking sector.

Two approaches extend knowledge distillation to Gaussian Processes, showing relationships to existing methods.

problem Applying knowledge distillation to Gaussian Processes for regression and classification.
method Data-centric and distribution-centric approaches to extend distillation to GPR and GPC.
result Distribution-centric approach for GPC approximately corresponds to data duplication and scaling.

We discuss the relative merits of optimistic and randomized approaches to exploration in reinforcement learning. Optimistic approaches presented in the literature apply an optimistic boost to the value estimate at each state-action pair and select actions that are greedy with respect to the resulting optimistic value f…

2017-06-13abs ↗pdf ↗

Classical approaches for approximate inference depend on cleverly designed variational distributions and bounds. Modern approaches employ amortized variational inference, which uses a neural network to approximate any posterior without leveraging the structures of the generative models. In this paper, we propose Amorti…

2019-06-06abs ↗pdf ↗

Bayesian symbolic regression automates model discovery from data.

problem Learning closed-form mathematical models from data using heuristic methods.
method Probabilistic approach to symbolic regression, connecting to information theory and statistical physics.
result Probabilistic approach provides model plausibility and performance guarantees.

New approach interprets Nyström for kernel machines with geometric insight.

problem No comparative study over Nyström-based kernel machine approaches.
method Developed a new approach with geometric interpretation, showing equivalence to existing methods.
result Proposed approach offers insights into approximation errors and accuracy.

Common Representation Learning (CRL), wherein different descriptions (or views) of the data are embedded in a common subspace, is receiving a lot of attention recently. Two popular paradigms here are Canonical Correlation Analysis (CCA) based approaches and Autoencoder (AE) based approaches. CCA based approaches learn …

2015-04-27abs ↗pdf ↗

Paper evaluates CNN-based facial landmark detection methods.

problem Evaluate characteristics and performance of CNN-based facial landmark detection methods.
method Divided into regression and heatmap approaches, investigated using a hybrid loss function and discrimination network.
result Proposed model outperforms other models in all tested datasets.

This paper provides a comprehensive benchmark and taxonomy for certifiably robust DNN defenses.

problem Certifiably robust defenses against adversarial attacks for deep neural networks.
method Taxonomy and benchmark of certifiably robust approaches.
result First comprehensive benchmark of certifiably robust approaches on different datasets.

Saliency maps are often used in computer vision to provide intuitive interpretations of what input regions a model has used to produce a specific prediction. A number of approaches to saliency map generation are available, but most require access to model parameters. This work proposes an approach for saliency map gene…

2020-01-30abs ↗pdf ↗

Study proposes a new approach for deep hedging using artificial market simulations.

problem Challenges in selecting the best model for underlying asset simulations in deep hedging.
method Proposes artificial market simulations to replicate financial market stylized facts.
result Achieves similar performance to traditional approaches without mathematical finance models.

Some of recent developments, including recent results, ideas, techniques, and approaches, in the study of degenerate partial differential equations are surveyed and analyzed. Several examples of nonlinear degenerate, even mixed, partial differential equations, are presented, which arise naturally in some longstanding, …

2010-05-15abs ↗pdf ↗

We compare several approaches to learn an Optimal Map, represented as a neural network, between probability distributions. The approaches fall into two categories: ``Heuristics'' and approaches with a more sound mathematical justification, motivated by the dual of the Kantorovitch problem. Among the algorithms we consi…

2019-08-04abs ↗pdf ↗

This paper examines how optimization methods affect the reliability of detecting inputs outside a model's training distribution.

problem The unreliability of deep neural networks on out-of-distribution inputs.
method Analysis of optimization methods' impact on OOD detection approaches.
result Optimization methods significantly influence the robustness of OOD detection approaches.

Deep Neural Networks have shown tremendous success in the area of object recognition, image classification and natural language processing. However, designing optimal Neural Network architectures that can learn and output arbitrary graphs is an ongoing research problem. The objective of this survey is to summarize and …

2019-06-07abs ↗pdf ↗

In this paper, we propose three approaches for the estimation of the Tucker decomposition of multi-way arrays (tensors) from partial observations. All approaches are formulated as convex minimization problems. Therefore, the minimum is guaranteed to be unique. The proposed approaches can automatically estimate the numb…

2010-10-05abs ↗pdf ↗

Optimal sample complexity analysis for plug-in approach in average-reward MDPs.

problem Learning optimal policies in average-reward MDPs with a generative model.
method Plug-in approach that constructs a model estimate and computes an optimal policy.
result Optimal sample complexities for the plug-in approach without prior knowledge of problem parameters.

We devise a one-shot approach to distributed sparse regression in the high-dimensional setting. The key idea is to average "debiased" or "desparsified" lasso estimators. We show the approach converges at the same rate as the lasso as long as the dataset is not split across too many machines. We also extend the approach…

2015-03-14abs ↗pdf ↗