On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,742 papers · 148 categories

Trend · papers per month

count % of corpus

5 results for “Task-scaling”

Context-Scaling versus Task-Scaling in In-Context Learningcs.LG

Transformers can scale both context and task, but MLPs can only scale task.

problem Understanding and scaling In-Context Learning in transformers.

method Simplified transformer architecture, feature map, and MLP combination.

result Simplified transformer can perform ICL and context-scaling but not task-scaling.

In-Context Learning Transformers Multi-Layer Perceptrons Context-scaling Task-scaling

2024-10-16abs ↗pdf ↗

On the Theory of Transfer Learning: The Importance of Task Diversitycs.LG

New statistical guarantees for transfer learning with diverse tasks.

problem Statistical guarantees for transfer learning across different tasks.

method Formal analysis of

t+1

tasks with shared representation.

result Sample complexity reduction for new tasks using shared representation.

transfer learning representation learning statistical guarantees task diversity sample complexity

2020-06-20abs ↗pdf ↗

Learning ReLU networks to high uniform accuracy is intractablecs.LG

Learning ReLU networks to high uniform accuracy requires exponentially many samples.

problem Achieving high uniform accuracy on ReLU networks for security-critical applications.

method Quantified the number of training samples needed for any algorithm to guarantee uniform accuracy.

result The minimal number of training samples scales exponentially with network depth and input dimension.

ReLU networks uniform accuracy training samples exponential scaling statistical learning

2022-05-26abs ↗pdf ↗

Meta-learning for mixed linear regressioncs.LG

Meta-learning helps use small data from many tasks to compensate for lack of big data.

problem How to leverage small labeled data from many tasks to improve learning when big labeled data is scarce.

method Introduced a novel spectral approach to efficiently utilize small data tasks with the help of medium data tasks.

result The total number of examples necessary with only small data tasks scales similarly as when big data tasks are available.

meta-learning mixed linear regression supervised learning small data big data

2020-02-20abs ↗pdf ↗

How Feature Learning Can Improve Neural Scaling Lawsstat.ML

Model shows feature learning can improve neural scaling laws for hard tasks.

problem Understanding and improving neural network scaling laws for various task difficulties.

method Developed a solvable model of neural scaling laws, identified three scaling regimes, and demonstrated feature learning's impact on scaling exponents.

result Feature learning can improve scaling with training time and compute for hard tasks, nearly doubling the exponent.

neural scaling laws feature learning hard tasks scaling exponents neural tangent kernel

2024-09-26abs ↗pdf ↗